2024-03-04T08:35:30.266 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-03-04T08:35:30.266 DEBUG:teuthology.run:Teuthology command: teuthology --name leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi --owner scheduled_leonidus@teuthology --verbose --archive /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580801 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} -- /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580801/orig.config.yaml 2024-03-04T08:35:30.321 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T08:35:30.406 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580801 branch: wip-lusov-qdb description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7580801' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-qdb ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 1 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-qdb sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd owner: scheduled_leonidus@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5072 sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd sleep_before_teardown: 0 subset: 246/10000 suite: fs suite_branch: wip-lusov-qdb suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi071.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCI5lumJLIOBBoKRV3PpLAq4k8pwI8rhFJ/C2W/mXVB3ua6xd9G7pRL0udCko2yzmWCw2WI+tQleJSsocFekltY= smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFknlpOafaoCboo/OlEYCdIhARf48YrundikKW1zxB1eRIU4QbDPun+9iC6y9xn6M3+pcY/ZAbfM7VRiZNB+qxU= smithi158.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN6+WV/yctKuR5BmVuEDWPCoyY2hpOnkMo28inYRwRWW3TNevpZqlDJJtGiOPmDuU0XLU2e1xG4X1ZWGEVx5FBs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-03-04_07:05:03 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-04T08:35:30.406 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa; will attempt to use it 2024-03-04T08:35:30.407 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks 2024-03-04T08:35:30.407 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-04T08:35:30.409 INFO:teuthology.task.internal:Checking packages... 2024-03-04T08:35:30.434 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd' 2024-03-04T08:35:30.434 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-04T08:35:30.434 INFO:teuthology.packaging:ref: None 2024-03-04T08:35:30.434 INFO:teuthology.packaging:tag: None 2024-03-04T08:35:30.434 INFO:teuthology.packaging:branch: wip-lusov-qdb 2024-03-04T08:35:30.434 INFO:teuthology.packaging:sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T08:35:30.434 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-qdb 2024-03-04T08:35:30.660 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1739.g1bc250c7 2024-03-04T08:35:30.662 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-04T08:35:30.668 INFO:teuthology.task.internal:no buildpackages task found 2024-03-04T08:35:30.668 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-04T08:35:30.712 INFO:teuthology.task.internal:Saving configuration 2024-03-04T08:35:30.736 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-04T08:35:30.740 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-04T08:35:30.768 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580801', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 08:28:25.867081', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCI5lumJLIOBBoKRV3PpLAq4k8pwI8rhFJ/C2W/mXVB3ua6xd9G7pRL0udCko2yzmWCw2WI+tQleJSsocFekltY='} 2024-03-04T08:35:30.796 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580801', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 08:28:25.869693', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFknlpOafaoCboo/OlEYCdIhARf48YrundikKW1zxB1eRIU4QbDPun+9iC6y9xn6M3+pcY/ZAbfM7VRiZNB+qxU='} 2024-03-04T08:35:30.820 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi158.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580801', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 08:28:25.868499', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN6+WV/yctKuR5BmVuEDWPCoyY2hpOnkMo28inYRwRWW3TNevpZqlDJJtGiOPmDuU0XLU2e1xG4X1ZWGEVx5FBs='} 2024-03-04T08:35:30.821 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-04T08:35:30.827 INFO:teuthology.task.internal:roles: ubuntu@smithi071.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-04T08:35:30.827 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-04T08:35:30.827 INFO:teuthology.task.internal:roles: ubuntu@smithi158.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-04T08:35:30.827 INFO:teuthology.run_tasks:Running task console_log... 2024-03-04T08:35:30.935 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7effc783a0d0>, signals=[15]) 2024-03-04T08:35:30.935 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-04T08:35:30.941 INFO:teuthology.task.internal:Opening connections... 2024-03-04T08:35:30.941 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi071.front.sepia.ceph.com 2024-03-04T08:35:30.943 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T08:35:31.043 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2024-03-04T08:35:31.044 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T08:35:31.138 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi158.front.sepia.ceph.com 2024-03-04T08:35:31.139 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T08:35:31.251 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-04T08:35:31.258 DEBUG:teuthology.orchestra.run.smithi071:> uname -m 2024-03-04T08:35:31.276 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2024-03-04T08:35:31.276 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2024-03-04T08:35:31.333 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="CentOS Stream" 2024-03-04T08:35:31.333 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="9" 2024-03-04T08:35:31.334 INFO:teuthology.orchestra.run.smithi071.stdout:ID="centos" 2024-03-04T08:35:31.334 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE="rhel fedora" 2024-03-04T08:35:31.334 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="9" 2024-03-04T08:35:31.334 INFO:teuthology.orchestra.run.smithi071.stdout:PLATFORM_ID="platform:el9" 2024-03-04T08:35:31.334 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-04T08:35:31.334 INFO:teuthology.orchestra.run.smithi071.stdout:ANSI_COLOR="0;31" 2024-03-04T08:35:31.334 INFO:teuthology.orchestra.run.smithi071.stdout:LOGO="fedora-logo-icon" 2024-03-04T08:35:31.334 INFO:teuthology.orchestra.run.smithi071.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-04T08:35:31.334 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://centos.org/" 2024-03-04T08:35:31.334 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-04T08:35:31.334 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-04T08:35:31.334 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-04T08:35:31.335 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2024-03-04T08:35:31.367 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2024-03-04T08:35:31.388 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2024-03-04T08:35:31.389 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-03-04T08:35:31.459 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Stream" 2024-03-04T08:35:31.459 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="9" 2024-03-04T08:35:31.459 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos" 2024-03-04T08:35:31.460 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora" 2024-03-04T08:35:31.460 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="9" 2024-03-04T08:35:31.460 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el9" 2024-03-04T08:35:31.460 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-04T08:35:31.460 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2024-03-04T08:35:31.460 INFO:teuthology.orchestra.run.smithi150.stdout:LOGO="fedora-logo-icon" 2024-03-04T08:35:31.460 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-04T08:35:31.460 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/" 2024-03-04T08:35:31.460 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-04T08:35:31.460 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-04T08:35:31.460 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-04T08:35:31.460 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-03-04T08:35:31.483 DEBUG:teuthology.orchestra.run.smithi158:> uname -m 2024-03-04T08:35:31.501 INFO:teuthology.orchestra.run.smithi158.stdout:x86_64 2024-03-04T08:35:31.501 DEBUG:teuthology.orchestra.run.smithi158:> cat /etc/os-release 2024-03-04T08:35:31.556 INFO:teuthology.orchestra.run.smithi158.stdout:NAME="CentOS Stream" 2024-03-04T08:35:31.556 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION="9" 2024-03-04T08:35:31.556 INFO:teuthology.orchestra.run.smithi158.stdout:ID="centos" 2024-03-04T08:35:31.556 INFO:teuthology.orchestra.run.smithi158.stdout:ID_LIKE="rhel fedora" 2024-03-04T08:35:31.556 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION_ID="9" 2024-03-04T08:35:31.556 INFO:teuthology.orchestra.run.smithi158.stdout:PLATFORM_ID="platform:el9" 2024-03-04T08:35:31.557 INFO:teuthology.orchestra.run.smithi158.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-04T08:35:31.557 INFO:teuthology.orchestra.run.smithi158.stdout:ANSI_COLOR="0;31" 2024-03-04T08:35:31.557 INFO:teuthology.orchestra.run.smithi158.stdout:LOGO="fedora-logo-icon" 2024-03-04T08:35:31.557 INFO:teuthology.orchestra.run.smithi158.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-04T08:35:31.557 INFO:teuthology.orchestra.run.smithi158.stdout:HOME_URL="https://centos.org/" 2024-03-04T08:35:31.557 INFO:teuthology.orchestra.run.smithi158.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-04T08:35:31.557 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-04T08:35:31.557 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-04T08:35:31.557 INFO:teuthology.lock.ops:Updating smithi158.front.sepia.ceph.com on lock server 2024-03-04T08:35:31.583 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-04T08:35:31.593 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-04T08:35:31.628 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-04T08:35:31.628 DEBUG:teuthology.orchestra.run.smithi071:> test '!' -e /home/ubuntu/cephtest 2024-03-04T08:35:31.631 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2024-03-04T08:35:31.633 DEBUG:teuthology.orchestra.run.smithi158:> test '!' -e /home/ubuntu/cephtest 2024-03-04T08:35:31.650 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-04T08:35:31.656 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-04T08:35:31.656 DEBUG:teuthology.orchestra.run.smithi071:> test -z $(ls -A /var/lib/ceph) 2024-03-04T08:35:31.689 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2024-03-04T08:35:31.694 DEBUG:teuthology.orchestra.run.smithi158:> test -z $(ls -A /var/lib/ceph) 2024-03-04T08:35:31.782 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-04T08:35:31.867 INFO:teuthology.run_tasks:Running task kernel... 2024-03-04T08:35:31.894 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-04T08:35:31.894 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-03-04T08:35:31.895 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-03-04T08:35:31.895 DEBUG:teuthology.orchestra.run.smithi071:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-04T08:35:31.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T08:35:31.913 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-03-04T08:35:31.971 INFO:teuthology.orchestra.run.smithi071.stdout:5.14.0-361.el9.x86_64 2024-03-04T08:35:31.971 INFO:teuthology.task.kernel:Running kernel on smithi071: 5.14.0-361.el9.x86_64 2024-03-04T08:35:31.971 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum install -y kernel 2024-03-04T08:35:34.104 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-04T08:35:35.780 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.0 MB 00:00 2024-03-04T08:35:38.154 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.0 MB 00:02 2024-03-04T08:35:44.571 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - AppStream 3.6 MB/s | 19 MB 00:05 2024-03-04T08:35:48.621 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.1 MB 00:00 2024-03-04T08:35:50.230 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 15 kB 00:00 2024-03-04T08:35:51.015 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 31 MB/s | 20 MB 00:00 2024-03-04T08:35:55.915 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-03-04T08:35:58.189 INFO:teuthology.orchestra.run.smithi071.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-04T08:35:58.189 INFO:teuthology.orchestra.run.smithi071.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-04T08:35:58.283 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-03-04T08:35:58.290 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-03-04T08:35:58.290 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-03-04T08:35:58.290 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-03-04T08:35:58.290 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-03-04T08:35:58.290 INFO:teuthology.orchestra.run.smithi071.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-04T08:35:58.290 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-03-04T08:35:58.290 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-04T08:35:58.290 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-04T08:35:58.291 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-04T08:35:58.291 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:35:58.291 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-03-04T08:35:58.291 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-03-04T08:35:58.291 INFO:teuthology.orchestra.run.smithi071.stdout:Install 4 Packages 2024-03-04T08:35:58.291 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:35:58.291 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 100 M 2024-03-04T08:35:58.292 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 123 M 2024-03-04T08:35:58.292 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-03-04T08:35:58.600 INFO:teuthology.orchestra.run.smithi071.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 22 MB/s | 6.0 MB 00:00 2024-03-04T08:35:59.475 INFO:teuthology.orchestra.run.smithi071.stdout:(2/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 34 MB/s | 39 MB 00:01 2024-03-04T08:36:00.058 INFO:teuthology.orchestra.run.smithi071.stdout:(3/4): kernel-modules-core-5.14.0-427.el9.x86_6 23 MB/s | 33 MB 00:01 2024-03-04T08:36:00.267 INFO:teuthology.orchestra.run.smithi071.stdout:(4/4): kernel-core-5.14.0-427.el9.x86_64.rpm 11 MB/s | 21 MB 00:01 2024-03-04T08:36:00.267 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-03-04T08:36:00.267 INFO:teuthology.orchestra.run.smithi071.stdout:Total 50 MB/s | 100 MB 00:01 2024-03-04T08:36:01.153 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-03-04T08:36:01.522 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-03-04T08:36:01.522 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-03-04T08:36:03.171 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-03-04T08:36:03.171 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-03-04T08:36:05.887 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-03-04T08:36:06.552 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-04T08:36:06.686 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-04T08:36:09.439 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-04T08:36:09.642 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-04T08:36:14.374 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-04T08:36:14.510 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-04T08:36:19.278 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-04T08:36:48.688 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-04T08:36:48.688 INFO:teuthology.orchestra.run.smithi071.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-04T08:36:48.688 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:36:48.770 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-04T08:36:49.247 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-04T08:36:49.247 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-04T08:36:49.247 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-04T08:36:49.247 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-04T08:36:49.661 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-04T08:36:49.661 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:36:49.661 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-03-04T08:36:49.662 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-04T08:36:49.662 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-04T08:36:49.662 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-04T08:36:49.662 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-04T08:36:49.662 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:36:49.662 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-03-04T08:36:50.211 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q kernel | sort -rV | head -n 1 2024-03-04T08:36:50.355 INFO:teuthology.orchestra.run.smithi071.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-04T08:36:50.355 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-04T08:36:50.355 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-04T08:36:50.355 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-04T08:36:50.355 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-04T08:36:50.355 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-04T08:36:50.356 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi071.front.sepia.ceph.com, path=None, version=distro) 2024-03-04T08:36:50.356 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q kernel | sort -rV | head -n 1 2024-03-04T08:36:50.476 INFO:teuthology.orchestra.run.smithi071.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-04T08:36:50.476 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-04T08:36:50.476 DEBUG:teuthology.orchestra.run.smithi071:> sudo rpm -qi grub2-tools 2024-03-04T08:36:50.510 INFO:teuthology.orchestra.run.smithi071.stdout:Name : grub2-tools 2024-03-04T08:36:50.510 INFO:teuthology.orchestra.run.smithi071.stdout:Epoch : 1 2024-03-04T08:36:50.510 INFO:teuthology.orchestra.run.smithi071.stdout:Version : 2.06 2024-03-04T08:36:50.510 INFO:teuthology.orchestra.run.smithi071.stdout:Release : 68.el9 2024-03-04T08:36:50.510 INFO:teuthology.orchestra.run.smithi071.stdout:Architecture: x86_64 2024-03-04T08:36:50.510 INFO:teuthology.orchestra.run.smithi071.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-04T08:36:50.510 INFO:teuthology.orchestra.run.smithi071.stdout:Group : Unspecified 2024-03-04T08:36:50.510 INFO:teuthology.orchestra.run.smithi071.stdout:Size : 8264254 2024-03-04T08:36:50.510 INFO:teuthology.orchestra.run.smithi071.stdout:License : GPLv3+ 2024-03-04T08:36:50.510 INFO:teuthology.orchestra.run.smithi071.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-04T08:36:50.511 INFO:teuthology.orchestra.run.smithi071.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-04T08:36:50.511 INFO:teuthology.orchestra.run.smithi071.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-04T08:36:50.511 INFO:teuthology.orchestra.run.smithi071.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-04T08:36:50.511 INFO:teuthology.orchestra.run.smithi071.stdout:Packager : builder@centos.org 2024-03-04T08:36:50.511 INFO:teuthology.orchestra.run.smithi071.stdout:Vendor : CentOS 2024-03-04T08:36:50.511 INFO:teuthology.orchestra.run.smithi071.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-04T08:36:50.511 INFO:teuthology.orchestra.run.smithi071.stdout:Summary : Support tools for GRUB. 2024-03-04T08:36:50.511 INFO:teuthology.orchestra.run.smithi071.stdout:Description : 2024-03-04T08:36:50.511 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:36:50.511 INFO:teuthology.orchestra.run.smithi071.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-04T08:36:50.511 INFO:teuthology.orchestra.run.smithi071.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-04T08:36:50.511 INFO:teuthology.orchestra.run.smithi071.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-04T08:36:50.511 INFO:teuthology.orchestra.run.smithi071.stdout:hardware devices. 2024-03-04T08:36:50.511 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:36:50.512 INFO:teuthology.orchestra.run.smithi071.stdout:This subpackage provides tools for support of all platforms. 2024-03-04T08:36:50.512 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-04T08:36:50.512 INFO:teuthology.task.kernel:Updating grub on smithi071 to boot 5.14.0-427.el9.x86_64 2024-03-04T08:36:50.513 DEBUG:teuthology.orchestra.run.smithi071:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-04T08:36:51.097 INFO:teuthology.orchestra.run.smithi071.stderr:Generating grub configuration file ... 2024-03-04T08:36:52.393 INFO:teuthology.orchestra.run.smithi071.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-04T08:36:52.445 INFO:teuthology.orchestra.run.smithi071.stderr:done 2024-03-04T08:36:52.447 DEBUG:teuthology.orchestra.run.smithi071:> mktemp 2024-03-04T08:36:52.463 INFO:teuthology.orchestra.run.smithi071.stdout:/tmp/tmp.1sdfNjxhQ6 2024-03-04T08:36:52.463 DEBUG:teuthology.orchestra.run.smithi071:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.1sdfNjxhQ6 2024-03-04T08:36:52.535 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0666 /tmp/tmp.1sdfNjxhQ6 2024-03-04T08:36:52.679 DEBUG:teuthology.orchestra.remote:smithi071:/tmp/tmp.1sdfNjxhQ6 is 6KB 2024-03-04T08:36:52.730 DEBUG:teuthology.orchestra.run.smithi071:> rm -fr /tmp/tmp.1sdfNjxhQ6 2024-03-04T08:36:52.746 DEBUG:teuthology.orchestra.run.smithi071:> sudo /bin/ls /boot/loader/entries || true 2024-03-04T08:36:52.810 INFO:teuthology.orchestra.run.smithi071.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-04T08:36:52.810 INFO:teuthology.orchestra.run.smithi071.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-04T08:36:52.810 INFO:teuthology.orchestra.run.smithi071.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-04T08:36:52.810 INFO:teuthology.orchestra.run.smithi071.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-04T08:36:52.812 DEBUG:teuthology.orchestra.run.smithi071:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-04T08:36:52.966 DEBUG:teuthology.orchestra.run.smithi071:> sudo shutdown -r now 2024-03-04T08:37:22.969 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-04T08:37:22.970 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-03-04T08:37:22.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T08:38:22.973 DEBUG:teuthology.orchestra.remote:timed out 2024-03-04T08:38:31.974 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-03-04T08:38:31.975 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T08:38:35.312 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-03-04T08:38:35.691 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-03-04T08:38:35.691 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-427.el9.x86_64"... 2024-03-04T08:38:35.691 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-03-04T08:38:35.705 INFO:teuthology.orchestra.run.smithi071.stdout:5.14.0-427.el9.x86_64 2024-03-04T08:38:35.705 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-04T08:38:35.705 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-04T08:38:35.705 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-04T08:38:36.706 DEBUG:teuthology.parallel:result is None 2024-03-04T08:38:36.706 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-04T08:38:36.715 INFO:teuthology.task.internal:Creating test directory... 2024-03-04T08:38:36.716 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-04T08:38:36.718 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-04T08:38:36.721 DEBUG:teuthology.orchestra.run.smithi158:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-04T08:38:36.738 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-04T08:38:36.787 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-04T08:38:36.798 INFO:teuthology.task.internal:Creating archive directory... 2024-03-04T08:38:36.798 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-04T08:38:36.801 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-04T08:38:36.803 DEBUG:teuthology.orchestra.run.smithi158:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-04T08:38:36.843 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-04T08:38:36.853 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-04T08:38:36.853 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-04T08:38:36.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T08:38:36.888 DEBUG:teuthology.orchestra.run.smithi158:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-04T08:38:36.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T08:38:36.907 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-04T08:38:36.910 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-04T08:38:36.931 DEBUG:teuthology.orchestra.run.smithi158:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-04T08:38:36.980 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T08:38:36.983 INFO:teuthology.orchestra.run.smithi158.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T08:38:37.020 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T08:38:37.022 INFO:teuthology.orchestra.run.smithi158.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T08:38:37.027 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T08:38:37.041 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T08:38:37.043 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-04T08:38:37.053 INFO:teuthology.task.internal:Configuring sudo... 2024-03-04T08:38:37.053 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-04T08:38:37.056 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-04T08:38:37.058 DEBUG:teuthology.orchestra.run.smithi158:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-04T08:38:37.094 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-04T08:38:37.103 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-04T08:38:37.104 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-04T08:38:37.124 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-04T08:38:37.129 DEBUG:teuthology.orchestra.run.smithi158:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-04T08:38:37.150 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T08:38:37.217 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T08:38:37.266 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T08:38:37.347 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T08:38:37.388 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-04T08:38:37.389 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-04T08:38:37.457 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T08:38:37.502 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T08:38:37.563 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T08:38:37.608 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T08:38:37.651 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-03-04T08:38:37.652 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-04T08:38:37.721 DEBUG:teuthology.orchestra.run.smithi158:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T08:38:37.767 DEBUG:teuthology.orchestra.run.smithi158:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T08:38:37.828 DEBUG:teuthology.orchestra.run.smithi158:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T08:38:37.870 DEBUG:teuthology.orchestra.run.smithi158:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T08:38:37.917 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2024-03-04T08:38:37.917 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-04T08:38:37.986 DEBUG:teuthology.orchestra.run.smithi071:> sudo service rsyslog restart 2024-03-04T08:38:37.988 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2024-03-04T08:38:37.991 DEBUG:teuthology.orchestra.run.smithi158:> sudo service rsyslog restart 2024-03-04T08:38:38.029 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-04T08:38:38.032 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-04T08:38:38.066 INFO:teuthology.orchestra.run.smithi158.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-04T08:38:38.310 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-04T08:38:38.319 INFO:teuthology.task.internal:Starting timer... 2024-03-04T08:38:38.319 INFO:teuthology.run_tasks:Running task pcp... 2024-03-04T08:38:38.328 INFO:teuthology.run_tasks:Running task selinux... 2024-03-04T08:38:38.337 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-03-04T08:38:38.337 DEBUG:teuthology.orchestra.run.smithi071:> sudo service auditd rotate 2024-03-04T08:38:38.383 INFO:teuthology.orchestra.run.smithi071.stdout:Rotating logs: 2024-03-04T08:38:38.385 DEBUG:teuthology.orchestra.run.smithi150:> sudo service auditd rotate 2024-03-04T08:38:38.432 INFO:teuthology.orchestra.run.smithi150.stdout:Rotating logs: 2024-03-04T08:38:38.434 DEBUG:teuthology.orchestra.run.smithi158:> sudo service auditd rotate 2024-03-04T08:38:38.478 INFO:teuthology.orchestra.run.smithi158.stdout:Rotating logs: 2024-03-04T08:38:38.480 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-04T08:38:38.480 DEBUG:teuthology.orchestra.run.smithi071:> /usr/sbin/getenforce 2024-03-04T08:38:38.502 INFO:teuthology.orchestra.run.smithi071.stdout:Permissive 2024-03-04T08:38:38.503 DEBUG:teuthology.orchestra.run.smithi150:> /usr/sbin/getenforce 2024-03-04T08:38:38.522 INFO:teuthology.orchestra.run.smithi150.stdout:Permissive 2024-03-04T08:38:38.522 DEBUG:teuthology.orchestra.run.smithi158:> /usr/sbin/getenforce 2024-03-04T08:38:38.544 INFO:teuthology.orchestra.run.smithi158.stdout:Permissive 2024-03-04T08:38:38.544 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi071.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi150.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi158.front.sepia.ceph.com': 'permissive'} 2024-03-04T08:38:38.545 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-04T08:38:38.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T08:38:38.572 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-04T08:38:38.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T08:38:38.601 DEBUG:teuthology.orchestra.run.smithi158:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-04T08:38:38.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T08:38:38.629 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-04T08:38:38.629 DEBUG:teuthology.orchestra.run.smithi071:> sudo /usr/sbin/setenforce permissive 2024-03-04T08:38:38.721 DEBUG:teuthology.orchestra.run.smithi150:> sudo /usr/sbin/setenforce permissive 2024-03-04T08:38:38.766 DEBUG:teuthology.orchestra.run.smithi158:> sudo /usr/sbin/setenforce permissive 2024-03-04T08:38:38.795 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-04T08:38:38.804 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-04T08:38:38.828 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-04T08:38:38.959 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-04T08:38:38.991 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-04T08:38:38.992 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi071.front.sepia.ceph.com,smithi150.front.sepia.ceph.com,smithi158.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-04T08:44:00.772 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi071.front.sepia.ceph.com'), Remote(name='ubuntu@smithi150.front.sepia.ceph.com'), Remote(name='ubuntu@smithi158.front.sepia.ceph.com')] 2024-03-04T08:44:00.773 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-03-04T08:44:00.774 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T08:44:00.866 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-03-04T08:44:00.947 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-03-04T08:44:00.948 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-03-04T08:44:00.948 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T08:44:01.036 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-03-04T08:44:01.107 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-03-04T08:44:01.107 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com' 2024-03-04T08:44:01.107 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T08:44:01.198 DEBUG:teuthology.orchestra.run.smithi158:> true 2024-03-04T08:44:01.271 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi158.front.sepia.ceph.com' 2024-03-04T08:44:01.271 INFO:teuthology.run_tasks:Running task clock... 2024-03-04T08:44:01.282 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-04T08:44:01.282 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-04T08:44:01.283 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T08:44:01.285 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-04T08:44:01.285 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T08:44:01.287 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-04T08:44:01.288 DEBUG:teuthology.orchestra.run.smithi158:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T08:44:01.315 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-04T08:44:01.316 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-04T08:44:01.332 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-04T08:44:01.332 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-04T08:44:01.344 INFO:teuthology.orchestra.run.smithi158.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-04T08:44:01.361 INFO:teuthology.orchestra.run.smithi158.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-04T08:44:01.362 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: ntpd: command not found 2024-03-04T08:44:01.362 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: ntpd: command not found 2024-03-04T08:44:01.375 INFO:teuthology.orchestra.run.smithi071.stdout:506 Cannot talk to daemon 2024-03-04T08:44:01.375 INFO:teuthology.orchestra.run.smithi150.stdout:506 Cannot talk to daemon 2024-03-04T08:44:01.391 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-04T08:44:01.392 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-04T08:44:01.392 INFO:teuthology.orchestra.run.smithi158.stderr:sudo: ntpd: command not found 2024-03-04T08:44:01.406 INFO:teuthology.orchestra.run.smithi158.stdout:506 Cannot talk to daemon 2024-03-04T08:44:01.407 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-04T08:44:01.408 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-04T08:44:01.422 INFO:teuthology.orchestra.run.smithi158.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-04T08:44:01.438 INFO:teuthology.orchestra.run.smithi158.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-04T08:44:01.452 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-03-04T08:44:01.456 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-04T08:44:01.456 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-03-04T08:44:01.456 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T08:44:01.456 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T08:44:01.456 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T08:44:01.456 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T08:44:01.457 INFO:teuthology.orchestra.run.smithi071.stderr:bash: line 1: ntpq: command not found 2024-03-04T08:44:01.459 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-04T08:44:01.459 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-03-04T08:44:01.459 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T08:44:01.460 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T08:44:01.460 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T08:44:01.460 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T08:44:01.482 INFO:teuthology.orchestra.run.smithi158.stderr:bash: line 1: ntpq: command not found 2024-03-04T08:44:01.485 INFO:teuthology.orchestra.run.smithi158.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-04T08:44:01.485 INFO:teuthology.orchestra.run.smithi158.stdout:=============================================================================== 2024-03-04T08:44:01.485 INFO:teuthology.orchestra.run.smithi158.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T08:44:01.485 INFO:teuthology.orchestra.run.smithi158.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T08:44:01.485 INFO:teuthology.orchestra.run.smithi158.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T08:44:01.485 INFO:teuthology.orchestra.run.smithi158.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T08:44:01.486 INFO:teuthology.run_tasks:Running task install... 2024-03-04T08:44:01.495 DEBUG:teuthology.task.install:project ceph 2024-03-04T08:44:01.496 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'}} 2024-03-04T08:44:01.496 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'} 2024-03-04T08:44:01.496 INFO:teuthology.task.install:Using flavor: default 2024-03-04T08:44:01.504 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-04T08:44:01.504 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-04T08:44:01.504 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', 'tag': None, 'wait_for_package': False} 2024-03-04T08:44:01.505 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T08:44:01.507 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', 'tag': None, 'wait_for_package': False} 2024-03-04T08:44:01.507 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T08:44:01.508 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', 'tag': None, 'wait_for_package': False} 2024-03-04T08:44:01.508 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T08:44:01.720 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/ 2024-03-04T08:44:01.720 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1739.g1bc250c7 2024-03-04T08:44:01.739 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/ 2024-03-04T08:44:01.739 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1739.g1bc250c7 2024-03-04T08:44:01.743 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/ 2024-03-04T08:44:01.743 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1739.g1bc250c7 2024-03-04T08:44:01.928 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-04T08:44:01.929 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2024-03-04T08:44:01.929 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-04T08:44:01.957 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-04T08:44:01.957 DEBUG:teuthology.orchestra.run.smithi158:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-04T08:44:01.961 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-04T08:44:01.961 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-04T08:44:01.961 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-04T08:44:01.967 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-04T08:44:01.967 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-03-04T08:44:01.967 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-04T08:44:01.990 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-04T08:44:01.991 DEBUG:teuthology.orchestra.run.smithi071:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-04T08:44:01.996 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-04T08:44:01.996 DEBUG:teuthology.orchestra.run.smithi150:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-04T08:44:02.024 DEBUG:teuthology.orchestra.run.smithi158:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-04T08:44:02.059 DEBUG:teuthology.orchestra.run.smithi071:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-04T08:44:02.064 DEBUG:teuthology.orchestra.run.smithi150:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-04T08:44:02.104 DEBUG:teuthology.orchestra.run.smithi158:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-04T08:44:02.139 DEBUG:teuthology.orchestra.run.smithi071:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-04T08:44:02.143 DEBUG:teuthology.orchestra.run.smithi150:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-04T08:44:02.170 INFO:teuthology.orchestra.run.smithi158.stdout:check_obsoletes = 1 2024-03-04T08:44:02.171 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum clean all 2024-03-04T08:44:02.205 INFO:teuthology.orchestra.run.smithi071.stdout:check_obsoletes = 1 2024-03-04T08:44:02.208 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum clean all 2024-03-04T08:44:02.209 INFO:teuthology.orchestra.run.smithi150.stdout:check_obsoletes = 1 2024-03-04T08:44:02.211 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum clean all 2024-03-04T08:44:02.405 INFO:teuthology.orchestra.run.smithi158.stdout:74 files removed 2024-03-04T08:44:02.433 DEBUG:teuthology.orchestra.run.smithi158:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-04T08:44:02.446 INFO:teuthology.orchestra.run.smithi150.stdout:74 files removed 2024-03-04T08:44:02.447 INFO:teuthology.orchestra.run.smithi071.stdout:74 files removed 2024-03-04T08:44:02.474 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-04T08:44:02.475 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-04T08:44:03.122 INFO:teuthology.orchestra.run.smithi150.stdout:ceph packages for x86_64 236 kB/s | 83 kB 00:00 2024-03-04T08:44:03.142 INFO:teuthology.orchestra.run.smithi158.stdout:ceph packages for x86_64 198 kB/s | 83 kB 00:00 2024-03-04T08:44:03.204 INFO:teuthology.orchestra.run.smithi071.stdout:ceph packages for x86_64 201 kB/s | 83 kB 00:00 2024-03-04T08:44:03.539 INFO:teuthology.orchestra.run.smithi150.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-03-04T08:44:03.568 INFO:teuthology.orchestra.run.smithi158.stdout:ceph noarch packages 26 kB/s | 10 kB 00:00 2024-03-04T08:44:03.646 INFO:teuthology.orchestra.run.smithi071.stdout:ceph noarch packages 25 kB/s | 10 kB 00:00 2024-03-04T08:44:03.981 INFO:teuthology.orchestra.run.smithi150.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-03-04T08:44:03.993 INFO:teuthology.orchestra.run.smithi158.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-03-04T08:44:04.021 INFO:teuthology.orchestra.run.smithi071.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-03-04T08:44:04.495 INFO:teuthology.orchestra.run.smithi158.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-04T08:44:04.501 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-04T08:44:04.515 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-04T08:44:05.917 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS-9 - BaseOS 6.5 MB/s | 2.0 MB 00:00 2024-03-04T08:44:05.932 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS-9 - BaseOS 6.5 MB/s | 2.0 MB 00:00 2024-03-04T08:44:05.955 INFO:teuthology.orchestra.run.smithi158.stdout:CentOS-9 - BaseOS 5.8 MB/s | 2.0 MB 00:00 2024-03-04T08:44:07.325 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 7.5 MB/s | 8.0 MB 00:01 2024-03-04T08:44:07.962 INFO:teuthology.orchestra.run.smithi158.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.0 MB 00:01 2024-03-04T08:44:09.337 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-03-04T08:44:14.152 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.1 MB 00:01 2024-03-04T08:44:15.188 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - BaseOS 921 kB/s | 8.0 MB 00:08 2024-03-04T08:44:15.954 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 15 kB 00:00 2024-03-04T08:44:18.099 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 10 MB/s | 20 MB 00:02 2024-03-04T08:44:18.114 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-03-04T08:44:20.770 INFO:teuthology.orchestra.run.smithi158.stdout:CentOS Stream 9 - AppStream 1.6 MB/s | 19 MB 00:11 2024-03-04T08:44:22.531 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - CRB 7.1 MB/s | 6.1 MB 00:00 2024-03-04T08:44:23.063 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-03-04T08:44:24.340 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - Extras packages 22 kB/s | 15 kB 00:00 2024-03-04T08:44:25.158 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 29 MB/s | 20 MB 00:00 2024-03-04T08:44:25.521 INFO:teuthology.orchestra.run.smithi158.stdout:CentOS Stream 9 - CRB 4.9 MB/s | 6.1 MB 00:01 2024-03-04T08:44:25.586 INFO:teuthology.orchestra.run.smithi150.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-04T08:44:25.587 INFO:teuthology.orchestra.run.smithi150.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-04T08:44:25.601 INFO:teuthology.orchestra.run.smithi150.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-04T08:44:25.602 INFO:teuthology.orchestra.run.smithi150.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-04T08:44:25.604 INFO:teuthology.orchestra.run.smithi150.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-04T08:44:25.605 INFO:teuthology.orchestra.run.smithi150.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-04T08:44:25.607 INFO:teuthology.orchestra.run.smithi150.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-04T08:44:25.610 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-04T08:44:25.611 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-04T08:44:25.695 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-03-04T08:44:25.722 INFO:teuthology.orchestra.run.smithi150.stdout:============================================================================================ 2024-03-04T08:44:25.722 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-03-04T08:44:25.722 INFO:teuthology.orchestra.run.smithi150.stdout:============================================================================================ 2024-03-04T08:44:25.722 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-03-04T08:44:25.722 INFO:teuthology.orchestra.run.smithi150.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-04T08:44:25.722 INFO:teuthology.orchestra.run.smithi150.stdout: ceph x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 6.6 k 2024-03-04T08:44:25.722 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 5.3 M 2024-03-04T08:44:25.722 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 888 k 2024-03-04T08:44:25.722 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 145 k 2024-03-04T08:44:25.722 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 1.5 M 2024-03-04T08:44:25.722 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 138 k 2024-03-04T08:44:25.723 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 1.8 M 2024-03-04T08:44:25.723 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 7.4 M 2024-03-04T08:44:25.723 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 49 k 2024-03-04T08:44:25.723 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 9.8 M 2024-03-04T08:44:25.723 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 47 M 2024-03-04T08:44:25.723 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 264 k 2024-03-04T08:44:25.723 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 748 k 2024-03-04T08:44:25.723 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 220 k 2024-03-04T08:44:25.723 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-shell x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 27 k 2024-03-04T08:44:25.723 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 18 k 2024-03-04T08:44:25.723 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-03-04T08:44:25.723 INFO:teuthology.orchestra.run.smithi150.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-04T08:44:25.723 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-04T08:44:25.724 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 31 k 2024-03-04T08:44:25.724 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 747 k 2024-03-04T08:44:25.724 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 127 k 2024-03-04T08:44:25.724 INFO:teuthology.orchestra.run.smithi150.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-04T08:44:25.724 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-03-04T08:44:25.724 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel x86_64 1:3.0.7-27.el9 CentOS-AppStream 3.0 M 2024-03-04T08:44:25.724 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-04T08:44:25.724 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-04T08:44:25.724 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-04T08:44:25.724 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 162 k 2024-03-04T08:44:25.724 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 321 k 2024-03-04T08:44:25.724 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 298 k 2024-03-04T08:44:25.725 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 99 k 2024-03-04T08:44:25.725 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 85 k 2024-03-04T08:44:25.725 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.1 M 2024-03-04T08:44:25.725 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 171 k 2024-03-04T08:44:25.725 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-03-04T08:44:25.725 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrading: 2024-03-04T08:44:25.725 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager x86_64 1:1.46.0-1.el9 CentOS-BaseOS 2.3 M 2024-03-04T08:44:25.725 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 CentOS-BaseOS 1.8 M 2024-03-04T08:44:25.725 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 CentOS-BaseOS 41 k 2024-03-04T08:44:25.725 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 CentOS-BaseOS 247 k 2024-03-04T08:44:25.725 INFO:teuthology.orchestra.run.smithi150.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-03-04T08:44:25.725 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-03-04T08:44:25.726 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-03-04T08:44:25.726 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-03-04T08:44:25.726 INFO:teuthology.orchestra.run.smithi150.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-03-04T08:44:25.726 INFO:teuthology.orchestra.run.smithi150.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-03-04T08:44:25.726 INFO:teuthology.orchestra.run.smithi150.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-03-04T08:44:25.726 INFO:teuthology.orchestra.run.smithi150.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-03-04T08:44:25.726 INFO:teuthology.orchestra.run.smithi150.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-03-04T08:44:25.726 INFO:teuthology.orchestra.run.smithi150.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-03-04T08:44:25.726 INFO:teuthology.orchestra.run.smithi150.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-03-04T08:44:25.726 INFO:teuthology.orchestra.run.smithi150.stdout: librados2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.4 M 2024-03-04T08:44:25.726 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.1 M 2024-03-04T08:44:25.726 INFO:teuthology.orchestra.run.smithi150.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-03-04T08:44:25.727 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-03-04T08:44:25.727 INFO:teuthology.orchestra.run.smithi150.stdout: openssl x86_64 1:3.0.7-27.el9 CentOS-BaseOS 1.2 M 2024-03-04T08:44:25.727 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs x86_64 1:3.0.7-27.el9 CentOS-BaseOS 2.2 M 2024-03-04T08:44:25.727 INFO:teuthology.orchestra.run.smithi150.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-03-04T08:44:25.727 INFO:teuthology.orchestra.run.smithi150.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-03-04T08:44:25.727 INFO:teuthology.orchestra.run.smithi150.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-03-04T08:44:25.727 INFO:teuthology.orchestra.run.smithi150.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-04T08:44:25.727 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-03-04T08:44:25.727 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-03-04T08:44:25.727 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-04T08:44:25.727 INFO:teuthology.orchestra.run.smithi150.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-04T08:44:25.727 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-04T08:44:25.727 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 20 M 2024-03-04T08:44:25.728 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 31 k 2024-03-04T08:44:25.728 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 2.3 M 2024-03-04T08:44:25.728 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 248 k 2024-03-04T08:44:25.728 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 4.7 M 2024-03-04T08:44:25.728 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 17 M 2024-03-04T08:44:25.728 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 15 k 2024-03-04T08:44:25.728 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 25 k 2024-03-04T08:44:25.728 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-04T08:44:25.728 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-04T08:44:25.728 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-04T08:44:25.728 INFO:teuthology.orchestra.run.smithi150.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-04T08:44:25.728 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-04T08:44:25.729 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-04T08:44:25.729 INFO:teuthology.orchestra.run.smithi150.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-03-04T08:44:25.729 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-04T08:44:25.729 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-04T08:44:25.729 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 163 k 2024-03-04T08:44:25.729 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-03-04T08:44:25.729 INFO:teuthology.orchestra.run.smithi150.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-04T08:44:25.729 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-04T08:44:25.729 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-03-04T08:44:25.729 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-04T08:44:25.729 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 501 k 2024-03-04T08:44:25.729 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-04T08:44:25.729 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 5.1 M 2024-03-04T08:44:25.729 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-04T08:44:25.730 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-04T08:44:25.730 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-04T08:44:25.730 INFO:teuthology.orchestra.run.smithi150.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-04T08:44:25.730 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-04T08:44:25.730 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-04T08:44:25.730 INFO:teuthology.orchestra.run.smithi150.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-03-04T08:44:25.730 INFO:teuthology.orchestra.run.smithi150.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-04T08:44:25.730 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-04T08:44:25.730 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-04T08:44:25.730 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-04T08:44:25.730 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-04T08:44:25.730 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-04T08:44:25.730 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-04T08:44:25.730 INFO:teuthology.orchestra.run.smithi150.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-04T08:44:25.731 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-04T08:44:25.731 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-04T08:44:25.731 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-04T08:44:25.731 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 45 k 2024-03-04T08:44:25.731 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 131 k 2024-03-04T08:44:25.731 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-04T08:44:25.731 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-04T08:44:25.731 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-04T08:44:25.731 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-04T08:44:25.731 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-04T08:44:25.731 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-04T08:44:25.731 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-04T08:44:25.731 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-04T08:44:25.731 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-04T08:44:25.732 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-04T08:44:25.732 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-04T08:44:25.732 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-04T08:44:25.732 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-04T08:44:25.732 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-04T08:44:25.732 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-04T08:44:25.732 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-04T08:44:25.732 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-03-04T08:44:25.732 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-04T08:44:25.732 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-04T08:44:25.732 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-04T08:44:25.732 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-04T08:44:25.733 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-04T08:44:25.733 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-04T08:44:25.733 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-04T08:44:25.733 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-04T08:44:25.733 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-04T08:44:25.733 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-04T08:44:25.733 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-04T08:44:25.733 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-04T08:44:25.733 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-04T08:44:25.733 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-04T08:44:25.733 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-04T08:44:25.733 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-04T08:44:25.733 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-04T08:44:25.733 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-04T08:44:25.734 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-04T08:44:25.734 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-04T08:44:25.734 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-04T08:44:25.734 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-04T08:44:25.734 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-04T08:44:25.734 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-04T08:44:25.734 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-03-04T08:44:25.734 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-04T08:44:25.734 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-04T08:44:25.734 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-04T08:44:25.734 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-04T08:44:25.734 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-04T08:44:25.734 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-04T08:44:25.734 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-04T08:44:25.735 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-04T08:44:25.735 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-04T08:44:25.735 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-04T08:44:25.735 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-04T08:44:25.735 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-04T08:44:25.735 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-04T08:44:25.735 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-03-04T08:44:25.735 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-03-04T08:44:25.735 INFO:teuthology.orchestra.run.smithi150.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-04T08:44:25.735 INFO:teuthology.orchestra.run.smithi150.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-04T08:44:25.735 INFO:teuthology.orchestra.run.smithi150.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-04T08:44:25.735 INFO:teuthology.orchestra.run.smithi150.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-04T08:44:25.735 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-04T08:44:25.735 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-04T08:44:25.736 INFO:teuthology.orchestra.run.smithi150.stdout:Installing weak dependencies: 2024-03-04T08:44:25.736 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-03-04T08:44:25.736 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-04T08:44:25.736 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-03-04T08:44:25.736 INFO:teuthology.orchestra.run.smithi150.stdout:============================================================================================ 2024-03-04T08:44:25.736 INFO:teuthology.orchestra.run.smithi150.stdout:Install 150 Packages 2024-03-04T08:44:25.736 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrade 26 Packages 2024-03-04T08:44:25.736 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-04T08:44:25.737 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 224 M 2024-03-04T08:44:25.737 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-03-04T08:44:27.047 INFO:teuthology.orchestra.run.smithi158.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 15 kB 00:00 2024-03-04T08:44:27.064 INFO:teuthology.orchestra.run.smithi150.stdout:(1/176): ceph-19.0.0-1739.g1bc250c7.el9.x86_64. 12 kB/s | 6.6 kB 00:00 2024-03-04T08:44:27.756 INFO:teuthology.orchestra.run.smithi150.stdout:(2/176): ceph-common-19.0.0-1739.g1bc250c7.el9. 16 MB/s | 20 MB 00:01 2024-03-04T08:44:27.806 INFO:teuthology.orchestra.run.smithi150.stdout:(3/176): ceph-immutable-object-cache-19.0.0-173 2.9 MB/s | 145 kB 00:00 2024-03-04T08:44:27.940 INFO:teuthology.orchestra.run.smithi150.stdout:(4/176): ceph-mds-19.0.0-1739.g1bc250c7.el9.x86 17 MB/s | 2.3 MB 00:00 2024-03-04T08:44:28.040 INFO:teuthology.orchestra.run.smithi150.stdout:(5/176): ceph-mgr-19.0.0-1739.g1bc250c7.el9.x86 15 MB/s | 1.5 MB 00:00 2024-03-04T08:44:28.290 INFO:teuthology.orchestra.run.smithi150.stdout:(6/176): ceph-mon-19.0.0-1739.g1bc250c7.el9.x86 19 MB/s | 4.7 MB 00:00 2024-03-04T08:44:28.490 INFO:teuthology.orchestra.run.smithi150.stdout:(7/176): ceph-fuse-19.0.0-1739.g1bc250c7.el9.x8 623 kB/s | 888 kB 00:01 2024-03-04T08:44:29.117 INFO:teuthology.orchestra.run.smithi150.stdout:(8/176): ceph-osd-19.0.0-1739.g1bc250c7.el9.x86 20 MB/s | 17 MB 00:00 2024-03-04T08:44:29.199 INFO:teuthology.orchestra.run.smithi150.stdout:(9/176): ceph-selinux-19.0.0-1739.g1bc250c7.el9 308 kB/s | 25 kB 00:00 2024-03-04T08:44:29.951 INFO:teuthology.orchestra.run.smithi158.stdout:Extra Packages for Enterprise Linux 7.4 MB/s | 20 MB 00:02 2024-03-04T08:44:30.075 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-03-04T08:44:31.436 INFO:teuthology.orchestra.run.smithi150.stdout:(10/176): ceph-test-19.0.0-1739.g1bc250c7.el9.x 21 MB/s | 47 MB 00:02 2024-03-04T08:44:31.483 INFO:teuthology.orchestra.run.smithi150.stdout:(11/176): cephfs-mirror-19.0.0-1739.g1bc250c7.e 4.6 MB/s | 220 kB 00:00 2024-03-04T08:44:31.525 INFO:teuthology.orchestra.run.smithi150.stdout:(12/176): cephfs-shell-19.0.0-1739.g1bc250c7.el 651 kB/s | 27 kB 00:00 2024-03-04T08:44:31.558 INFO:teuthology.orchestra.run.smithi150.stdout:(13/176): libcephfs-devel-19.0.0-1739.g1bc250c7 942 kB/s | 31 kB 00:00 2024-03-04T08:44:31.625 INFO:teuthology.orchestra.run.smithi150.stdout:(14/176): libcephfs2-19.0.0-1739.g1bc250c7.el9. 11 MB/s | 747 kB 00:00 2024-03-04T08:44:31.676 INFO:teuthology.orchestra.run.smithi150.stdout:(15/176): libcephsqlite-19.0.0-1739.g1bc250c7.e 3.2 MB/s | 163 kB 00:00 2024-03-04T08:44:31.726 INFO:teuthology.orchestra.run.smithi150.stdout:(16/176): librados-devel-19.0.0-1739.g1bc250c7. 2.5 MB/s | 127 kB 00:00 2024-03-04T08:44:31.785 INFO:teuthology.orchestra.run.smithi150.stdout:(17/176): libradosstriper1-19.0.0-1739.g1bc250c 8.4 MB/s | 501 kB 00:00 2024-03-04T08:44:32.052 INFO:teuthology.orchestra.run.smithi150.stdout:(18/176): librgw2-19.0.0-1739.g1bc250c7.el9.x86 19 MB/s | 5.1 MB 00:00 2024-03-04T08:44:32.093 INFO:teuthology.orchestra.run.smithi150.stdout:(19/176): python3-ceph-argparse-19.0.0-1739.g1b 1.1 MB/s | 45 kB 00:00 2024-03-04T08:44:32.144 INFO:teuthology.orchestra.run.smithi150.stdout:(20/176): python3-ceph-common-19.0.0-1739.g1bc2 2.6 MB/s | 131 kB 00:00 2024-03-04T08:44:32.194 INFO:teuthology.orchestra.run.smithi150.stdout:(21/176): python3-cephfs-19.0.0-1739.g1bc250c7. 3.2 MB/s | 162 kB 00:00 2024-03-04T08:44:32.244 INFO:teuthology.orchestra.run.smithi150.stdout:(22/176): python3-rados-19.0.0-1739.g1bc250c7.e 6.3 MB/s | 321 kB 00:00 2024-03-04T08:44:32.295 INFO:teuthology.orchestra.run.smithi150.stdout:(23/176): python3-rbd-19.0.0-1739.g1bc250c7.el9 5.8 MB/s | 298 kB 00:00 2024-03-04T08:44:32.345 INFO:teuthology.orchestra.run.smithi150.stdout:(24/176): python3-rgw-19.0.0-1739.g1bc250c7.el9 1.9 MB/s | 99 kB 00:00 2024-03-04T08:44:32.387 INFO:teuthology.orchestra.run.smithi150.stdout:(25/176): rbd-fuse-19.0.0-1739.g1bc250c7.el9.x8 2.0 MB/s | 85 kB 00:00 2024-03-04T08:44:32.562 INFO:teuthology.orchestra.run.smithi150.stdout:(26/176): rbd-mirror-19.0.0-1739.g1bc250c7.el9. 18 MB/s | 3.1 MB 00:00 2024-03-04T08:44:32.611 INFO:teuthology.orchestra.run.smithi071.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-04T08:44:32.612 INFO:teuthology.orchestra.run.smithi071.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-04T08:44:32.612 INFO:teuthology.orchestra.run.smithi150.stdout:(27/176): rbd-nbd-19.0.0-1739.g1bc250c7.el9.x86 3.3 MB/s | 171 kB 00:00 2024-03-04T08:44:32.624 INFO:teuthology.orchestra.run.smithi071.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-04T08:44:32.624 INFO:teuthology.orchestra.run.smithi071.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-04T08:44:32.626 INFO:teuthology.orchestra.run.smithi071.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-04T08:44:32.626 INFO:teuthology.orchestra.run.smithi071.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-04T08:44:32.628 INFO:teuthology.orchestra.run.smithi071.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-04T08:44:32.631 INFO:teuthology.orchestra.run.smithi071.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-04T08:44:32.631 INFO:teuthology.orchestra.run.smithi071.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-04T08:44:32.654 INFO:teuthology.orchestra.run.smithi150.stdout:(28/176): ceph-grafana-dashboards-19.0.0-1739.g 756 kB/s | 31 kB 00:00 2024-03-04T08:44:32.705 INFO:teuthology.orchestra.run.smithi150.stdout:(29/176): ceph-mgr-cephadm-19.0.0-1739.g1bc250c 2.7 MB/s | 138 kB 00:00 2024-03-04T08:44:32.712 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-03-04T08:44:32.732 INFO:teuthology.orchestra.run.smithi071.stdout:============================================================================================ 2024-03-04T08:44:32.732 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-03-04T08:44:32.733 INFO:teuthology.orchestra.run.smithi071.stdout:============================================================================================ 2024-03-04T08:44:32.733 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-03-04T08:44:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-04T08:44:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: ceph x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 6.6 k 2024-03-04T08:44:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 5.3 M 2024-03-04T08:44:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 888 k 2024-03-04T08:44:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 145 k 2024-03-04T08:44:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 1.5 M 2024-03-04T08:44:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 138 k 2024-03-04T08:44:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 1.8 M 2024-03-04T08:44:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 7.4 M 2024-03-04T08:44:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 49 k 2024-03-04T08:44:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 9.8 M 2024-03-04T08:44:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 47 M 2024-03-04T08:44:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 264 k 2024-03-04T08:44:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 748 k 2024-03-04T08:44:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-mirror x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 220 k 2024-03-04T08:44:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-shell x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 27 k 2024-03-04T08:44:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-top noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 18 k 2024-03-04T08:44:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-03-04T08:44:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-04T08:44:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-04T08:44:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 31 k 2024-03-04T08:44:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 747 k 2024-03-04T08:44:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 127 k 2024-03-04T08:44:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-04T08:44:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-03-04T08:44:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-devel x86_64 1:3.0.7-27.el9 CentOS-AppStream 3.0 M 2024-03-04T08:44:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-04T08:44:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-04T08:44:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-04T08:44:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 162 k 2024-03-04T08:44:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 321 k 2024-03-04T08:44:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 298 k 2024-03-04T08:44:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 99 k 2024-03-04T08:44:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 85 k 2024-03-04T08:44:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.1 M 2024-03-04T08:44:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 171 k 2024-03-04T08:44:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-03-04T08:44:32.736 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrading: 2024-03-04T08:44:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager x86_64 1:1.46.0-1.el9 CentOS-BaseOS 2.3 M 2024-03-04T08:44:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 CentOS-BaseOS 1.8 M 2024-03-04T08:44:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 CentOS-BaseOS 41 k 2024-03-04T08:44:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 CentOS-BaseOS 247 k 2024-03-04T08:44:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-03-04T08:44:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-03-04T08:44:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-03-04T08:44:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-03-04T08:44:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-03-04T08:44:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-03-04T08:44:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-03-04T08:44:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-03-04T08:44:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-03-04T08:44:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-03-04T08:44:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-03-04T08:44:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: librados2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.4 M 2024-03-04T08:44:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.1 M 2024-03-04T08:44:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-03-04T08:44:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-03-04T08:44:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: openssl x86_64 1:3.0.7-27.el9 CentOS-BaseOS 1.2 M 2024-03-04T08:44:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-libs x86_64 1:3.0.7-27.el9 CentOS-BaseOS 2.2 M 2024-03-04T08:44:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-03-04T08:44:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-03-04T08:44:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-03-04T08:44:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-04T08:44:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-03-04T08:44:32.738 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-03-04T08:44:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-04T08:44:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-04T08:44:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-04T08:44:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 20 M 2024-03-04T08:44:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 31 k 2024-03-04T08:44:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 2.3 M 2024-03-04T08:44:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 248 k 2024-03-04T08:44:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 4.7 M 2024-03-04T08:44:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 17 M 2024-03-04T08:44:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 15 k 2024-03-04T08:44:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 25 k 2024-03-04T08:44:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-04T08:44:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-04T08:44:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-04T08:44:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-04T08:44:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-04T08:44:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-04T08:44:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-03-04T08:44:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-04T08:44:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-04T08:44:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 163 k 2024-03-04T08:44:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-03-04T08:44:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-04T08:44:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-04T08:44:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-03-04T08:44:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-04T08:44:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 501 k 2024-03-04T08:44:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-04T08:44:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 5.1 M 2024-03-04T08:44:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-04T08:44:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-04T08:44:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-04T08:44:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-04T08:44:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-04T08:44:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-04T08:44:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-03-04T08:44:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-04T08:44:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-04T08:44:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-04T08:44:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-04T08:44:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-04T08:44:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-04T08:44:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-04T08:44:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-04T08:44:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-04T08:44:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-04T08:44:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-04T08:44:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 45 k 2024-03-04T08:44:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 131 k 2024-03-04T08:44:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-04T08:44:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-04T08:44:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-04T08:44:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-04T08:44:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-04T08:44:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-04T08:44:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-04T08:44:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-04T08:44:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-04T08:44:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-04T08:44:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-04T08:44:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-04T08:44:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-04T08:44:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-04T08:44:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-04T08:44:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-04T08:44:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-03-04T08:44:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-04T08:44:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-04T08:44:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-04T08:44:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-04T08:44:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-04T08:44:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-04T08:44:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-04T08:44:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-04T08:44:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-04T08:44:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-04T08:44:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-04T08:44:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-04T08:44:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-04T08:44:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-04T08:44:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-04T08:44:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-04T08:44:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-04T08:44:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-04T08:44:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-04T08:44:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-04T08:44:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-04T08:44:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-04T08:44:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-04T08:44:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-04T08:44:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-03-04T08:44:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-04T08:44:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-04T08:44:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-04T08:44:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-04T08:44:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-04T08:44:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-04T08:44:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-04T08:44:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-04T08:44:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-04T08:44:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-04T08:44:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-04T08:44:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-04T08:44:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-04T08:44:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-03-04T08:44:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-03-04T08:44:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-04T08:44:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-04T08:44:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-04T08:44:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-04T08:44:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-04T08:44:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-04T08:44:32.746 INFO:teuthology.orchestra.run.smithi071.stdout:Installing weak dependencies: 2024-03-04T08:44:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-03-04T08:44:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:44:32.746 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-03-04T08:44:32.746 INFO:teuthology.orchestra.run.smithi071.stdout:============================================================================================ 2024-03-04T08:44:32.746 INFO:teuthology.orchestra.run.smithi071.stdout:Install 150 Packages 2024-03-04T08:44:32.746 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrade 26 Packages 2024-03-04T08:44:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:44:32.747 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 224 M 2024-03-04T08:44:32.747 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-03-04T08:44:32.822 INFO:teuthology.orchestra.run.smithi150.stdout:(30/176): ceph-mgr-dashboard-19.0.0-1739.g1bc25 15 MB/s | 1.8 MB 00:00 2024-03-04T08:44:33.189 INFO:teuthology.orchestra.run.smithi150.stdout:(31/176): ceph-mgr-diskprediction-local-19.0.0- 20 MB/s | 7.4 MB 00:00 2024-03-04T08:44:33.217 INFO:teuthology.orchestra.run.smithi071.stdout:(1/176): ceph-19.0.0-1739.g1bc250c7.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-03-04T08:44:33.239 INFO:teuthology.orchestra.run.smithi150.stdout:(32/176): ceph-mgr-modules-core-19.0.0-1739.g1b 4.9 MB/s | 248 kB 00:00 2024-03-04T08:44:33.281 INFO:teuthology.orchestra.run.smithi150.stdout:(33/176): ceph-mgr-rook-19.0.0-1739.g1bc250c7.e 1.2 MB/s | 49 kB 00:00 2024-03-04T08:44:33.323 INFO:teuthology.orchestra.run.smithi150.stdout:(34/176): ceph-prometheus-alerts-19.0.0-1739.g1 369 kB/s | 15 kB 00:00 2024-03-04T08:44:33.373 INFO:teuthology.orchestra.run.smithi150.stdout:(35/176): ceph-volume-19.0.0-1739.g1bc250c7.el9 5.2 MB/s | 264 kB 00:00 2024-03-04T08:44:33.448 INFO:teuthology.orchestra.run.smithi150.stdout:(36/176): cephadm-19.0.0-1739.g1bc250c7.el9.noa 9.8 MB/s | 748 kB 00:00 2024-03-04T08:44:33.490 INFO:teuthology.orchestra.run.smithi150.stdout:(37/176): cephfs-top-19.0.0-1739.g1bc250c7.el9. 440 kB/s | 18 kB 00:00 2024-03-04T08:44:33.526 INFO:teuthology.orchestra.run.smithi071.stdout:(2/176): ceph-fuse-19.0.0-1739.g1bc250c7.el9.x8 2.8 MB/s | 888 kB 00:00 2024-03-04T08:44:33.609 INFO:teuthology.orchestra.run.smithi071.stdout:(3/176): ceph-immutable-object-cache-19.0.0-173 1.7 MB/s | 145 kB 00:00 2024-03-04T08:44:33.699 INFO:teuthology.orchestra.run.smithi150.stdout:(38/176): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 662 kB 00:00 2024-03-04T08:44:33.832 INFO:teuthology.orchestra.run.smithi150.stdout:(39/176): bison-3.7.4-5.el9.x86_64.rpm 6.8 MB/s | 920 kB 00:00 2024-03-04T08:44:33.899 INFO:teuthology.orchestra.run.smithi150.stdout:(40/176): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-03-04T08:44:33.966 INFO:teuthology.orchestra.run.smithi150.stdout:(41/176): elfutils-libelf-devel-0.190-2.el9.x86 306 kB/s | 20 kB 00:00 2024-03-04T08:44:34.042 INFO:teuthology.orchestra.run.smithi150.stdout:(42/176): flex-2.6.4-9.el9.x86_64.rpm 4.0 MB/s | 307 kB 00:00 2024-03-04T08:44:34.100 INFO:teuthology.orchestra.run.smithi150.stdout:(43/176): flexiblas-3.0.4-8.el9.x86_64.rpm 545 kB/s | 32 kB 00:00 2024-03-04T08:44:34.143 INFO:teuthology.orchestra.run.smithi071.stdout:(4/176): ceph-mds-19.0.0-1739.g1bc250c7.el9.x86 4.3 MB/s | 2.3 MB 00:00 2024-03-04T08:44:34.277 INFO:teuthology.orchestra.run.smithi071.stdout:(5/176): ceph-base-19.0.0-1739.g1bc250c7.el9.x8 4.0 MB/s | 5.3 MB 00:01 2024-03-04T08:44:34.367 INFO:teuthology.orchestra.run.smithi150.stdout:(44/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-03-04T08:44:34.426 INFO:teuthology.orchestra.run.smithi150.stdout:(45/176): flexiblas-openblas-openmp-3.0.4-8.el9 294 kB/s | 17 kB 00:00 2024-03-04T08:44:34.476 INFO:teuthology.orchestra.run.smithi150.stdout:(46/176): libaio-devel-0.3.111-13.el9.x86_64.rp 202 kB/s | 10 kB 00:00 2024-03-04T08:44:34.485 INFO:teuthology.orchestra.run.smithi071.stdout:(6/176): ceph-mgr-19.0.0-1739.g1bc250c7.el9.x86 4.4 MB/s | 1.5 MB 00:00 2024-03-04T08:44:34.543 INFO:teuthology.orchestra.run.smithi150.stdout:(47/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 159 kB 00:00 2024-03-04T08:44:34.610 INFO:teuthology.orchestra.run.smithi150.stdout:(48/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 664 kB/s | 44 kB 00:00 2024-03-04T08:44:34.702 INFO:teuthology.orchestra.run.smithi150.stdout:(49/176): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 661 kB 00:00 2024-03-04T08:44:34.786 INFO:teuthology.orchestra.run.smithi150.stdout:(50/176): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 578 kB 00:00 2024-03-04T08:44:34.789 INFO:teuthology.orchestra.run.smithi158.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-03-04T08:44:34.844 INFO:teuthology.orchestra.run.smithi150.stdout:(51/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-03-04T08:44:34.902 INFO:teuthology.orchestra.run.smithi071.stdout:(7/176): ceph-mon-19.0.0-1739.g1bc250c7.el9.x86 7.5 MB/s | 4.7 MB 00:00 2024-03-04T08:44:34.911 INFO:teuthology.orchestra.run.smithi150.stdout:(52/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 696 kB/s | 46 kB 00:00 2024-03-04T08:44:34.971 INFO:teuthology.orchestra.run.smithi150.stdout:(53/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.7 MB/s | 282 kB 00:00 2024-03-04T08:44:35.012 INFO:teuthology.orchestra.run.smithi150.stdout:(54/176): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2024-03-04T08:44:35.071 INFO:teuthology.orchestra.run.smithi150.stdout:(55/176): openblas-0.3.21-2.el9.x86_64.rpm 587 kB/s | 34 kB 00:00 2024-03-04T08:44:35.488 INFO:teuthology.orchestra.run.smithi150.stdout:(56/176): openblas-openmp-0.3.21-2.el9.x86_64.r 11 MB/s | 4.7 MB 00:00 2024-03-04T08:44:35.713 INFO:teuthology.orchestra.run.smithi150.stdout:(57/176): openssl-devel-3.0.7-27.el9.x86_64.rpm 13 MB/s | 3.0 MB 00:00 2024-03-04T08:44:35.846 INFO:teuthology.orchestra.run.smithi150.stdout:(58/176): postgresql-13.11-1.el9.x86_64.rpm 11 MB/s | 1.5 MB 00:00 2024-03-04T08:44:35.947 INFO:teuthology.orchestra.run.smithi150.stdout:(59/176): postgresql-contrib-13.11-1.el9.x86_64 7.9 MB/s | 811 kB 00:00 2024-03-04T08:44:35.988 INFO:teuthology.orchestra.run.smithi150.stdout:(60/176): postgresql-private-libs-13.11-1.el9.x 3.2 MB/s | 134 kB 00:00 2024-03-04T08:44:36.069 INFO:teuthology.orchestra.run.smithi071.stdout:(8/176): ceph-common-19.0.0-1739.g1bc250c7.el9. 6.5 MB/s | 20 MB 00:03 2024-03-04T08:44:36.186 INFO:teuthology.orchestra.run.smithi071.stdout:(9/176): ceph-radosgw-19.0.0-1739.g1bc250c7.el9 7.6 MB/s | 9.8 MB 00:01 2024-03-04T08:44:36.211 INFO:teuthology.orchestra.run.smithi071.stdout:(10/176): ceph-selinux-19.0.0-1739.g1bc250c7.el 177 kB/s | 25 kB 00:00 2024-03-04T08:44:36.336 INFO:teuthology.orchestra.run.smithi071.stdout:(11/176): cephfs-mirror-19.0.0-1739.g1bc250c7.e 1.7 MB/s | 220 kB 00:00 2024-03-04T08:44:36.436 INFO:teuthology.orchestra.run.smithi071.stdout:(12/176): cephfs-shell-19.0.0-1739.g1bc250c7.el 271 kB/s | 27 kB 00:00 2024-03-04T08:44:36.447 INFO:teuthology.orchestra.run.smithi150.stdout:(61/176): postgresql-server-13.11-1.el9.x86_64. 12 MB/s | 5.7 MB 00:00 2024-03-04T08:44:36.539 INFO:teuthology.orchestra.run.smithi150.stdout:(62/176): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-03-04T08:44:36.545 INFO:teuthology.orchestra.run.smithi071.stdout:(13/176): libcephfs-devel-19.0.0-1739.g1bc250c7 288 kB/s | 31 kB 00:00 2024-03-04T08:44:36.581 INFO:teuthology.orchestra.run.smithi150.stdout:(63/176): python3-attrs-20.3.0-7.el9.noarch.rpm 1.8 MB/s | 78 kB 00:00 2024-03-04T08:44:36.779 INFO:teuthology.orchestra.run.smithi071.stdout:(14/176): libcephfs2-19.0.0-1739.g1bc250c7.el9. 3.1 MB/s | 747 kB 00:00 2024-03-04T08:44:36.956 INFO:teuthology.orchestra.run.smithi150.stdout:(64/176): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 5.8 MB 00:00 2024-03-04T08:44:36.979 INFO:teuthology.orchestra.run.smithi071.stdout:(15/176): ceph-osd-19.0.0-1739.g1bc250c7.el9.x8 6.7 MB/s | 17 MB 00:02 2024-03-04T08:44:37.004 INFO:teuthology.orchestra.run.smithi071.stdout:(16/176): libcephsqlite-19.0.0-1739.g1bc250c7.e 725 kB/s | 163 kB 00:00 2024-03-04T08:44:37.015 INFO:teuthology.orchestra.run.smithi150.stdout:(65/176): python3-jinja2-2.11.3-5.el9.noarch.rp 3.8 MB/s | 227 kB 00:00 2024-03-04T08:44:37.073 INFO:teuthology.orchestra.run.smithi150.stdout:(66/176): python3-jsonpatch-1.21-16.el9.noarch. 420 kB/s | 24 kB 00:00 2024-03-04T08:44:37.079 INFO:teuthology.orchestra.run.smithi071.stdout:(17/176): librados-devel-19.0.0-1739.g1bc250c7. 1.2 MB/s | 127 kB 00:00 2024-03-04T08:44:37.121 INFO:teuthology.orchestra.run.smithi071.stdout:(18/176): libradosstriper1-19.0.0-1739.g1bc250c 4.2 MB/s | 501 kB 00:00 2024-03-04T08:44:37.140 INFO:teuthology.orchestra.run.smithi150.stdout:(67/176): python3-jsonpointer-2.0-4.el9.noarch. 271 kB/s | 18 kB 00:00 2024-03-04T08:44:37.182 INFO:teuthology.orchestra.run.smithi150.stdout:(68/176): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-03-04T08:44:37.221 INFO:teuthology.orchestra.run.smithi071.stdout:(19/176): python3-ceph-argparse-19.0.0-1739.g1b 452 kB/s | 45 kB 00:00 2024-03-04T08:44:37.224 INFO:teuthology.orchestra.run.smithi150.stdout:(69/176): python3-markupsafe-1.1.1-12.el9.x86_6 759 kB/s | 32 kB 00:00 2024-03-04T08:44:37.320 INFO:teuthology.orchestra.run.smithi158.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-04T08:44:37.321 INFO:teuthology.orchestra.run.smithi158.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-04T08:44:37.335 INFO:teuthology.orchestra.run.smithi158.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-04T08:44:37.336 INFO:teuthology.orchestra.run.smithi158.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-04T08:44:37.337 INFO:teuthology.orchestra.run.smithi158.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-04T08:44:37.338 INFO:teuthology.orchestra.run.smithi158.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-04T08:44:37.340 INFO:teuthology.orchestra.run.smithi158.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-04T08:44:37.343 INFO:teuthology.orchestra.run.smithi158.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-04T08:44:37.344 INFO:teuthology.orchestra.run.smithi158.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-04T08:44:37.346 INFO:teuthology.orchestra.run.smithi071.stdout:(20/176): python3-ceph-common-19.0.0-1739.g1bc2 1.0 MB/s | 131 kB 00:00 2024-03-04T08:44:37.426 INFO:teuthology.orchestra.run.smithi158.stdout:Dependencies resolved. 2024-03-04T08:44:37.453 INFO:teuthology.orchestra.run.smithi158.stdout:============================================================================================ 2024-03-04T08:44:37.453 INFO:teuthology.orchestra.run.smithi158.stdout: Package Arch Version Repository Size 2024-03-04T08:44:37.454 INFO:teuthology.orchestra.run.smithi158.stdout:============================================================================================ 2024-03-04T08:44:37.454 INFO:teuthology.orchestra.run.smithi158.stdout:Installing: 2024-03-04T08:44:37.454 INFO:teuthology.orchestra.run.smithi158.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-04T08:44:37.454 INFO:teuthology.orchestra.run.smithi158.stdout: ceph x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 6.6 k 2024-03-04T08:44:37.454 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-base x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 5.3 M 2024-03-04T08:44:37.454 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-fuse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 888 k 2024-03-04T08:44:37.454 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 145 k 2024-03-04T08:44:37.454 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 1.5 M 2024-03-04T08:44:37.454 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 138 k 2024-03-04T08:44:37.454 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 1.8 M 2024-03-04T08:44:37.454 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 7.4 M 2024-03-04T08:44:37.454 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-rook noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 49 k 2024-03-04T08:44:37.455 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-radosgw x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 9.8 M 2024-03-04T08:44:37.455 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-test x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 47 M 2024-03-04T08:44:37.455 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-volume noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 264 k 2024-03-04T08:44:37.455 INFO:teuthology.orchestra.run.smithi158.stdout: cephadm noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 748 k 2024-03-04T08:44:37.455 INFO:teuthology.orchestra.run.smithi158.stdout: cephfs-mirror x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 220 k 2024-03-04T08:44:37.455 INFO:teuthology.orchestra.run.smithi158.stdout: cephfs-shell x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 27 k 2024-03-04T08:44:37.455 INFO:teuthology.orchestra.run.smithi158.stdout: cephfs-top noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 18 k 2024-03-04T08:44:37.455 INFO:teuthology.orchestra.run.smithi158.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-03-04T08:44:37.455 INFO:teuthology.orchestra.run.smithi158.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-04T08:44:37.455 INFO:teuthology.orchestra.run.smithi158.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-04T08:44:37.455 INFO:teuthology.orchestra.run.smithi158.stdout: libcephfs-devel x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 31 k 2024-03-04T08:44:37.455 INFO:teuthology.orchestra.run.smithi158.stdout: libcephfs2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 747 k 2024-03-04T08:44:37.455 INFO:teuthology.orchestra.run.smithi158.stdout: librados-devel x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 127 k 2024-03-04T08:44:37.456 INFO:teuthology.orchestra.run.smithi158.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-04T08:44:37.456 INFO:teuthology.orchestra.run.smithi158.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-03-04T08:44:37.456 INFO:teuthology.orchestra.run.smithi158.stdout: openssl-devel x86_64 1:3.0.7-27.el9 CentOS-AppStream 3.0 M 2024-03-04T08:44:37.456 INFO:teuthology.orchestra.run.smithi158.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-04T08:44:37.456 INFO:teuthology.orchestra.run.smithi158.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-04T08:44:37.456 INFO:teuthology.orchestra.run.smithi158.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-04T08:44:37.456 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cephfs x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 162 k 2024-03-04T08:44:37.456 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rados x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 321 k 2024-03-04T08:44:37.456 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rbd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 298 k 2024-03-04T08:44:37.456 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rgw x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 99 k 2024-03-04T08:44:37.456 INFO:teuthology.orchestra.run.smithi158.stdout: rbd-fuse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 85 k 2024-03-04T08:44:37.456 INFO:teuthology.orchestra.run.smithi158.stdout: rbd-mirror x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.1 M 2024-03-04T08:44:37.456 INFO:teuthology.orchestra.run.smithi158.stdout: rbd-nbd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 171 k 2024-03-04T08:44:37.457 INFO:teuthology.orchestra.run.smithi158.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-03-04T08:44:37.457 INFO:teuthology.orchestra.run.smithi158.stdout:Upgrading: 2024-03-04T08:44:37.457 INFO:teuthology.orchestra.run.smithi158.stdout: NetworkManager x86_64 1:1.46.0-1.el9 CentOS-BaseOS 2.3 M 2024-03-04T08:44:37.457 INFO:teuthology.orchestra.run.smithi158.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 CentOS-BaseOS 1.8 M 2024-03-04T08:44:37.457 INFO:teuthology.orchestra.run.smithi158.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 CentOS-BaseOS 41 k 2024-03-04T08:44:37.457 INFO:teuthology.orchestra.run.smithi158.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 CentOS-BaseOS 247 k 2024-03-04T08:44:37.457 INFO:teuthology.orchestra.run.smithi158.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-03-04T08:44:37.457 INFO:teuthology.orchestra.run.smithi158.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-03-04T08:44:37.457 INFO:teuthology.orchestra.run.smithi158.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-03-04T08:44:37.457 INFO:teuthology.orchestra.run.smithi158.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-03-04T08:44:37.457 INFO:teuthology.orchestra.run.smithi158.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-03-04T08:44:37.457 INFO:teuthology.orchestra.run.smithi158.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-03-04T08:44:37.457 INFO:teuthology.orchestra.run.smithi158.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-03-04T08:44:37.458 INFO:teuthology.orchestra.run.smithi158.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-03-04T08:44:37.458 INFO:teuthology.orchestra.run.smithi158.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-03-04T08:44:37.458 INFO:teuthology.orchestra.run.smithi158.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-03-04T08:44:37.458 INFO:teuthology.orchestra.run.smithi158.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-03-04T08:44:37.458 INFO:teuthology.orchestra.run.smithi158.stdout: librados2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.4 M 2024-03-04T08:44:37.458 INFO:teuthology.orchestra.run.smithi158.stdout: librbd1 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.1 M 2024-03-04T08:44:37.458 INFO:teuthology.orchestra.run.smithi158.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-03-04T08:44:37.458 INFO:teuthology.orchestra.run.smithi158.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-03-04T08:44:37.458 INFO:teuthology.orchestra.run.smithi158.stdout: openssl x86_64 1:3.0.7-27.el9 CentOS-BaseOS 1.2 M 2024-03-04T08:44:37.458 INFO:teuthology.orchestra.run.smithi158.stdout: openssl-libs x86_64 1:3.0.7-27.el9 CentOS-BaseOS 2.2 M 2024-03-04T08:44:37.458 INFO:teuthology.orchestra.run.smithi158.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-03-04T08:44:37.458 INFO:teuthology.orchestra.run.smithi158.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-03-04T08:44:37.458 INFO:teuthology.orchestra.run.smithi158.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-03-04T08:44:37.459 INFO:teuthology.orchestra.run.smithi158.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-04T08:44:37.459 INFO:teuthology.orchestra.run.smithi158.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-03-04T08:44:37.459 INFO:teuthology.orchestra.run.smithi158.stdout:Installing dependencies: 2024-03-04T08:44:37.459 INFO:teuthology.orchestra.run.smithi158.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-04T08:44:37.459 INFO:teuthology.orchestra.run.smithi158.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-04T08:44:37.459 INFO:teuthology.orchestra.run.smithi158.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-04T08:44:37.459 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-common x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 20 M 2024-03-04T08:44:37.459 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 31 k 2024-03-04T08:44:37.459 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mds x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 2.3 M 2024-03-04T08:44:37.459 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 248 k 2024-03-04T08:44:37.459 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mon x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 4.7 M 2024-03-04T08:44:37.459 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-osd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 17 M 2024-03-04T08:44:37.459 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 15 k 2024-03-04T08:44:37.459 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-selinux x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 25 k 2024-03-04T08:44:37.460 INFO:teuthology.orchestra.run.smithi158.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-04T08:44:37.460 INFO:teuthology.orchestra.run.smithi158.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-04T08:44:37.460 INFO:teuthology.orchestra.run.smithi158.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-04T08:44:37.460 INFO:teuthology.orchestra.run.smithi158.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-04T08:44:37.460 INFO:teuthology.orchestra.run.smithi158.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-04T08:44:37.460 INFO:teuthology.orchestra.run.smithi158.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-04T08:44:37.460 INFO:teuthology.orchestra.run.smithi158.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-03-04T08:44:37.460 INFO:teuthology.orchestra.run.smithi158.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-04T08:44:37.460 INFO:teuthology.orchestra.run.smithi158.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-04T08:44:37.460 INFO:teuthology.orchestra.run.smithi158.stdout: libcephsqlite x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 163 k 2024-03-04T08:44:37.460 INFO:teuthology.orchestra.run.smithi158.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-03-04T08:44:37.460 INFO:teuthology.orchestra.run.smithi158.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-04T08:44:37.460 INFO:teuthology.orchestra.run.smithi158.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-04T08:44:37.461 INFO:teuthology.orchestra.run.smithi158.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-03-04T08:44:37.461 INFO:teuthology.orchestra.run.smithi158.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-04T08:44:37.461 INFO:teuthology.orchestra.run.smithi158.stdout: libradosstriper1 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 501 k 2024-03-04T08:44:37.461 INFO:teuthology.orchestra.run.smithi158.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-04T08:44:37.461 INFO:teuthology.orchestra.run.smithi158.stdout: librgw2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 5.1 M 2024-03-04T08:44:37.461 INFO:teuthology.orchestra.run.smithi158.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-04T08:44:37.461 INFO:teuthology.orchestra.run.smithi158.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-04T08:44:37.461 INFO:teuthology.orchestra.run.smithi158.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-04T08:44:37.461 INFO:teuthology.orchestra.run.smithi158.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-04T08:44:37.461 INFO:teuthology.orchestra.run.smithi158.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-04T08:44:37.461 INFO:teuthology.orchestra.run.smithi158.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-04T08:44:37.461 INFO:teuthology.orchestra.run.smithi158.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-03-04T08:44:37.461 INFO:teuthology.orchestra.run.smithi158.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-04T08:44:37.462 INFO:teuthology.orchestra.run.smithi158.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-04T08:44:37.462 INFO:teuthology.orchestra.run.smithi158.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-04T08:44:37.462 INFO:teuthology.orchestra.run.smithi158.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-04T08:44:37.462 INFO:teuthology.orchestra.run.smithi158.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-04T08:44:37.462 INFO:teuthology.orchestra.run.smithi158.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-04T08:44:37.462 INFO:teuthology.orchestra.run.smithi158.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-04T08:44:37.462 INFO:teuthology.orchestra.run.smithi158.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-04T08:44:37.462 INFO:teuthology.orchestra.run.smithi158.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-04T08:44:37.462 INFO:teuthology.orchestra.run.smithi158.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-04T08:44:37.462 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-04T08:44:37.462 INFO:teuthology.orchestra.run.smithi158.stdout: python3-ceph-argparse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 45 k 2024-03-04T08:44:37.462 INFO:teuthology.orchestra.run.smithi158.stdout: python3-ceph-common x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 131 k 2024-03-04T08:44:37.462 INFO:teuthology.orchestra.run.smithi158.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-04T08:44:37.463 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-04T08:44:37.463 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-04T08:44:37.463 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-04T08:44:37.463 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-04T08:44:37.463 INFO:teuthology.orchestra.run.smithi158.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-04T08:44:37.463 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-04T08:44:37.463 INFO:teuthology.orchestra.run.smithi158.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-04T08:44:37.463 INFO:teuthology.orchestra.run.smithi158.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-04T08:44:37.463 INFO:teuthology.orchestra.run.smithi158.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-04T08:44:37.463 INFO:teuthology.orchestra.run.smithi158.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-04T08:44:37.463 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-04T08:44:37.463 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-04T08:44:37.463 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-04T08:44:37.463 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-04T08:44:37.464 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-04T08:44:37.464 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-03-04T08:44:37.464 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-04T08:44:37.464 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-04T08:44:37.464 INFO:teuthology.orchestra.run.smithi158.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-04T08:44:37.464 INFO:teuthology.orchestra.run.smithi158.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-04T08:44:37.464 INFO:teuthology.orchestra.run.smithi158.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-04T08:44:37.464 INFO:teuthology.orchestra.run.smithi158.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-04T08:44:37.464 INFO:teuthology.orchestra.run.smithi158.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-04T08:44:37.464 INFO:teuthology.orchestra.run.smithi158.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-04T08:44:37.464 INFO:teuthology.orchestra.run.smithi158.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-04T08:44:37.464 INFO:teuthology.orchestra.run.smithi158.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-04T08:44:37.464 INFO:teuthology.orchestra.run.smithi158.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-04T08:44:37.464 INFO:teuthology.orchestra.run.smithi158.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-04T08:44:37.465 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-04T08:44:37.465 INFO:teuthology.orchestra.run.smithi158.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-04T08:44:37.465 INFO:teuthology.orchestra.run.smithi158.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-04T08:44:37.465 INFO:teuthology.orchestra.run.smithi158.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-04T08:44:37.465 INFO:teuthology.orchestra.run.smithi158.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-04T08:44:37.465 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-04T08:44:37.465 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-04T08:44:37.465 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-04T08:44:37.465 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-04T08:44:37.465 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-04T08:44:37.465 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-04T08:44:37.465 INFO:teuthology.orchestra.run.smithi158.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-04T08:44:37.465 INFO:teuthology.orchestra.run.smithi158.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-03-04T08:44:37.465 INFO:teuthology.orchestra.run.smithi158.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-04T08:44:37.466 INFO:teuthology.orchestra.run.smithi158.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-04T08:44:37.466 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-04T08:44:37.466 INFO:teuthology.orchestra.run.smithi158.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-04T08:44:37.466 INFO:teuthology.orchestra.run.smithi158.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-04T08:44:37.466 INFO:teuthology.orchestra.run.smithi158.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-04T08:44:37.466 INFO:teuthology.orchestra.run.smithi158.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-04T08:44:37.466 INFO:teuthology.orchestra.run.smithi158.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-04T08:44:37.466 INFO:teuthology.orchestra.run.smithi158.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-04T08:44:37.466 INFO:teuthology.orchestra.run.smithi158.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-04T08:44:37.466 INFO:teuthology.orchestra.run.smithi158.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-04T08:44:37.466 INFO:teuthology.orchestra.run.smithi158.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-04T08:44:37.466 INFO:teuthology.orchestra.run.smithi158.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-04T08:44:37.466 INFO:teuthology.orchestra.run.smithi158.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-03-04T08:44:37.466 INFO:teuthology.orchestra.run.smithi158.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-03-04T08:44:37.467 INFO:teuthology.orchestra.run.smithi158.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-04T08:44:37.467 INFO:teuthology.orchestra.run.smithi158.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-04T08:44:37.467 INFO:teuthology.orchestra.run.smithi158.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-04T08:44:37.467 INFO:teuthology.orchestra.run.smithi158.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-04T08:44:37.467 INFO:teuthology.orchestra.run.smithi158.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-04T08:44:37.467 INFO:teuthology.orchestra.run.smithi158.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-04T08:44:37.467 INFO:teuthology.orchestra.run.smithi158.stdout:Installing weak dependencies: 2024-03-04T08:44:37.467 INFO:teuthology.orchestra.run.smithi158.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-03-04T08:44:37.467 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-03-04T08:44:37.467 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction Summary 2024-03-04T08:44:37.467 INFO:teuthology.orchestra.run.smithi158.stdout:============================================================================================ 2024-03-04T08:44:37.467 INFO:teuthology.orchestra.run.smithi158.stdout:Install 150 Packages 2024-03-04T08:44:37.467 INFO:teuthology.orchestra.run.smithi158.stdout:Upgrade 26 Packages 2024-03-04T08:44:37.467 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-03-04T08:44:37.468 INFO:teuthology.orchestra.run.smithi071.stdout:(21/176): python3-cephfs-19.0.0-1739.g1bc250c7. 1.5 MB/s | 162 kB 00:00 2024-03-04T08:44:37.468 INFO:teuthology.orchestra.run.smithi158.stdout:Total download size: 224 M 2024-03-04T08:44:37.468 INFO:teuthology.orchestra.run.smithi158.stdout:Downloading Packages: 2024-03-04T08:44:37.483 INFO:teuthology.orchestra.run.smithi150.stdout:(70/176): ceph-base-19.0.0-1739.g1bc250c7.el9.x 498 kB/s | 5.3 MB 00:10 2024-03-04T08:44:37.550 INFO:teuthology.orchestra.run.smithi150.stdout:(71/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 4.8 MB 00:00 2024-03-04T08:44:37.575 INFO:teuthology.orchestra.run.smithi150.stdout:(72/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 217 kB 00:00 2024-03-04T08:44:37.580 INFO:teuthology.orchestra.run.smithi071.stdout:(22/176): python3-rados-19.0.0-1739.g1bc250c7.e 2.5 MB/s | 321 kB 00:00 2024-03-04T08:44:37.615 INFO:teuthology.orchestra.run.smithi150.stdout:(73/176): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 174 kB 00:00 2024-03-04T08:44:37.641 INFO:teuthology.orchestra.run.smithi150.stdout:(74/176): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-03-04T08:44:37.674 INFO:teuthology.orchestra.run.smithi150.stdout:(75/176): python3-prettytable-0.7.2-27.el9.noar 701 kB/s | 41 kB 00:00 2024-03-04T08:44:37.698 INFO:teuthology.orchestra.run.smithi071.stdout:(23/176): python3-rbd-19.0.0-1739.g1bc250c7.el9 2.5 MB/s | 298 kB 00:00 2024-03-04T08:44:37.717 INFO:teuthology.orchestra.run.smithi150.stdout:(76/176): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-04T08:44:37.742 INFO:teuthology.orchestra.run.smithi150.stdout:(77/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-03-04T08:44:37.790 INFO:teuthology.orchestra.run.smithi071.stdout:(24/176): python3-rgw-19.0.0-1739.g1bc250c7.el9 1.1 MB/s | 99 kB 00:00 2024-03-04T08:44:37.851 INFO:teuthology.orchestra.run.smithi150.stdout:(78/176): python3-pyasn1-modules-0.4.8-6.el9.no 1.5 MB/s | 211 kB 00:00 2024-03-04T08:44:37.876 INFO:teuthology.orchestra.run.smithi150.stdout:(79/176): python3-pytz-2021.1-5.el9.noarch.rpm 355 kB/s | 47 kB 00:00 2024-03-04T08:44:37.902 INFO:teuthology.orchestra.run.smithi150.stdout:(80/176): python3-requests-oauthlib-1.3.0-12.el 854 kB/s | 43 kB 00:00 2024-03-04T08:44:37.981 INFO:teuthology.orchestra.run.smithi071.stdout:(25/176): librgw2-19.0.0-1739.g1bc250c7.el9.x86 5.7 MB/s | 5.1 MB 00:00 2024-03-04T08:44:38.007 INFO:teuthology.orchestra.run.smithi071.stdout:(26/176): rbd-fuse-19.0.0-1739.g1bc250c7.el9.x8 393 kB/s | 85 kB 00:00 2024-03-04T08:44:38.135 INFO:teuthology.orchestra.run.smithi150.stdout:(81/176): python3-toml-0.10.2-6.el9.noarch.rpm 166 kB/s | 39 kB 00:00 2024-03-04T08:44:38.157 INFO:teuthology.orchestra.run.smithi071.stdout:(27/176): rbd-nbd-19.0.0-1739.g1bc250c7.el9.x86 1.1 MB/s | 171 kB 00:00 2024-03-04T08:44:38.207 INFO:teuthology.orchestra.run.smithi158.stdout:(1/176): ceph-19.0.0-1739.g1bc250c7.el9.x86_64. 37 kB/s | 6.6 kB 00:00 2024-03-04T08:44:38.257 INFO:teuthology.orchestra.run.smithi071.stdout:(28/176): ceph-grafana-dashboards-19.0.0-1739.g 314 kB/s | 31 kB 00:00 2024-03-04T08:44:38.374 INFO:teuthology.orchestra.run.smithi071.stdout:(29/176): ceph-mgr-cephadm-19.0.0-1739.g1bc250c 1.2 MB/s | 138 kB 00:00 2024-03-04T08:44:38.527 INFO:teuthology.orchestra.run.smithi150.stdout:(82/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 105 kB/s | 41 kB 00:00 2024-03-04T08:44:38.683 INFO:teuthology.orchestra.run.smithi158.stdout:(2/176): ceph-fuse-19.0.0-1739.g1bc250c7.el9.x8 1.8 MB/s | 888 kB 00:00 2024-03-04T08:44:38.741 INFO:teuthology.orchestra.run.smithi071.stdout:(30/176): rbd-mirror-19.0.0-1739.g1bc250c7.el9. 4.1 MB/s | 3.1 MB 00:00 2024-03-04T08:44:38.783 INFO:teuthology.orchestra.run.smithi158.stdout:(3/176): ceph-immutable-object-cache-19.0.0-173 1.4 MB/s | 145 kB 00:00 2024-03-04T08:44:38.783 INFO:teuthology.orchestra.run.smithi071.stdout:(31/176): ceph-mgr-dashboard-19.0.0-1739.g1bc25 4.4 MB/s | 1.8 MB 00:00 2024-03-04T08:44:38.861 INFO:teuthology.orchestra.run.smithi150.stdout:(83/176): qatlib-23.11.0-1.el9.x86_64.rpm 651 kB/s | 215 kB 00:00 2024-03-04T08:44:38.883 INFO:teuthology.orchestra.run.smithi071.stdout:(32/176): ceph-mgr-modules-core-19.0.0-1739.g1b 2.4 MB/s | 248 kB 00:00 2024-03-04T08:44:39.009 INFO:teuthology.orchestra.run.smithi071.stdout:(33/176): ceph-mgr-rook-19.0.0-1739.g1bc250c7.e 394 kB/s | 49 kB 00:00 2024-03-04T08:44:39.011 INFO:teuthology.orchestra.run.smithi150.stdout:(84/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 14 MB/s | 16 MB 00:01 2024-03-04T08:44:39.028 INFO:teuthology.orchestra.run.smithi150.stdout:(85/176): qatlib-service-23.11.0-1.el9.x86_64.r 203 kB/s | 34 kB 00:00 2024-03-04T08:44:39.078 INFO:teuthology.orchestra.run.smithi150.stdout:(86/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 654 kB/s | 43 kB 00:00 2024-03-04T08:44:39.112 INFO:teuthology.orchestra.run.smithi150.stdout:(87/176): socat-1.7.4.1-5.el9.x86_64.rpm 3.5 MB/s | 300 kB 00:00 2024-03-04T08:44:39.146 INFO:teuthology.orchestra.run.smithi150.stdout:(88/176): uuid-1.6.2-55.el9.x86_64.rpm 842 kB/s | 56 kB 00:00 2024-03-04T08:44:39.151 INFO:teuthology.orchestra.run.smithi071.stdout:(34/176): ceph-prometheus-alerts-19.0.0-1739.g1 108 kB/s | 15 kB 00:00 2024-03-04T08:44:39.179 INFO:teuthology.orchestra.run.smithi150.stdout:(89/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.3 MB/s | 87 kB 00:00 2024-03-04T08:44:39.192 INFO:teuthology.orchestra.run.smithi158.stdout:(4/176): ceph-base-19.0.0-1739.g1bc250c7.el9.x8 4.6 MB/s | 5.3 MB 00:01 2024-03-04T08:44:39.204 INFO:teuthology.orchestra.run.smithi150.stdout:(90/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.0 MB/s | 63 kB 00:00 2024-03-04T08:44:39.230 INFO:teuthology.orchestra.run.smithi150.stdout:(91/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 878 kB/s | 44 kB 00:00 2024-03-04T08:44:39.243 INFO:teuthology.orchestra.run.smithi071.stdout:(35/176): ceph-volume-19.0.0-1739.g1bc250c7.el9 2.8 MB/s | 264 kB 00:00 2024-03-04T08:44:39.255 INFO:teuthology.orchestra.run.smithi150.stdout:(92/176): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-04T08:44:39.289 INFO:teuthology.orchestra.run.smithi150.stdout:(93/176): jq-1.6-16.el9.x86_64.rpm 3.1 MB/s | 186 kB 00:00 2024-03-04T08:44:39.339 INFO:teuthology.orchestra.run.smithi150.stdout:(94/176): libgfortran-11.4.1-3.el9.x86_64.rpm 9.4 MB/s | 802 kB 00:00 2024-03-04T08:44:39.364 INFO:teuthology.orchestra.run.smithi150.stdout:(95/176): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 194 kB 00:00 2024-03-04T08:44:39.398 INFO:teuthology.orchestra.run.smithi150.stdout:(96/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.6 MB/s | 217 kB 00:00 2024-03-04T08:44:39.423 INFO:teuthology.orchestra.run.smithi150.stdout:(97/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.0 MB/s | 241 kB 00:00 2024-03-04T08:44:39.475 INFO:teuthology.orchestra.run.smithi158.stdout:(5/176): ceph-mgr-19.0.0-1739.g1bc250c7.el9.x86 5.3 MB/s | 1.5 MB 00:00 2024-03-04T08:44:39.523 INFO:teuthology.orchestra.run.smithi150.stdout:(98/176): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.1 MB 00:00 2024-03-04T08:44:39.543 INFO:teuthology.orchestra.run.smithi071.stdout:(36/176): cephadm-19.0.0-1739.g1bc250c7.el9.noa 2.4 MB/s | 748 kB 00:00 2024-03-04T08:44:39.549 INFO:teuthology.orchestra.run.smithi150.stdout:(99/176): python3-ply-3.11-14.el9.noarch.rpm 825 kB/s | 103 kB 00:00 2024-03-04T08:44:39.590 INFO:teuthology.orchestra.run.smithi150.stdout:(100/176): python3-pycparser-2.20-6.el9.noarch. 1.8 MB/s | 124 kB 00:00 2024-03-04T08:44:39.616 INFO:teuthology.orchestra.run.smithi150.stdout:(101/176): python3-requests-2.25.1-8.el9.noarch 1.7 MB/s | 113 kB 00:00 2024-03-04T08:44:39.626 INFO:teuthology.orchestra.run.smithi071.stdout:(37/176): cephfs-top-19.0.0-1739.g1bc250c7.el9. 219 kB/s | 18 kB 00:00 2024-03-04T08:44:39.641 INFO:teuthology.orchestra.run.smithi150.stdout:(102/176): python3-urllib3-1.26.5-5.el9.noarch. 3.7 MB/s | 187 kB 00:00 2024-03-04T08:44:39.777 INFO:teuthology.orchestra.run.smithi071.stdout:(38/176): automake-1.16.2-8.el9.noarch.rpm 4.3 MB/s | 662 kB 00:00 2024-03-04T08:44:39.825 INFO:teuthology.orchestra.run.smithi150.stdout:(103/176): lua-devel-5.4.4-4.el9.x86_64.rpm 122 kB/s | 22 kB 00:00 2024-03-04T08:44:39.835 INFO:teuthology.orchestra.run.smithi071.stdout:(39/176): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 920 kB 00:00 2024-03-04T08:44:39.869 INFO:teuthology.orchestra.run.smithi071.stdout:(40/176): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-03-04T08:44:39.903 INFO:teuthology.orchestra.run.smithi071.stdout:(41/176): elfutils-libelf-devel-0.190-2.el9.x86 613 kB/s | 20 kB 00:00 2024-03-04T08:44:39.917 INFO:teuthology.orchestra.run.smithi150.stdout:(104/176): python3-devel-3.9.17-2.el9.x86_64.rp 818 kB/s | 245 kB 00:00 2024-03-04T08:44:39.953 INFO:teuthology.orchestra.run.smithi071.stdout:(42/176): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-04T08:44:39.987 INFO:teuthology.orchestra.run.smithi071.stdout:(43/176): flexiblas-3.0.4-8.el9.x86_64.rpm 945 kB/s | 32 kB 00:00 2024-03-04T08:44:39.992 INFO:teuthology.orchestra.run.smithi150.stdout:(105/176): protobuf-compiler-3.14.0-13.el9.x86_ 5.1 MB/s | 863 kB 00:00 2024-03-04T08:44:40.009 INFO:teuthology.orchestra.run.smithi158.stdout:(6/176): ceph-mds-19.0.0-1739.g1bc250c7.el9.x86 1.9 MB/s | 2.3 MB 00:01 2024-03-04T08:44:40.095 INFO:teuthology.orchestra.run.smithi071.stdout:(44/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-03-04T08:44:40.129 INFO:teuthology.orchestra.run.smithi071.stdout:(45/176): flexiblas-openblas-openmp-3.0.4-8.el9 515 kB/s | 17 kB 00:00 2024-03-04T08:44:40.162 INFO:teuthology.orchestra.run.smithi071.stdout:(46/176): libaio-devel-0.3.111-13.el9.x86_64.rp 302 kB/s | 10 kB 00:00 2024-03-04T08:44:40.205 INFO:teuthology.orchestra.run.smithi071.stdout:(47/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-03-04T08:44:40.226 INFO:teuthology.orchestra.run.smithi150.stdout:(106/176): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-03-04T08:44:40.246 INFO:teuthology.orchestra.run.smithi071.stdout:(48/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-04T08:44:40.251 INFO:teuthology.orchestra.run.smithi150.stdout:(107/176): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-03-04T08:44:40.284 INFO:teuthology.orchestra.run.smithi150.stdout:(108/176): grpc-data-1.46.7-10.el9.noarch.rpm 333 kB/s | 19 kB 00:00 2024-03-04T08:44:40.305 INFO:teuthology.orchestra.run.smithi071.stdout:(49/176): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-04T08:44:40.335 INFO:teuthology.orchestra.run.smithi150.stdout:(109/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-03-04T08:44:40.363 INFO:teuthology.orchestra.run.smithi071.stdout:(50/176): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 578 kB 00:00 2024-03-04T08:44:40.367 INFO:teuthology.orchestra.run.smithi158.stdout:(7/176): ceph-mon-19.0.0-1739.g1bc250c7.el9.x86 5.3 MB/s | 4.7 MB 00:00 2024-03-04T08:44:40.385 INFO:teuthology.orchestra.run.smithi150.stdout:(110/176): liboath-2.6.7-2.el9.x86_64.rpm 974 kB/s | 49 kB 00:00 2024-03-04T08:44:40.405 INFO:teuthology.orchestra.run.smithi071.stdout:(51/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 585 kB/s | 24 kB 00:00 2024-03-04T08:44:40.444 INFO:teuthology.orchestra.run.smithi150.stdout:(111/176): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-03-04T08:44:40.456 INFO:teuthology.orchestra.run.smithi071.stdout:(52/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 926 kB/s | 46 kB 00:00 2024-03-04T08:44:40.502 INFO:teuthology.orchestra.run.smithi150.stdout:(112/176): libarrow-9.0.0-10.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-03-04T08:44:40.528 INFO:teuthology.orchestra.run.smithi150.stdout:(113/176): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-03-04T08:44:40.531 INFO:teuthology.orchestra.run.smithi071.stdout:(53/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-04T08:44:40.570 INFO:teuthology.orchestra.run.smithi150.stdout:(114/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2024-03-04T08:44:40.603 INFO:teuthology.orchestra.run.smithi150.stdout:(115/176): python3-asyncssh-2.13.2-4.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-03-04T08:44:40.629 INFO:teuthology.orchestra.run.smithi150.stdout:(116/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 743 kB/s | 43 kB 00:00 2024-03-04T08:44:40.663 INFO:teuthology.orchestra.run.smithi150.stdout:(117/176): python3-cachetools-4.2.4-1.el9.noarc 549 kB/s | 32 kB 00:00 2024-03-04T08:44:40.665 INFO:teuthology.orchestra.run.smithi071.stdout:(54/176): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 187 kB 00:00 2024-03-04T08:44:40.688 INFO:teuthology.orchestra.run.smithi150.stdout:(118/176): python3-certifi-2021.10.8-2.el9.noar 255 kB/s | 15 kB 00:00 2024-03-04T08:44:40.707 INFO:teuthology.orchestra.run.smithi071.stdout:(55/176): openblas-0.3.21-2.el9.x86_64.rpm 822 kB/s | 34 kB 00:00 2024-03-04T08:44:40.721 INFO:teuthology.orchestra.run.smithi150.stdout:(119/176): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-03-04T08:44:40.755 INFO:teuthology.orchestra.run.smithi150.stdout:(120/176): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-03-04T08:44:40.788 INFO:teuthology.orchestra.run.smithi150.stdout:(121/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-03-04T08:44:40.814 INFO:teuthology.orchestra.run.smithi150.stdout:(122/176): python3-colorama-0.4.4-12.el9.noarch 603 kB/s | 35 kB 00:00 2024-03-04T08:44:40.840 INFO:teuthology.orchestra.run.smithi071.stdout:(56/176): openblas-openmp-0.3.21-2.el9.x86_64.r 36 MB/s | 4.7 MB 00:00 2024-03-04T08:44:40.847 INFO:teuthology.orchestra.run.smithi150.stdout:(123/176): python3-google-auth-2.28.0-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-03-04T08:44:40.908 INFO:teuthology.orchestra.run.smithi150.stdout:(124/176): python3-grpcio-tools-1.46.7-10.el9.x 2.3 MB/s | 144 kB 00:00 2024-03-04T08:44:40.941 INFO:teuthology.orchestra.run.smithi071.stdout:(57/176): openssl-devel-3.0.7-27.el9.x86_64.rpm 30 MB/s | 3.0 MB 00:00 2024-03-04T08:44:40.950 INFO:teuthology.orchestra.run.smithi150.stdout:(125/176): python3-grpcio-1.46.7-10.el9.x86_64. 15 MB/s | 2.0 MB 00:00 2024-03-04T08:44:40.975 INFO:teuthology.orchestra.run.smithi150.stdout:(126/176): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-03-04T08:44:40.999 INFO:teuthology.orchestra.run.smithi071.stdout:(58/176): postgresql-13.11-1.el9.x86_64.rpm 26 MB/s | 1.5 MB 00:00 2024-03-04T08:44:41.009 INFO:teuthology.orchestra.run.smithi150.stdout:(127/176): python3-jaraco-classes-3.2.1-5.el9.n 303 kB/s | 18 kB 00:00 2024-03-04T08:44:41.034 INFO:teuthology.orchestra.run.smithi150.stdout:(128/176): python3-jaraco-collections-3.0.0-8.e 396 kB/s | 23 kB 00:00 2024-03-04T08:44:41.049 INFO:teuthology.orchestra.run.smithi071.stdout:(59/176): postgresql-contrib-13.11-1.el9.x86_64 16 MB/s | 811 kB 00:00 2024-03-04T08:44:41.059 INFO:teuthology.orchestra.run.smithi150.stdout:(129/176): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-03-04T08:44:41.083 INFO:teuthology.orchestra.run.smithi071.stdout:(60/176): postgresql-private-libs-13.11-1.el9.x 3.9 MB/s | 134 kB 00:00 2024-03-04T08:44:41.084 INFO:teuthology.orchestra.run.smithi150.stdout:(130/176): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-03-04T08:44:41.143 INFO:teuthology.orchestra.run.smithi150.stdout:(131/176): python3-logutils-0.3.5-21.el9.noarch 794 kB/s | 46 kB 00:00 2024-03-04T08:44:41.177 INFO:teuthology.orchestra.run.smithi150.stdout:(132/176): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-03-04T08:44:41.202 INFO:teuthology.orchestra.run.smithi150.stdout:(133/176): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-03-04T08:44:41.234 INFO:teuthology.orchestra.run.smithi071.stdout:(61/176): postgresql-server-13.11-1.el9.x86_64. 38 MB/s | 5.7 MB 00:00 2024-03-04T08:44:41.236 INFO:teuthology.orchestra.run.smithi150.stdout:(134/176): python3-natsort-7.1.1-5.el9.noarch.r 987 kB/s | 58 kB 00:00 2024-03-04T08:44:41.261 INFO:teuthology.orchestra.run.smithi150.stdout:(135/176): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-03-04T08:44:41.286 INFO:teuthology.orchestra.run.smithi150.stdout:(136/176): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-03-04T08:44:41.292 INFO:teuthology.orchestra.run.smithi071.stdout:(62/176): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-03-04T08:44:41.320 INFO:teuthology.orchestra.run.smithi150.stdout:(137/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-03-04T08:44:41.327 INFO:teuthology.orchestra.run.smithi071.stdout:(63/176): python3-attrs-20.3.0-7.el9.noarch.rpm 2.3 MB/s | 78 kB 00:00 2024-03-04T08:44:41.337 INFO:teuthology.orchestra.run.smithi150.stdout:(138/176): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-03-04T08:44:41.370 INFO:teuthology.orchestra.run.smithi150.stdout:(139/176): python3-repoze-lru-0.7-16.el9.noarch 616 kB/s | 31 kB 00:00 2024-03-04T08:44:41.396 INFO:teuthology.orchestra.run.smithi150.stdout:(140/176): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-03-04T08:44:41.421 INFO:teuthology.orchestra.run.smithi150.stdout:(141/176): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-03-04T08:44:41.446 INFO:teuthology.orchestra.run.smithi150.stdout:(142/176): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-03-04T08:44:41.459 INFO:teuthology.orchestra.run.smithi071.stdout:(64/176): ceph-mgr-diskprediction-local-19.0.0- 2.7 MB/s | 7.4 MB 00:02 2024-03-04T08:44:41.480 INFO:teuthology.orchestra.run.smithi150.stdout:(143/176): python3-typing-extensions-4.4.0-2.el 873 kB/s | 51 kB 00:00 2024-03-04T08:44:41.501 INFO:teuthology.orchestra.run.smithi158.stdout:(8/176): ceph-radosgw-19.0.0-1739.g1bc250c7.el9 8.6 MB/s | 9.8 MB 00:01 2024-03-04T08:44:41.515 INFO:teuthology.orchestra.run.smithi150.stdout:(144/176): python3-webob-1.8.7-6.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-03-04T08:44:41.535 INFO:teuthology.orchestra.run.smithi071.stdout:(65/176): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 5.8 MB 00:00 2024-03-04T08:44:41.539 INFO:teuthology.orchestra.run.smithi150.stdout:(145/176): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-03-04T08:44:41.560 INFO:teuthology.orchestra.run.smithi158.stdout:(9/176): ceph-selinux-19.0.0-1739.g1bc250c7.el9 433 kB/s | 25 kB 00:00 2024-03-04T08:44:41.560 INFO:teuthology.orchestra.run.smithi071.stdout:(66/176): python3-jinja2-2.11.3-5.el9.noarch.rp 2.3 MB/s | 227 kB 00:00 2024-03-04T08:44:41.581 INFO:teuthology.orchestra.run.smithi150.stdout:(146/176): python3-werkzeug-2.0.3-3.el9.1.noarc 6.3 MB/s | 427 kB 00:00 2024-03-04T08:44:41.585 INFO:teuthology.orchestra.run.smithi071.stdout:(67/176): python3-jsonpatch-1.21-16.el9.noarch. 490 kB/s | 24 kB 00:00 2024-03-04T08:44:41.606 INFO:teuthology.orchestra.run.smithi150.stdout:(147/176): python3-zc-lockfile-2.0-10.el9.noarc 298 kB/s | 20 kB 00:00 2024-03-04T08:44:41.611 INFO:teuthology.orchestra.run.smithi071.stdout:(68/176): python3-jsonpointer-2.0-4.el9.noarch. 360 kB/s | 18 kB 00:00 2024-03-04T08:44:41.636 INFO:teuthology.orchestra.run.smithi071.stdout:(69/176): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-04T08:44:41.640 INFO:teuthology.orchestra.run.smithi150.stdout:(148/176): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-03-04T08:44:41.661 INFO:teuthology.orchestra.run.smithi071.stdout:(70/176): python3-markupsafe-1.1.1-12.el9.x86_6 632 kB/s | 32 kB 00:00 2024-03-04T08:44:41.707 INFO:teuthology.orchestra.run.smithi150.stdout:(149/176): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-03-04T08:44:41.778 INFO:teuthology.orchestra.run.smithi071.stdout:(71/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2024-03-04T08:44:41.804 INFO:teuthology.orchestra.run.smithi071.stdout:(72/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-03-04T08:44:41.829 INFO:teuthology.orchestra.run.smithi071.stdout:(73/176): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-04T08:44:41.846 INFO:teuthology.orchestra.run.smithi071.stdout:(74/176): python3-packaging-20.9-5.el9.noarch.r 1.6 MB/s | 69 kB 00:00 2024-03-04T08:44:41.871 INFO:teuthology.orchestra.run.smithi071.stdout:(75/176): python3-prettytable-0.7.2-27.el9.noar 988 kB/s | 41 kB 00:00 2024-03-04T08:44:41.896 INFO:teuthology.orchestra.run.smithi071.stdout:(76/176): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-03-04T08:44:41.922 INFO:teuthology.orchestra.run.smithi071.stdout:(77/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-04T08:44:41.947 INFO:teuthology.orchestra.run.smithi071.stdout:(78/176): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-04T08:44:41.972 INFO:teuthology.orchestra.run.smithi071.stdout:(79/176): python3-pytz-2021.1-5.el9.noarch.rpm 952 kB/s | 47 kB 00:00 2024-03-04T08:44:41.997 INFO:teuthology.orchestra.run.smithi071.stdout:(80/176): python3-requests-oauthlib-1.3.0-12.el 853 kB/s | 43 kB 00:00 2024-03-04T08:44:42.122 INFO:teuthology.orchestra.run.smithi071.stdout:(81/176): python3-toml-0.10.2-6.el9.noarch.rpm 314 kB/s | 39 kB 00:00 2024-03-04T08:44:42.249 INFO:teuthology.orchestra.run.smithi150.stdout:(150/176): librbd1-19.0.0-1739.g1bc250c7.el9.x8 5.8 MB/s | 3.1 MB 00:00 2024-03-04T08:44:42.282 INFO:teuthology.orchestra.run.smithi150.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2024-03-04T08:44:42.298 INFO:teuthology.orchestra.run.smithi071.stdout:(82/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 48 MB/s | 16 MB 00:00 2024-03-04T08:44:42.323 INFO:teuthology.orchestra.run.smithi071.stdout:(83/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 207 kB/s | 41 kB 00:00 2024-03-04T08:44:42.348 INFO:teuthology.orchestra.run.smithi071.stdout:(84/176): qatlib-23.11.0-1.el9.x86_64.rpm 4.2 MB/s | 215 kB 00:00 2024-03-04T08:44:42.365 INFO:teuthology.orchestra.run.smithi071.stdout:(85/176): qatlib-service-23.11.0-1.el9.x86_64.r 815 kB/s | 34 kB 00:00 2024-03-04T08:44:42.382 INFO:teuthology.orchestra.run.smithi071.stdout:(86/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 1.3 MB/s | 43 kB 00:00 2024-03-04T08:44:42.408 INFO:teuthology.orchestra.run.smithi071.stdout:(87/176): socat-1.7.4.1-5.el9.x86_64.rpm 7.0 MB/s | 300 kB 00:00 2024-03-04T08:44:42.416 INFO:teuthology.orchestra.run.smithi150.stdout:(152/176): NetworkManager-1.46.0-1.el9.x86_64.r 17 MB/s | 2.3 MB 00:00 2024-03-04T08:44:42.425 INFO:teuthology.orchestra.run.smithi071.stdout:(88/176): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 56 kB 00:00 2024-03-04T08:44:42.450 INFO:teuthology.orchestra.run.smithi071.stdout:(89/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2024-03-04T08:44:42.475 INFO:teuthology.orchestra.run.smithi071.stdout:(90/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-04T08:44:42.500 INFO:teuthology.orchestra.run.smithi150.stdout:(153/176): NetworkManager-libnm-1.46.0-1.el9.x8 22 MB/s | 1.8 MB 00:00 2024-03-04T08:44:42.501 INFO:teuthology.orchestra.run.smithi071.stdout:(91/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 874 kB/s | 44 kB 00:00 2024-03-04T08:44:42.526 INFO:teuthology.orchestra.run.smithi071.stdout:(92/176): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-04T08:44:42.542 INFO:teuthology.orchestra.run.smithi150.stdout:(154/176): NetworkManager-team-1.46.0-1.el9.x86 996 kB/s | 41 kB 00:00 2024-03-04T08:44:42.551 INFO:teuthology.orchestra.run.smithi071.stdout:(93/176): jq-1.6-16.el9.x86_64.rpm 3.6 MB/s | 186 kB 00:00 2024-03-04T08:44:42.576 INFO:teuthology.orchestra.run.smithi071.stdout:(94/176): libgfortran-11.4.1-3.el9.x86_64.rpm 16 MB/s | 802 kB 00:00 2024-03-04T08:44:42.584 INFO:teuthology.orchestra.run.smithi150.stdout:(155/176): NetworkManager-tui-1.46.0-1.el9.x86_ 5.8 MB/s | 247 kB 00:00 2024-03-04T08:44:42.602 INFO:teuthology.orchestra.run.smithi071.stdout:(95/176): libquadmath-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 194 kB 00:00 2024-03-04T08:44:42.617 INFO:teuthology.orchestra.run.smithi150.stdout:(156/176): acl-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 69 kB 00:00 2024-03-04T08:44:42.627 INFO:teuthology.orchestra.run.smithi071.stdout:(96/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-03-04T08:44:42.651 INFO:teuthology.orchestra.run.smithi150.stdout:(157/176): elfutils-debuginfod-client-0.190-2.e 1.0 MB/s | 35 kB 00:00 2024-03-04T08:44:42.652 INFO:teuthology.orchestra.run.smithi071.stdout:(97/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-04T08:44:42.686 INFO:teuthology.orchestra.run.smithi071.stdout:(98/176): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2024-03-04T08:44:42.692 INFO:teuthology.orchestra.run.smithi150.stdout:(158/176): elfutils-libelf-0.190-2.el9.x86_64.r 4.5 MB/s | 191 kB 00:00 2024-03-04T08:44:42.712 INFO:teuthology.orchestra.run.smithi071.stdout:(99/176): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-04T08:44:42.735 INFO:teuthology.orchestra.run.smithi150.stdout:(159/176): elfutils-libs-0.190-2.el9.x86_64.rpm 6.0 MB/s | 254 kB 00:00 2024-03-04T08:44:42.736 INFO:teuthology.orchestra.run.smithi071.stdout:(100/176): python3-pycparser-2.20-6.el9.noarch. 2.4 MB/s | 124 kB 00:00 2024-03-04T08:44:42.754 INFO:teuthology.orchestra.run.smithi071.stdout:(101/176): python3-requests-2.25.1-8.el9.noarch 2.6 MB/s | 113 kB 00:00 2024-03-04T08:44:42.779 INFO:teuthology.orchestra.run.smithi071.stdout:(102/176): python3-urllib3-1.26.5-5.el9.noarch. 4.4 MB/s | 187 kB 00:00 2024-03-04T08:44:42.793 INFO:teuthology.orchestra.run.smithi150.stdout:(160/176): iproute-6.7.0-2.el9.x86_64.rpm 13 MB/s | 802 kB 00:00 2024-03-04T08:44:42.835 INFO:teuthology.orchestra.run.smithi150.stdout:(161/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 10 MB/s | 433 kB 00:00 2024-03-04T08:44:42.879 INFO:teuthology.orchestra.run.smithi071.stdout:(103/176): lua-devel-5.4.4-4.el9.x86_64.rpm 223 kB/s | 22 kB 00:00 2024-03-04T08:44:42.952 INFO:teuthology.orchestra.run.smithi150.stdout:(162/176): libacl-2.3.1-4.el9.x86_64.rpm 188 kB/s | 22 kB 00:00 2024-03-04T08:44:43.029 INFO:teuthology.orchestra.run.smithi071.stdout:(104/176): python3-devel-3.9.17-2.el9.x86_64.rp 893 kB/s | 245 kB 00:00 2024-03-04T08:44:43.063 INFO:teuthology.orchestra.run.smithi071.stdout:(105/176): protobuf-compiler-3.14.0-13.el9.x86_ 4.6 MB/s | 863 kB 00:00 2024-03-04T08:44:43.077 INFO:teuthology.orchestra.run.smithi150.stdout:(163/176): libblkid-2.37.4-18.el9.x86_64.rpm 859 kB/s | 107 kB 00:00 2024-03-04T08:44:43.120 INFO:teuthology.orchestra.run.smithi150.stdout:(164/176): libfdisk-2.37.4-18.el9.x86_64.rpm 3.6 MB/s | 154 kB 00:00 2024-03-04T08:44:43.153 INFO:teuthology.orchestra.run.smithi150.stdout:(165/176): libmount-2.37.4-18.el9.x86_64.rpm 4.0 MB/s | 135 kB 00:00 2024-03-04T08:44:43.187 INFO:teuthology.orchestra.run.smithi150.stdout:(166/176): libsmartcols-2.37.4-18.el9.x86_64.rp 1.8 MB/s | 63 kB 00:00 2024-03-04T08:44:43.221 INFO:teuthology.orchestra.run.smithi150.stdout:(167/176): libuuid-2.37.4-18.el9.x86_64.rpm 832 kB/s | 28 kB 00:00 2024-03-04T08:44:43.304 INFO:teuthology.orchestra.run.smithi150.stdout:(168/176): openssl-3.0.7-27.el9.x86_64.rpm 14 MB/s | 1.2 MB 00:00 2024-03-04T08:44:43.330 INFO:teuthology.orchestra.run.smithi071.stdout:(106/176): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-03-04T08:44:43.355 INFO:teuthology.orchestra.run.smithi071.stdout:(107/176): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-03-04T08:44:43.380 INFO:teuthology.orchestra.run.smithi071.stdout:(108/176): grpc-data-1.46.7-10.el9.noarch.rpm 390 kB/s | 19 kB 00:00 2024-03-04T08:44:43.388 INFO:teuthology.orchestra.run.smithi150.stdout:(169/176): openssl-libs-3.0.7-27.el9.x86_64.rpm 26 MB/s | 2.2 MB 00:00 2024-03-04T08:44:43.456 INFO:teuthology.orchestra.run.smithi071.stdout:(109/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 346 kB/s | 26 kB 00:00 2024-03-04T08:44:43.480 INFO:teuthology.orchestra.run.smithi150.stdout:(170/176): util-linux-2.37.4-18.el9.x86_64.rpm 24 MB/s | 2.2 MB 00:00 2024-03-04T08:44:43.524 INFO:teuthology.orchestra.run.smithi150.stdout:(171/176): util-linux-core-2.37.4-18.el9.x86_64 10 MB/s | 438 kB 00:00 2024-03-04T08:44:43.547 INFO:teuthology.orchestra.run.smithi150.stdout:(172/176): util-linux-user-2.37.4-18.el9.x86_64 1.2 MB/s | 30 kB 00:00 2024-03-04T08:44:43.581 INFO:teuthology.orchestra.run.smithi071.stdout:(110/176): libarrow-9.0.0-10.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-03-04T08:44:43.598 INFO:teuthology.orchestra.run.smithi150.stdout:(173/176): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-03-04T08:44:43.606 INFO:teuthology.orchestra.run.smithi071.stdout:(111/176): liboath-2.6.7-2.el9.x86_64.rpm 328 kB/s | 49 kB 00:00 2024-03-04T08:44:43.698 INFO:teuthology.orchestra.run.smithi150.stdout:(174/176): ceph-radosgw-19.0.0-1739.g1bc250c7.e 658 kB/s | 9.8 MB 00:15 2024-03-04T08:44:43.731 INFO:teuthology.orchestra.run.smithi150.stdout:(175/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.9 MB/s | 1.1 MB 00:00 2024-03-04T08:44:43.748 INFO:teuthology.orchestra.run.smithi071.stdout:(112/176): libunwind-1.6.2-1.el9.x86_64.rpm 404 kB/s | 67 kB 00:00 2024-03-04T08:44:44.098 INFO:teuthology.orchestra.run.smithi150.stdout:(176/176): librados2-19.0.0-1739.g1bc250c7.el9. 1.4 MB/s | 3.4 MB 00:02 2024-03-04T08:44:44.105 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-03-04T08:44:44.105 INFO:teuthology.orchestra.run.smithi150.stdout:Total 12 MB/s | 224 MB 00:18 2024-03-04T08:44:44.152 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-03-04T08:44:44.331 INFO:teuthology.orchestra.run.smithi071.stdout:(113/176): luarocks-3.9.2-1.el9.noarch.rpm 208 kB/s | 151 kB 00:00 2024-03-04T08:44:44.365 INFO:teuthology.orchestra.run.smithi071.stdout:(114/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.3 MB/s | 838 kB 00:00 2024-03-04T08:44:44.428 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-03-04T08:44:44.428 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-03-04T08:44:44.718 INFO:teuthology.orchestra.run.smithi158.stdout:(10/176): ceph-osd-19.0.0-1739.g1bc250c7.el9.x8 3.6 MB/s | 17 MB 00:04 2024-03-04T08:44:44.801 INFO:teuthology.orchestra.run.smithi158.stdout:(11/176): cephfs-mirror-19.0.0-1739.g1bc250c7.e 2.6 MB/s | 220 kB 00:00 2024-03-04T08:44:44.852 INFO:teuthology.orchestra.run.smithi158.stdout:(12/176): cephfs-shell-19.0.0-1739.g1bc250c7.el 544 kB/s | 27 kB 00:00 2024-03-04T08:44:44.882 INFO:teuthology.orchestra.run.smithi071.stdout:(115/176): python3-asyncssh-2.13.2-4.el9.noarch 998 kB/s | 548 kB 00:00 2024-03-04T08:44:44.902 INFO:teuthology.orchestra.run.smithi158.stdout:(13/176): libcephfs-devel-19.0.0-1739.g1bc250c7 625 kB/s | 31 kB 00:00 2024-03-04T08:44:44.907 INFO:teuthology.orchestra.run.smithi071.stdout:(116/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 80 kB/s | 43 kB 00:00 2024-03-04T08:44:44.957 INFO:teuthology.orchestra.run.smithi071.stdout:(117/176): python3-cachetools-4.2.4-1.el9.noarc 646 kB/s | 32 kB 00:00 2024-03-04T08:44:44.983 INFO:teuthology.orchestra.run.smithi071.stdout:(118/176): python3-certifi-2021.10.8-2.el9.noar 200 kB/s | 15 kB 00:00 2024-03-04T08:44:45.017 INFO:teuthology.orchestra.run.smithi071.stdout:(119/176): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-03-04T08:44:45.060 INFO:teuthology.orchestra.run.smithi158.stdout:(14/176): libcephfs2-19.0.0-1739.g1bc250c7.el9. 4.6 MB/s | 747 kB 00:00 2024-03-04T08:44:45.075 INFO:teuthology.orchestra.run.smithi071.stdout:(120/176): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-03-04T08:44:45.100 INFO:teuthology.orchestra.run.smithi071.stdout:(121/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-03-04T08:44:45.136 INFO:teuthology.orchestra.run.smithi158.stdout:(15/176): libcephsqlite-19.0.0-1739.g1bc250c7.e 2.1 MB/s | 163 kB 00:00 2024-03-04T08:44:45.142 INFO:teuthology.orchestra.run.smithi071.stdout:(122/176): python3-colorama-0.4.4-12.el9.noarch 534 kB/s | 35 kB 00:00 2024-03-04T08:44:45.186 INFO:teuthology.orchestra.run.smithi158.stdout:(16/176): librados-devel-19.0.0-1739.g1bc250c7. 2.5 MB/s | 127 kB 00:00 2024-03-04T08:44:45.192 INFO:teuthology.orchestra.run.smithi071.stdout:(123/176): python3-google-auth-2.28.0-1.el9.noa 2.2 MB/s | 203 kB 00:00 2024-03-04T08:44:45.310 INFO:teuthology.orchestra.run.smithi071.stdout:(124/176): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-03-04T08:44:45.320 INFO:teuthology.orchestra.run.smithi158.stdout:(17/176): libradosstriper1-19.0.0-1739.g1bc250c 3.7 MB/s | 501 kB 00:00 2024-03-04T08:44:45.335 INFO:teuthology.orchestra.run.smithi071.stdout:(125/176): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-03-04T08:44:45.377 INFO:teuthology.orchestra.run.smithi071.stdout:(126/176): python3-jaraco-8.2.1-3.el9.noarch.rp 161 kB/s | 11 kB 00:00 2024-03-04T08:44:45.402 INFO:teuthology.orchestra.run.smithi071.stdout:(127/176): python3-jaraco-classes-3.2.1-5.el9.n 265 kB/s | 18 kB 00:00 2024-03-04T08:44:45.427 INFO:teuthology.orchestra.run.smithi071.stdout:(128/176): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-03-04T08:44:45.494 INFO:teuthology.orchestra.run.smithi071.stdout:(129/176): python3-jaraco-functools-3.5.0-2.el9 212 kB/s | 19 kB 00:00 2024-03-04T08:44:45.519 INFO:teuthology.orchestra.run.smithi071.stdout:(130/176): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-03-04T08:44:45.570 INFO:teuthology.orchestra.run.smithi071.stdout:(131/176): python3-kubernetes-26.1.0-2.el9.noar 14 MB/s | 1.0 MB 00:00 2024-03-04T08:44:45.595 INFO:teuthology.orchestra.run.smithi071.stdout:(132/176): python3-logutils-0.3.5-21.el9.noarch 617 kB/s | 46 kB 00:00 2024-03-04T08:44:45.620 INFO:teuthology.orchestra.run.smithi071.stdout:(133/176): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-03-04T08:44:45.645 INFO:teuthology.orchestra.run.smithi071.stdout:(134/176): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-03-04T08:44:46.096 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-03-04T08:44:46.096 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-03-04T08:44:46.303 INFO:teuthology.orchestra.run.smithi158.stdout:(18/176): ceph-test-19.0.0-1739.g1bc250c7.el9.x 10 MB/s | 47 MB 00:04 2024-03-04T08:44:46.353 INFO:teuthology.orchestra.run.smithi158.stdout:(19/176): python3-ceph-argparse-19.0.0-1739.g1b 909 kB/s | 45 kB 00:00 2024-03-04T08:44:46.362 INFO:teuthology.orchestra.run.smithi071.stdout:(135/176): python3-pecan-1.4.2-3.el9.noarch.rpm 367 kB/s | 272 kB 00:00 2024-03-04T08:44:46.388 INFO:teuthology.orchestra.run.smithi071.stdout:(136/176): python3-portend-3.1.0-2.el9.noarch.r 22 kB/s | 16 kB 00:00 2024-03-04T08:44:46.421 INFO:teuthology.orchestra.run.smithi158.stdout:(20/176): python3-ceph-common-19.0.0-1739.g1bc2 1.9 MB/s | 131 kB 00:00 2024-03-04T08:44:46.429 INFO:teuthology.orchestra.run.smithi071.stdout:(137/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-03-04T08:44:46.455 INFO:teuthology.orchestra.run.smithi071.stdout:(138/176): python3-pyperclip-1.8.2-2.el9.noarch 421 kB/s | 28 kB 00:00 2024-03-04T08:44:46.480 INFO:teuthology.orchestra.run.smithi071.stdout:(139/176): python3-repoze-lru-0.7-16.el9.noarch 615 kB/s | 31 kB 00:00 2024-03-04T08:44:46.505 INFO:teuthology.orchestra.run.smithi071.stdout:(140/176): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-03-04T08:44:46.513 INFO:teuthology.orchestra.run.smithi158.stdout:(21/176): librgw2-19.0.0-1739.g1bc250c7.el9.x86 4.3 MB/s | 5.1 MB 00:01 2024-03-04T08:44:46.531 INFO:teuthology.orchestra.run.smithi071.stdout:(141/176): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-03-04T08:44:46.538 INFO:teuthology.orchestra.run.smithi158.stdout:(22/176): python3-cephfs-19.0.0-1739.g1bc250c7. 1.4 MB/s | 162 kB 00:00 2024-03-04T08:44:46.573 INFO:teuthology.orchestra.run.smithi071.stdout:(142/176): python3-tempora-5.0.0-2.el9.noarch.r 537 kB/s | 36 kB 00:00 2024-03-04T08:44:46.597 INFO:teuthology.orchestra.run.smithi158.stdout:(23/176): python3-rados-19.0.0-1739.g1bc250c7.e 3.8 MB/s | 321 kB 00:00 2024-03-04T08:44:46.598 INFO:teuthology.orchestra.run.smithi071.stdout:(143/176): python3-typing-extensions-4.4.0-2.el 762 kB/s | 51 kB 00:00 2024-03-04T08:44:46.622 INFO:teuthology.orchestra.run.smithi158.stdout:(24/176): python3-rbd-19.0.0-1739.g1bc250c7.el9 3.5 MB/s | 298 kB 00:00 2024-03-04T08:44:46.631 INFO:teuthology.orchestra.run.smithi071.stdout:(144/176): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-03-04T08:44:46.647 INFO:teuthology.orchestra.run.smithi158.stdout:(25/176): python3-rgw-19.0.0-1739.g1bc250c7.el9 1.9 MB/s | 99 kB 00:00 2024-03-04T08:44:46.665 INFO:teuthology.orchestra.run.smithi071.stdout:(145/176): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-03-04T08:44:46.672 INFO:teuthology.orchestra.run.smithi158.stdout:(26/176): rbd-fuse-19.0.0-1739.g1bc250c7.el9.x8 1.7 MB/s | 85 kB 00:00 2024-03-04T08:44:46.699 INFO:teuthology.orchestra.run.smithi071.stdout:(146/176): python3-werkzeug-2.0.3-3.el9.1.noarc 6.2 MB/s | 427 kB 00:00 2024-03-04T08:44:46.725 INFO:teuthology.orchestra.run.smithi071.stdout:(147/176): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-03-04T08:44:46.731 INFO:teuthology.orchestra.run.smithi158.stdout:(27/176): rbd-nbd-19.0.0-1739.g1bc250c7.el9.x86 2.9 MB/s | 171 kB 00:00 2024-03-04T08:44:46.766 INFO:teuthology.orchestra.run.smithi071.stdout:(148/176): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-03-04T08:44:46.773 INFO:teuthology.orchestra.run.smithi158.stdout:(28/176): ceph-grafana-dashboards-19.0.0-1739.g 755 kB/s | 31 kB 00:00 2024-03-04T08:44:46.824 INFO:teuthology.orchestra.run.smithi158.stdout:(29/176): ceph-mgr-cephadm-19.0.0-1739.g1bc250c 2.7 MB/s | 138 kB 00:00 2024-03-04T08:44:46.875 INFO:teuthology.orchestra.run.smithi071.stdout:(149/176): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-03-04T08:44:47.016 INFO:teuthology.orchestra.run.smithi158.stdout:(30/176): ceph-mgr-dashboard-19.0.0-1739.g1bc25 9.3 MB/s | 1.8 MB 00:00 2024-03-04T08:44:47.133 INFO:teuthology.orchestra.run.smithi158.stdout:(31/176): rbd-mirror-19.0.0-1739.g1bc250c7.el9. 6.4 MB/s | 3.1 MB 00:00 2024-03-04T08:44:47.225 INFO:teuthology.orchestra.run.smithi158.stdout:(32/176): ceph-mgr-modules-core-19.0.0-1739.g1b 2.6 MB/s | 248 kB 00:00 2024-03-04T08:44:47.283 INFO:teuthology.orchestra.run.smithi158.stdout:(33/176): ceph-mgr-rook-19.0.0-1739.g1bc250c7.e 847 kB/s | 49 kB 00:00 2024-03-04T08:44:47.342 INFO:teuthology.orchestra.run.smithi158.stdout:(34/176): ceph-prometheus-alerts-19.0.0-1739.g1 263 kB/s | 15 kB 00:00 2024-03-04T08:44:47.467 INFO:teuthology.orchestra.run.smithi158.stdout:(35/176): ceph-volume-19.0.0-1739.g1bc250c7.el9 2.1 MB/s | 264 kB 00:00 2024-03-04T08:44:47.625 INFO:teuthology.orchestra.run.smithi071.stdout:(150/176): librbd1-19.0.0-1739.g1bc250c7.el9.x8 4.2 MB/s | 3.1 MB 00:00 2024-03-04T08:44:47.659 INFO:teuthology.orchestra.run.smithi158.stdout:(36/176): cephadm-19.0.0-1739.g1bc250c7.el9.noa 3.8 MB/s | 748 kB 00:00 2024-03-04T08:44:47.676 INFO:teuthology.orchestra.run.smithi071.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.3 MB/s | 68 kB 00:00 2024-03-04T08:44:47.709 INFO:teuthology.orchestra.run.smithi158.stdout:(37/176): cephfs-top-19.0.0-1739.g1bc250c7.el9. 365 kB/s | 18 kB 00:00 2024-03-04T08:44:47.784 INFO:teuthology.orchestra.run.smithi071.stdout:(152/176): NetworkManager-1.46.0-1.el9.x86_64.r 21 MB/s | 2.3 MB 00:00 2024-03-04T08:44:47.818 INFO:teuthology.orchestra.run.smithi158.stdout:(38/176): ceph-mgr-diskprediction-local-19.0.0- 9.2 MB/s | 7.4 MB 00:00 2024-03-04T08:44:47.892 INFO:teuthology.orchestra.run.smithi071.stdout:(153/176): NetworkManager-libnm-1.46.0-1.el9.x8 17 MB/s | 1.8 MB 00:00 2024-03-04T08:44:47.951 INFO:teuthology.orchestra.run.smithi071.stdout:(154/176): NetworkManager-team-1.46.0-1.el9.x86 709 kB/s | 41 kB 00:00 2024-03-04T08:44:48.035 INFO:teuthology.orchestra.run.smithi071.stdout:(155/176): NetworkManager-tui-1.46.0-1.el9.x86_ 2.9 MB/s | 247 kB 00:00 2024-03-04T08:44:48.151 INFO:teuthology.orchestra.run.smithi071.stdout:(156/176): acl-2.3.1-4.el9.x86_64.rpm 589 kB/s | 69 kB 00:00 2024-03-04T08:44:48.210 INFO:teuthology.orchestra.run.smithi071.stdout:(157/176): librados2-19.0.0-1739.g1bc250c7.el9. 2.3 MB/s | 3.4 MB 00:01 2024-03-04T08:44:48.218 INFO:teuthology.orchestra.run.smithi158.stdout:(39/176): bison-3.7.4-5.el9.x86_64.rpm 2.2 MB/s | 920 kB 00:00 2024-03-04T08:44:48.236 INFO:teuthology.orchestra.run.smithi071.stdout:(158/176): elfutils-debuginfod-client-0.190-2.e 422 kB/s | 35 kB 00:00 2024-03-04T08:44:48.252 INFO:teuthology.orchestra.run.smithi158.stdout:(40/176): automake-1.16.2-8.el9.noarch.rpm 1.2 MB/s | 662 kB 00:00 2024-03-04T08:44:48.261 INFO:teuthology.orchestra.run.smithi071.stdout:(159/176): elfutils-libelf-0.190-2.el9.x86_64.r 3.7 MB/s | 191 kB 00:00 2024-03-04T08:44:48.277 INFO:teuthology.orchestra.run.smithi158.stdout:(41/176): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-03-04T08:44:48.286 INFO:teuthology.orchestra.run.smithi071.stdout:(160/176): elfutils-libs-0.190-2.el9.x86_64.rpm 5.0 MB/s | 254 kB 00:00 2024-03-04T08:44:48.303 INFO:teuthology.orchestra.run.smithi158.stdout:(42/176): elfutils-libelf-devel-0.190-2.el9.x86 407 kB/s | 20 kB 00:00 2024-03-04T08:44:48.320 INFO:teuthology.orchestra.run.smithi071.stdout:(161/176): iproute-6.7.0-2.el9.x86_64.rpm 14 MB/s | 802 kB 00:00 2024-03-04T08:44:48.336 INFO:teuthology.orchestra.run.smithi158.stdout:(43/176): flex-2.6.4-9.el9.x86_64.rpm 5.1 MB/s | 307 kB 00:00 2024-03-04T08:44:48.345 INFO:teuthology.orchestra.run.smithi071.stdout:(162/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 7.4 MB/s | 433 kB 00:00 2024-03-04T08:44:48.361 INFO:teuthology.orchestra.run.smithi158.stdout:(44/176): flexiblas-3.0.4-8.el9.x86_64.rpm 546 kB/s | 32 kB 00:00 2024-03-04T08:44:48.362 INFO:teuthology.orchestra.run.smithi071.stdout:(163/176): libacl-2.3.1-4.el9.x86_64.rpm 535 kB/s | 22 kB 00:00 2024-03-04T08:44:48.387 INFO:teuthology.orchestra.run.smithi071.stdout:(164/176): libblkid-2.37.4-18.el9.x86_64.rpm 2.5 MB/s | 107 kB 00:00 2024-03-04T08:44:48.438 INFO:teuthology.orchestra.run.smithi071.stdout:(165/176): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 154 kB 00:00 2024-03-04T08:44:48.463 INFO:teuthology.orchestra.run.smithi071.stdout:(166/176): libmount-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 135 kB 00:00 2024-03-04T08:44:48.487 INFO:teuthology.orchestra.run.smithi158.stdout:(45/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-03-04T08:44:48.488 INFO:teuthology.orchestra.run.smithi071.stdout:(167/176): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 63 kB 00:00 2024-03-04T08:44:48.504 INFO:teuthology.orchestra.run.smithi158.stdout:(46/176): flexiblas-openblas-openmp-3.0.4-8.el9 121 kB/s | 17 kB 00:00 2024-03-04T08:44:48.513 INFO:teuthology.orchestra.run.smithi071.stdout:(168/176): libuuid-2.37.4-18.el9.x86_64.rpm 552 kB/s | 28 kB 00:00 2024-03-04T08:44:48.521 INFO:teuthology.orchestra.run.smithi158.stdout:(47/176): libaio-devel-0.3.111-13.el9.x86_64.rp 303 kB/s | 10 kB 00:00 2024-03-04T08:44:48.546 INFO:teuthology.orchestra.run.smithi158.stdout:(48/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-03-04T08:44:48.555 INFO:teuthology.orchestra.run.smithi071.stdout:(169/176): openssl-3.0.7-27.el9.x86_64.rpm 17 MB/s | 1.2 MB 00:00 2024-03-04T08:44:48.563 INFO:teuthology.orchestra.run.smithi158.stdout:(49/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-04T08:44:48.597 INFO:teuthology.orchestra.run.smithi158.stdout:(50/176): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-04T08:44:48.614 INFO:teuthology.orchestra.run.smithi071.stdout:(170/176): openssl-libs-3.0.7-27.el9.x86_64.rpm 22 MB/s | 2.2 MB 00:00 2024-03-04T08:44:48.639 INFO:teuthology.orchestra.run.smithi158.stdout:(51/176): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 578 kB 00:00 2024-03-04T08:44:48.656 INFO:teuthology.orchestra.run.smithi158.stdout:(52/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 424 kB/s | 24 kB 00:00 2024-03-04T08:44:48.673 INFO:teuthology.orchestra.run.smithi071.stdout:(171/176): util-linux-2.37.4-18.el9.x86_64.rpm 19 MB/s | 2.2 MB 00:00 2024-03-04T08:44:48.673 INFO:teuthology.orchestra.run.smithi158.stdout:(53/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-03-04T08:44:48.698 INFO:teuthology.orchestra.run.smithi071.stdout:(172/176): util-linux-core-2.37.4-18.el9.x86_64 5.3 MB/s | 438 kB 00:00 2024-03-04T08:44:48.698 INFO:teuthology.orchestra.run.smithi158.stdout:(54/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-03-04T08:44:48.723 INFO:teuthology.orchestra.run.smithi071.stdout:(173/176): util-linux-user-2.37.4-18.el9.x86_64 610 kB/s | 30 kB 00:00 2024-03-04T08:44:48.724 INFO:teuthology.orchestra.run.smithi158.stdout:(55/176): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-04T08:44:48.740 INFO:teuthology.orchestra.run.smithi158.stdout:(56/176): openblas-0.3.21-2.el9.x86_64.rpm 822 kB/s | 34 kB 00:00 2024-03-04T08:44:48.748 INFO:teuthology.orchestra.run.smithi071.stdout:(174/176): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-03-04T08:44:48.807 INFO:teuthology.orchestra.run.smithi071.stdout:(175/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 13 MB/s | 1.1 MB 00:00 2024-03-04T08:44:48.874 INFO:teuthology.orchestra.run.smithi158.stdout:(57/176): openblas-openmp-0.3.21-2.el9.x86_64.r 31 MB/s | 4.7 MB 00:00 2024-03-04T08:44:48.942 INFO:teuthology.orchestra.run.smithi158.stdout:(58/176): openssl-devel-3.0.7-27.el9.x86_64.rpm 15 MB/s | 3.0 MB 00:00 2024-03-04T08:44:48.983 INFO:teuthology.orchestra.run.smithi158.stdout:(59/176): postgresql-13.11-1.el9.x86_64.rpm 14 MB/s | 1.5 MB 00:00 2024-03-04T08:44:49.017 INFO:teuthology.orchestra.run.smithi158.stdout:(60/176): postgresql-contrib-13.11-1.el9.x86_64 11 MB/s | 811 kB 00:00 2024-03-04T08:44:49.042 INFO:teuthology.orchestra.run.smithi158.stdout:(61/176): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2024-03-04T08:44:49.184 INFO:teuthology.orchestra.run.smithi158.stdout:(62/176): postgresql-server-13.11-1.el9.x86_64. 34 MB/s | 5.7 MB 00:00 2024-03-04T08:44:49.218 INFO:teuthology.orchestra.run.smithi158.stdout:(63/176): protobuf-3.14.0-13.el9.x86_64.rpm 5.8 MB/s | 1.0 MB 00:00 2024-03-04T08:44:49.245 INFO:teuthology.orchestra.run.smithi158.stdout:(64/176): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-04T08:44:49.396 INFO:teuthology.orchestra.run.smithi158.stdout:(65/176): python3-babel-2.9.1-2.el9.noarch.rpm 33 MB/s | 5.8 MB 00:00 2024-03-04T08:44:49.421 INFO:teuthology.orchestra.run.smithi158.stdout:(66/176): python3-jinja2-2.11.3-5.el9.noarch.rp 1.3 MB/s | 227 kB 00:00 2024-03-04T08:44:49.446 INFO:teuthology.orchestra.run.smithi158.stdout:(67/176): python3-jsonpatch-1.21-16.el9.noarch. 489 kB/s | 24 kB 00:00 2024-03-04T08:44:49.464 INFO:teuthology.orchestra.run.smithi158.stdout:(68/176): python3-jsonpointer-2.0-4.el9.noarch. 426 kB/s | 18 kB 00:00 2024-03-04T08:44:49.489 INFO:teuthology.orchestra.run.smithi158.stdout:(69/176): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 144 kB 00:00 2024-03-04T08:44:49.506 INFO:teuthology.orchestra.run.smithi158.stdout:(70/176): python3-markupsafe-1.1.1-12.el9.x86_6 761 kB/s | 32 kB 00:00 2024-03-04T08:44:49.549 INFO:teuthology.orchestra.run.smithi071.stdout:(176/176): ceph-test-19.0.0-1739.g1bc250c7.el9. 3.5 MB/s | 47 MB 00:13 2024-03-04T08:44:49.555 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-03-04T08:44:49.555 INFO:teuthology.orchestra.run.smithi071.stdout:Total 13 MB/s | 224 MB 00:16 2024-03-04T08:44:49.603 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-03-04T08:44:49.631 INFO:teuthology.orchestra.run.smithi158.stdout:(71/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2024-03-04T08:44:49.657 INFO:teuthology.orchestra.run.smithi158.stdout:(72/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2024-03-04T08:44:49.682 INFO:teuthology.orchestra.run.smithi158.stdout:(73/176): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-04T08:44:49.707 INFO:teuthology.orchestra.run.smithi158.stdout:(74/176): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-04T08:44:49.724 INFO:teuthology.orchestra.run.smithi158.stdout:(75/176): python3-prettytable-0.7.2-27.el9.noar 984 kB/s | 41 kB 00:00 2024-03-04T08:44:49.758 INFO:teuthology.orchestra.run.smithi158.stdout:(76/176): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-03-04T08:44:49.783 INFO:teuthology.orchestra.run.smithi158.stdout:(77/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.2 MB/s | 132 kB 00:00 2024-03-04T08:44:49.809 INFO:teuthology.orchestra.run.smithi158.stdout:(78/176): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-04T08:44:49.825 INFO:teuthology.orchestra.run.smithi158.stdout:(79/176): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-03-04T08:44:49.851 INFO:teuthology.orchestra.run.smithi158.stdout:(80/176): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 43 kB 00:00 2024-03-04T08:44:49.922 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-03-04T08:44:49.922 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-03-04T08:44:50.001 INFO:teuthology.orchestra.run.smithi158.stdout:(81/176): python3-toml-0.10.2-6.el9.noarch.rpm 262 kB/s | 39 kB 00:00 2024-03-04T08:44:50.235 INFO:teuthology.orchestra.run.smithi158.stdout:(82/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-03-04T08:44:50.260 INFO:teuthology.orchestra.run.smithi158.stdout:(83/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 160 kB/s | 41 kB 00:00 2024-03-04T08:44:50.294 INFO:teuthology.orchestra.run.smithi158.stdout:(84/176): qatlib-23.11.0-1.el9.x86_64.rpm 3.6 MB/s | 215 kB 00:00 2024-03-04T08:44:50.319 INFO:teuthology.orchestra.run.smithi158.stdout:(85/176): qatlib-service-23.11.0-1.el9.x86_64.r 580 kB/s | 34 kB 00:00 2024-03-04T08:44:50.344 INFO:teuthology.orchestra.run.smithi158.stdout:(86/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 861 kB/s | 43 kB 00:00 2024-03-04T08:44:50.370 INFO:teuthology.orchestra.run.smithi158.stdout:(87/176): socat-1.7.4.1-5.el9.x86_64.rpm 5.8 MB/s | 300 kB 00:00 2024-03-04T08:44:50.395 INFO:teuthology.orchestra.run.smithi158.stdout:(88/176): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-04T08:44:50.420 INFO:teuthology.orchestra.run.smithi158.stdout:(89/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2024-03-04T08:44:50.446 INFO:teuthology.orchestra.run.smithi158.stdout:(90/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-04T08:44:50.471 INFO:teuthology.orchestra.run.smithi158.stdout:(91/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 879 kB/s | 44 kB 00:00 2024-03-04T08:44:50.496 INFO:teuthology.orchestra.run.smithi158.stdout:(92/176): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-04T08:44:50.522 INFO:teuthology.orchestra.run.smithi158.stdout:(93/176): jq-1.6-16.el9.x86_64.rpm 3.6 MB/s | 186 kB 00:00 2024-03-04T08:44:50.564 INFO:teuthology.orchestra.run.smithi158.stdout:(94/176): libgfortran-11.4.1-3.el9.x86_64.rpm 12 MB/s | 802 kB 00:00 2024-03-04T08:44:50.589 INFO:teuthology.orchestra.run.smithi158.stdout:(95/176): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 194 kB 00:00 2024-03-04T08:44:50.615 INFO:teuthology.orchestra.run.smithi158.stdout:(96/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-04T08:44:50.640 INFO:teuthology.orchestra.run.smithi158.stdout:(97/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-04T08:44:50.682 INFO:teuthology.orchestra.run.smithi158.stdout:(98/176): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.1 MB 00:00 2024-03-04T08:44:50.707 INFO:teuthology.orchestra.run.smithi158.stdout:(99/176): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-04T08:44:50.732 INFO:teuthology.orchestra.run.smithi158.stdout:(100/176): python3-pycparser-2.20-6.el9.noarch. 2.4 MB/s | 124 kB 00:00 2024-03-04T08:44:50.758 INFO:teuthology.orchestra.run.smithi158.stdout:(101/176): python3-requests-2.25.1-8.el9.noarch 2.2 MB/s | 113 kB 00:00 2024-03-04T08:44:50.783 INFO:teuthology.orchestra.run.smithi158.stdout:(102/176): python3-urllib3-1.26.5-5.el9.noarch. 3.7 MB/s | 187 kB 00:00 2024-03-04T08:44:50.867 INFO:teuthology.orchestra.run.smithi158.stdout:(103/176): lua-devel-5.4.4-4.el9.x86_64.rpm 268 kB/s | 22 kB 00:00 2024-03-04T08:44:50.950 INFO:teuthology.orchestra.run.smithi158.stdout:(104/176): python3-devel-3.9.17-2.el9.x86_64.rp 1.2 MB/s | 245 kB 00:00 2024-03-04T08:44:51.042 INFO:teuthology.orchestra.run.smithi158.stdout:(105/176): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-03-04T08:44:51.701 INFO:teuthology.orchestra.run.smithi158.stdout:(106/176): abseil-cpp-20211102.0-3.el9.x86_64.r 730 kB/s | 548 kB 00:00 2024-03-04T08:44:51.726 INFO:teuthology.orchestra.run.smithi158.stdout:(107/176): gperftools-libs-2.9.1-2.el9.x86_64.r 470 kB/s | 309 kB 00:00 2024-03-04T08:44:51.818 INFO:teuthology.orchestra.run.smithi158.stdout:(108/176): grpc-data-1.46.7-10.el9.noarch.rpm 167 kB/s | 19 kB 00:00 2024-03-04T08:44:51.891 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-03-04T08:44:51.892 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-03-04T08:44:51.918 INFO:teuthology.orchestra.run.smithi158.stdout:(109/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-03-04T08:44:52.035 INFO:teuthology.orchestra.run.smithi158.stdout:(110/176): liboath-2.6.7-2.el9.x86_64.rpm 416 kB/s | 49 kB 00:00 2024-03-04T08:44:52.152 INFO:teuthology.orchestra.run.smithi158.stdout:(111/176): libunwind-1.6.2-1.el9.x86_64.rpm 577 kB/s | 67 kB 00:00 2024-03-04T08:44:52.228 INFO:teuthology.orchestra.run.smithi158.stdout:(112/176): libarrow-9.0.0-10.el9.x86_64.rpm 8.8 MB/s | 4.4 MB 00:00 2024-03-04T08:44:52.270 INFO:teuthology.orchestra.run.smithi158.stdout:(113/176): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-03-04T08:44:52.362 INFO:teuthology.orchestra.run.smithi158.stdout:(114/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.1 MB/s | 838 kB 00:00 2024-03-04T08:44:52.443 INFO:teuthology.orchestra.run.smithi158.stdout:(115/176): python3-asyncssh-2.13.2-4.el9.noarch 3.1 MB/s | 548 kB 00:00 2024-03-04T08:44:52.468 INFO:teuthology.orchestra.run.smithi158.stdout:(116/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 411 kB/s | 43 kB 00:00 2024-03-04T08:44:52.543 INFO:teuthology.orchestra.run.smithi158.stdout:(117/176): python3-cachetools-4.2.4-1.el9.noarc 321 kB/s | 32 kB 00:00 2024-03-04T08:44:52.569 INFO:teuthology.orchestra.run.smithi158.stdout:(118/176): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-03-04T08:44:52.660 INFO:teuthology.orchestra.run.smithi158.stdout:(119/176): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-03-04T08:44:52.894 INFO:teuthology.orchestra.run.smithi158.stdout:(120/176): ceph-common-19.0.0-1739.g1bc250c7.el 1.4 MB/s | 20 MB 00:14 2024-03-04T08:44:52.927 INFO:teuthology.orchestra.run.smithi158.stdout:(121/176): python3-cherrypy-18.6.1-2.el9.noarch 1.0 MB/s | 358 kB 00:00 2024-03-04T08:44:52.961 INFO:teuthology.orchestra.run.smithi158.stdout:(122/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.0 MB/s | 303 kB 00:00 2024-03-04T08:44:53.036 INFO:teuthology.orchestra.run.smithi158.stdout:(123/176): python3-google-auth-2.28.0-1.el9.noa 1.8 MB/s | 203 kB 00:00 2024-03-04T08:44:53.075 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-03-04T08:44:53.162 INFO:teuthology.orchestra.run.smithi158.stdout:(124/176): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-03-04T08:44:53.194 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/202 2024-03-04T08:44:53.228 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-03-04T08:44:53.254 INFO:teuthology.orchestra.run.smithi158.stdout:(125/176): python3-grpcio-1.46.7-10.el9.x86_64. 7.0 MB/s | 2.0 MB 00:00 2024-03-04T08:44:53.267 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-03-04T08:44:53.279 INFO:teuthology.orchestra.run.smithi158.stdout:(126/176): python3-jaraco-8.2.1-3.el9.noarch.rp 91 kB/s | 11 kB 00:00 2024-03-04T08:44:53.294 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-03-04T08:44:53.304 INFO:teuthology.orchestra.run.smithi158.stdout:(127/176): python3-colorama-0.4.4-12.el9.noarch 86 kB/s | 35 kB 00:00 2024-03-04T08:44:53.346 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-03-04T08:44:53.363 INFO:teuthology.orchestra.run.smithi158.stdout:(128/176): python3-jaraco-classes-3.2.1-5.el9.n 162 kB/s | 18 kB 00:00 2024-03-04T08:44:53.388 INFO:teuthology.orchestra.run.smithi158.stdout:(129/176): python3-jaraco-collections-3.0.0-8.e 212 kB/s | 23 kB 00:00 2024-03-04T08:44:53.392 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-03-04T08:44:53.438 INFO:teuthology.orchestra.run.smithi158.stdout:(130/176): python3-jaraco-functools-3.5.0-2.el9 145 kB/s | 19 kB 00:00 2024-03-04T08:44:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-03-04T08:44:53.472 INFO:teuthology.orchestra.run.smithi158.stdout:(131/176): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-03-04T08:44:53.480 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-03-04T08:44:53.506 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-03-04T08:44:53.522 INFO:teuthology.orchestra.run.smithi158.stdout:(132/176): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-03-04T08:44:53.548 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-03-04T08:44:53.600 INFO:teuthology.orchestra.run.smithi158.stdout:(133/176): python3-more-itertools-8.12.0-2.el9. 628 kB/s | 79 kB 00:00 2024-03-04T08:44:53.623 INFO:teuthology.orchestra.run.smithi158.stdout:(134/176): python3-logutils-0.3.5-21.el9.noarch 251 kB/s | 46 kB 00:00 2024-03-04T08:44:53.648 INFO:teuthology.orchestra.run.smithi158.stdout:(135/176): python3-natsort-7.1.1-5.el9.noarch.r 459 kB/s | 58 kB 00:00 2024-03-04T08:44:53.724 INFO:teuthology.orchestra.run.smithi158.stdout:(136/176): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-03-04T08:44:53.749 INFO:teuthology.orchestra.run.smithi158.stdout:(137/176): python3-portend-3.1.0-2.el9.noarch.r 131 kB/s | 16 kB 00:00 2024-03-04T08:44:53.782 INFO:teuthology.orchestra.run.smithi158.stdout:(138/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 671 kB/s | 90 kB 00:00 2024-03-04T08:44:53.833 INFO:teuthology.orchestra.run.smithi158.stdout:(139/176): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-03-04T08:44:53.867 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-03-04T08:44:53.891 INFO:teuthology.orchestra.run.smithi158.stdout:(140/176): python3-repoze-lru-0.7-16.el9.noarch 216 kB/s | 31 kB 00:00 2024-03-04T08:44:53.925 INFO:teuthology.orchestra.run.smithi158.stdout:(141/176): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-03-04T08:44:53.932 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-04T08:44:53.950 INFO:teuthology.orchestra.run.smithi158.stdout:(142/176): python3-rsa-4.9-2.el9.noarch.rpm 504 kB/s | 59 kB 00:00 2024-03-04T08:44:54.034 INFO:teuthology.orchestra.run.smithi158.stdout:(143/176): python3-typing-extensions-4.4.0-2.el 469 kB/s | 51 kB 00:00 2024-03-04T08:44:54.036 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-04T08:44:54.058 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-04T08:44:54.059 INFO:teuthology.orchestra.run.smithi158.stdout:(144/176): python3-tempora-5.0.0-2.el9.noarch.r 213 kB/s | 36 kB 00:00 2024-03-04T08:44:54.084 INFO:teuthology.orchestra.run.smithi158.stdout:(145/176): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-03-04T08:44:54.132 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-04T08:44:54.143 INFO:teuthology.orchestra.run.smithi158.stdout:(146/176): python3-websocket-client-1.2.3-2.el9 825 kB/s | 90 kB 00:00 2024-03-04T08:44:54.162 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 13/202 2024-03-04T08:44:54.193 INFO:teuthology.orchestra.run.smithi158.stdout:(147/176): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-03-04T08:44:54.268 INFO:teuthology.orchestra.run.smithi158.stdout:(148/176): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-03-04T08:44:54.311 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-03-04T08:44:54.352 INFO:teuthology.orchestra.run.smithi158.stdout:(149/176): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-03-04T08:44:54.367 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-03-04T08:44:54.395 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-04T08:44:54.460 INFO:teuthology.orchestra.run.smithi158.stdout:(150/176): python3-werkzeug-2.0.3-3.el9.1.noarc 1.0 MB/s | 427 kB 00:00 2024-03-04T08:44:54.494 INFO:teuthology.orchestra.run.smithi158.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2024-03-04T08:44:54.500 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-04T08:44:54.524 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 17/202 2024-03-04T08:44:54.578 INFO:teuthology.orchestra.run.smithi158.stdout:(152/176): NetworkManager-1.46.0-1.el9.x86_64.r 27 MB/s | 2.3 MB 00:00 2024-03-04T08:44:54.579 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 17/202 2024-03-04T08:44:54.610 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-03-04T08:44:54.633 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-03-04T08:44:54.649 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-03-04T08:44:54.654 INFO:teuthology.orchestra.run.smithi158.stdout:(153/176): NetworkManager-libnm-1.46.0-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-03-04T08:44:54.687 INFO:teuthology.orchestra.run.smithi158.stdout:(154/176): NetworkManager-team-1.46.0-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-03-04T08:44:54.728 INFO:teuthology.orchestra.run.smithi158.stdout:(155/176): NetworkManager-tui-1.46.0-1.el9.x86_ 5.8 MB/s | 247 kB 00:00 2024-03-04T08:44:54.762 INFO:teuthology.orchestra.run.smithi158.stdout:(156/176): acl-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 69 kB 00:00 2024-03-04T08:44:54.812 INFO:teuthology.orchestra.run.smithi158.stdout:(157/176): librbd1-19.0.0-1739.g1bc250c7.el9.x8 6.8 MB/s | 3.1 MB 00:00 2024-03-04T08:44:54.836 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-03-04T08:44:54.837 INFO:teuthology.orchestra.run.smithi158.stdout:(158/176): elfutils-debuginfod-client-0.190-2.e 469 kB/s | 35 kB 00:00 2024-03-04T08:44:54.863 INFO:teuthology.orchestra.run.smithi158.stdout:(159/176): elfutils-libelf-0.190-2.el9.x86_64.r 3.7 MB/s | 191 kB 00:00 2024-03-04T08:44:54.863 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-03-04T08:44:54.888 INFO:teuthology.orchestra.run.smithi158.stdout:(160/176): elfutils-libs-0.190-2.el9.x86_64.rpm 5.0 MB/s | 254 kB 00:00 2024-03-04T08:44:54.904 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-03-04T08:44:54.922 INFO:teuthology.orchestra.run.smithi158.stdout:(161/176): iproute-6.7.0-2.el9.x86_64.rpm 13 MB/s | 802 kB 00:00 2024-03-04T08:44:54.956 INFO:teuthology.orchestra.run.smithi158.stdout:(162/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 6.5 MB/s | 433 kB 00:00 2024-03-04T08:44:54.981 INFO:teuthology.orchestra.run.smithi158.stdout:(163/176): libacl-2.3.1-4.el9.x86_64.rpm 380 kB/s | 22 kB 00:00 2024-03-04T08:44:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-03-04T08:44:55.023 INFO:teuthology.orchestra.run.smithi158.stdout:(164/176): libblkid-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 107 kB 00:00 2024-03-04T08:44:55.048 INFO:teuthology.orchestra.run.smithi158.stdout:(165/176): libfdisk-2.37.4-18.el9.x86_64.rpm 2.2 MB/s | 154 kB 00:00 2024-03-04T08:44:55.074 INFO:teuthology.orchestra.run.smithi158.stdout:(166/176): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 135 kB 00:00 2024-03-04T08:44:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/202 2024-03-04T08:44:55.099 INFO:teuthology.orchestra.run.smithi158.stdout:(167/176): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 63 kB 00:00 2024-03-04T08:44:55.124 INFO:teuthology.orchestra.run.smithi158.stdout:(168/176): libuuid-2.37.4-18.el9.x86_64.rpm 552 kB/s | 28 kB 00:00 2024-03-04T08:44:55.158 INFO:teuthology.orchestra.run.smithi158.stdout:(169/176): openssl-3.0.7-27.el9.x86_64.rpm 20 MB/s | 1.2 MB 00:00 2024-03-04T08:44:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/202 2024-03-04T08:44:55.223 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-04T08:44:55.224 INFO:teuthology.orchestra.run.smithi158.stdout:(170/176): librados2-19.0.0-1739.g1bc250c7.el9. 3.5 MB/s | 3.4 MB 00:00 2024-03-04T08:44:55.275 INFO:teuthology.orchestra.run.smithi158.stdout:(171/176): openssl-libs-3.0.7-27.el9.x86_64.rpm 14 MB/s | 2.2 MB 00:00 2024-03-04T08:44:55.293 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-04T08:44:55.317 INFO:teuthology.orchestra.run.smithi158.stdout:(172/176): util-linux-2.37.4-18.el9.x86_64.rpm 14 MB/s | 2.2 MB 00:00 2024-03-04T08:44:55.342 INFO:teuthology.orchestra.run.smithi158.stdout:(173/176): util-linux-core-2.37.4-18.el9.x86_64 3.8 MB/s | 438 kB 00:00 2024-03-04T08:44:55.367 INFO:teuthology.orchestra.run.smithi158.stdout:(174/176): util-linux-user-2.37.4-18.el9.x86_64 334 kB/s | 30 kB 00:00 2024-03-04T08:44:55.401 INFO:teuthology.orchestra.run.smithi158.stdout:(175/176): xfsdump-3.1.12-4.el9.x86_64.rpm 5.4 MB/s | 321 kB 00:00 2024-03-04T08:44:55.434 INFO:teuthology.orchestra.run.smithi158.stdout:(176/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 12 MB/s | 1.1 MB 00:00 2024-03-04T08:44:55.442 INFO:teuthology.orchestra.run.smithi158.stdout:-------------------------------------------------------------------------------- 2024-03-04T08:44:55.442 INFO:teuthology.orchestra.run.smithi158.stdout:Total 12 MB/s | 224 MB 00:17 2024-03-04T08:44:55.490 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction check 2024-03-04T08:44:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-04T08:44:55.628 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-03-04T08:44:55.736 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-03-04T08:44:55.769 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction check succeeded. 2024-03-04T08:44:55.769 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction test 2024-03-04T08:44:55.772 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-03-04T08:44:55.850 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-03-04T08:44:55.904 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-03-04T08:44:55.934 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-03-04T08:44:55.970 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-03-04T08:44:56.010 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-03-04T08:44:56.074 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-03-04T08:44:56.109 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-03-04T08:44:56.144 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-03-04T08:44:56.177 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-03-04T08:44:56.204 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-03-04T08:44:56.273 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-03-04T08:44:56.309 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9. 39/202 2024-03-04T08:44:56.352 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el 40/202 2024-03-04T08:44:56.381 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_6 41/202 2024-03-04T08:44:56.450 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-03-04T08:44:56.496 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-03-04T08:44:56.569 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-03-04T08:44:56.703 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-03-04T08:44:56.757 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-03-04T08:44:57.274 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-03-04T08:44:57.352 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-03-04T08:44:57.421 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-03-04T08:44:57.446 INFO:teuthology.orchestra.run.smithi158.stdout:Transaction test succeeded. 2024-03-04T08:44:57.446 INFO:teuthology.orchestra.run.smithi158.stdout:Running transaction 2024-03-04T08:44:57.472 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-03-04T08:44:57.480 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-03-04T08:44:57.518 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-03-04T08:44:57.557 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_6 52/202 2024-03-04T08:44:57.588 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-03-04T08:44:57.680 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-03-04T08:44:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-03-04T08:44:57.975 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-03-04T08:44:58.158 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-03-04T08:44:58.216 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-03-04T08:44:58.259 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-03-04T08:44:58.393 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-03-04T08:44:58.438 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 61/202 2024-03-04T08:44:58.492 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-03-04T08:44:58.565 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-03-04T08:44:58.596 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-03-04T08:44:58.633 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-03-04T08:44:58.680 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-03-04T08:44:58.707 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-03-04T08:44:58.751 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-03-04T08:44:58.952 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-03-04T08:44:59.017 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-03-04T08:44:59.087 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-03-04T08:44:59.111 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/202 2024-03-04T08:44:59.131 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-03-04T08:44:59.149 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-03-04T08:44:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-03-04T08:44:59.203 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-03-04T08:44:59.212 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-03-04T08:44:59.236 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-03-04T08:44:59.291 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-03-04T08:44:59.298 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-03-04T08:44:59.332 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-03-04T08:44:59.337 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 75/202 2024-03-04T08:44:59.377 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-03-04T08:44:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-03-04T08:44:59.410 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-03-04T08:44:59.427 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-03-04T08:44:59.450 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-03-04T08:44:59.476 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-03-04T08:44:59.499 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-03-04T08:44:59.524 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-03-04T08:44:59.628 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-03-04T08:44:59.735 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-03-04T08:44:59.777 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-03-04T08:44:59.825 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-03-04T08:44:59.848 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-03-04T08:44:59.880 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-03-04T08:44:59.918 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-04T08:45:00.063 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-04T08:45:00.064 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-03-04T08:45:00.094 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-04T08:45:00.106 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-03-04T08:45:00.186 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-04T08:45:00.222 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 13/202 2024-03-04T08:45:00.234 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-03-04T08:45:00.279 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-03-04T08:45:00.320 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-03-04T08:45:00.352 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-03-04T08:45:00.384 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-03-04T08:45:00.395 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-03-04T08:45:00.427 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-03-04T08:45:00.438 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-03-04T08:45:00.454 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-03-04T08:45:00.465 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-04T08:45:00.489 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-03-04T08:45:00.528 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-03-04T08:45:00.552 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-04T08:45:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-03-04T08:45:00.574 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 17/202 2024-03-04T08:45:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-03-04T08:45:00.647 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 17/202 2024-03-04T08:45:00.666 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-03-04T08:45:00.687 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-03-04T08:45:00.700 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-03-04T08:45:00.709 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-03-04T08:45:00.725 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-03-04T08:45:00.733 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-03-04T08:45:00.774 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-03-04T08:45:00.888 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-03-04T08:45:00.915 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-03-04T08:45:00.950 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-03-04T08:45:00.990 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-03-04T08:45:01.099 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-03-04T08:45:01.176 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-03-04T08:45:01.300 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/202 2024-03-04T08:45:01.305 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-03-04T08:45:01.348 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-03-04T08:45:01.360 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/202 2024-03-04T08:45:01.380 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-03-04T08:45:01.426 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-03-04T08:45:01.499 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-04T08:45:01.525 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-03-04T08:45:01.553 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-04T08:45:01.575 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-03-04T08:45:01.752 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-04T08:45:01.874 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-03-04T08:45:01.961 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-03-04T08:45:02.000 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-03-04T08:45:02.068 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-03-04T08:45:02.121 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-03-04T08:45:02.153 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-03-04T08:45:02.188 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-03-04T08:45:02.228 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-03-04T08:45:02.292 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-03-04T08:45:02.305 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-03-04T08:45:02.327 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-03-04T08:45:02.353 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-03-04T08:45:02.370 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-03-04T08:45:02.387 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-03-04T08:45:02.412 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-03-04T08:45:02.435 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-03-04T08:45:02.439 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-03-04T08:45:02.509 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-03-04T08:45:02.522 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-03-04T08:45:02.544 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9. 39/202 2024-03-04T08:45:02.571 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-03-04T08:45:02.587 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el 40/202 2024-03-04T08:45:02.616 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_6 41/202 2024-03-04T08:45:02.685 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-03-04T08:45:02.731 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-03-04T08:45:02.750 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-03-04T08:45:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-03-04T08:45:02.804 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-03-04T08:45:02.846 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-03-04T08:45:02.938 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-03-04T08:45:02.991 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-03-04T08:45:03.488 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-03-04T08:45:03.528 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-03-04T08:45:03.599 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-03-04T08:45:03.603 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-03-04T08:45:03.681 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-03-04T08:45:03.731 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-03-04T08:45:03.740 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-03-04T08:45:03.778 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-03-04T08:45:03.816 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_6 52/202 2024-03-04T08:45:03.856 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-03-04T08:45:03.965 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-03-04T08:45:04.033 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-03-04T08:45:04.321 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-03-04T08:45:04.636 INFO:teuthology.orchestra.run.smithi158.stdout: Preparing : 1/1 2024-03-04T08:45:04.735 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-03-04T08:45:04.771 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/202 2024-03-04T08:45:04.814 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-03-04T08:45:04.821 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-03-04T08:45:04.828 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-03-04T08:45:04.853 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-03-04T08:45:04.874 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-03-04T08:45:04.875 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-03-04T08:45:04.905 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-03-04T08:45:04.932 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-03-04T08:45:04.932 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-03-04T08:45:04.978 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-03-04T08:45:05.012 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-03-04T08:45:05.032 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-03-04T08:45:05.049 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 61/202 2024-03-04T08:45:05.075 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-03-04T08:45:05.100 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-03-04T08:45:05.104 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-03-04T08:45:05.142 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-03-04T08:45:05.176 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-03-04T08:45:05.208 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-03-04T08:45:05.244 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-03-04T08:45:05.244 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-03-04T08:45:05.270 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 125/202 2024-03-04T08:45:05.292 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-03-04T08:45:05.327 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-03-04T08:45:05.345 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 125/202 2024-03-04T08:45:05.370 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-03-04T08:45:05.375 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 126/202 2024-03-04T08:45:05.412 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-03-04T08:45:05.419 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-03-04T08:45:05.469 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-03-04T08:45:05.543 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-04T08:45:05.555 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-03-04T08:45:05.610 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-03-04T08:45:05.640 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-03-04T08:45:05.664 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-04T08:45:05.681 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-03-04T08:45:05.702 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-04T08:45:05.775 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-03-04T08:45:05.792 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-04T08:45:05.814 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 75/202 2024-03-04T08:45:05.822 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 13/202 2024-03-04T08:45:05.855 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-03-04T08:45:05.879 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-03-04T08:45:05.880 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-03-04T08:45:05.905 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-03-04T08:45:05.915 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 129/202 2024-03-04T08:45:05.978 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 129/202 2024-03-04T08:45:05.981 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-03-04T08:45:05.984 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-03-04T08:45:06.025 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 130/202 2024-03-04T08:45:06.036 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-03-04T08:45:06.065 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-04T08:45:06.115 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-03-04T08:45:06.168 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-04T08:45:06.199 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 17/202 2024-03-04T08:45:06.223 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-03-04T08:45:06.263 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 17/202 2024-03-04T08:45:06.263 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-03-04T08:45:06.302 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-03-04T08:45:06.312 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-03-04T08:45:06.334 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-03-04T08:45:06.349 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-03-04T08:45:06.366 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-03-04T08:45:06.542 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-03-04T08:45:06.549 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-03-04T08:45:06.573 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-03-04T08:45:06.600 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-03-04T08:45:06.604 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-03-04T08:45:06.727 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-03-04T08:45:06.728 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-03-04T08:45:06.765 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-03-04T08:45:06.797 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-03-04T08:45:06.815 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/202 2024-03-04T08:45:06.829 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-03-04T08:45:06.865 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-03-04T08:45:06.895 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-03-04T08:45:06.903 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/202 2024-03-04T08:45:06.923 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-03-04T08:45:06.957 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-03-04T08:45:06.963 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-04T08:45:06.997 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-03-04T08:45:07.028 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-03-04T08:45:07.043 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-04T08:45:07.058 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-03-04T08:45:07.134 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-03-04T08:45:07.168 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-03-04T08:45:07.201 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-03-04T08:45:07.243 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-03-04T08:45:07.264 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-04T08:45:07.278 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-03-04T08:45:07.368 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-03-04T08:45:07.373 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-03-04T08:45:07.456 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-03-04T08:45:07.468 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-03-04T08:45:07.500 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-03-04T08:45:07.500 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-03-04T08:45:07.532 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-03-04T08:45:07.575 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-03-04T08:45:07.578 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-03-04T08:45:07.638 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-03-04T08:45:07.668 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-03-04T08:45:07.676 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-03-04T08:45:07.712 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-03-04T08:45:07.731 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-03-04T08:45:07.752 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-03-04T08:45:07.824 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-03-04T08:45:07.859 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-03-04T08:45:07.902 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-03-04T08:45:07.943 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-03-04T08:45:07.971 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-03-04T08:45:08.041 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-03-04T08:45:08.076 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9. 39/202 2024-03-04T08:45:08.120 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el 40/202 2024-03-04T08:45:08.149 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_6 41/202 2024-03-04T08:45:08.217 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-03-04T08:45:08.260 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-04T08:45:08.263 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-03-04T08:45:08.297 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-04T08:45:08.336 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-03-04T08:45:08.458 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-03-04T08:45:08.477 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-03-04T08:45:08.511 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-03-04T08:45:08.532 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-03-04T08:45:08.545 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-03-04T08:45:08.594 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-03-04T08:45:08.681 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-03-04T08:45:08.721 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-03-04T08:45:08.839 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-04T08:45:08.875 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 132/202 2024-03-04T08:45:08.890 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-03-04T08:45:08.927 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-03-04T08:45:08.939 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 132/202 2024-03-04T08:45:08.939 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-04T08:45:08.939 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-04T08:45:08.939 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-04T08:45:08.968 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 133/202 2024-03-04T08:45:09.009 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-03-04T08:45:09.051 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-03-04T08:45:09.144 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-03-04T08:45:09.245 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-03-04T08:45:09.289 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-03-04T08:45:09.297 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-03-04T08:45:09.336 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-03-04T08:45:09.373 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_6 52/202 2024-03-04T08:45:09.405 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-03-04T08:45:09.519 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-03-04T08:45:09.581 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-03-04T08:45:09.645 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-03-04T08:45:09.758 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-03-04T08:45:09.829 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-03-04T08:45:10.003 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-03-04T08:45:10.058 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-03-04T08:45:10.103 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-03-04T08:45:10.237 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-03-04T08:45:10.289 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 61/202 2024-03-04T08:45:10.353 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-03-04T08:45:10.425 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-03-04T08:45:10.591 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-03-04T08:45:10.820 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-03-04T08:45:10.877 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-03-04T08:45:10.920 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-03-04T08:45:10.965 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-03-04T08:45:10.973 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-03-04T08:45:11.015 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-03-04T08:45:11.055 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-03-04T08:45:11.073 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-03-04T08:45:11.163 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-03-04T08:45:11.211 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-03-04T08:45:11.249 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-03-04T08:45:11.290 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-03-04T08:45:11.385 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-03-04T08:45:11.389 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-03-04T08:45:11.420 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 125/202 2024-03-04T08:45:11.432 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 75/202 2024-03-04T08:45:11.472 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-03-04T08:45:11.509 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 125/202 2024-03-04T08:45:11.531 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-03-04T08:45:11.541 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 126/202 2024-03-04T08:45:11.556 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-03-04T08:45:11.577 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-03-04T08:45:11.628 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-03-04T08:45:11.766 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-03-04T08:45:11.873 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-03-04T08:45:11.922 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-03-04T08:45:11.980 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-03-04T08:45:12.041 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-03-04T08:45:12.043 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-03-04T08:45:12.073 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 129/202 2024-03-04T08:45:12.144 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 129/202 2024-03-04T08:45:12.183 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 130/202 2024-03-04T08:45:12.232 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-03-04T08:45:12.276 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-03-04T08:45:12.416 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-03-04T08:45:12.467 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-03-04T08:45:12.500 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-03-04T08:45:12.531 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-03-04T08:45:12.567 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-03-04T08:45:12.606 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-03-04T08:45:12.642 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-03-04T08:45:12.677 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-03-04T08:45:12.716 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-03-04T08:45:12.765 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-03-04T08:45:12.795 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-03-04T08:45:12.872 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-03-04T08:45:12.904 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-03-04T08:45:12.938 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-03-04T08:45:12.987 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-03-04T08:45:13.023 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-03-04T08:45:13.112 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-03-04T08:45:13.202 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-03-04T08:45:13.253 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-03-04T08:45:13.293 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-03-04T08:45:13.340 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-03-04T08:45:13.446 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-03-04T08:45:13.503 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-03-04T08:45:14.239 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-03-04T08:45:14.292 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-03-04T08:45:14.325 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-03-04T08:45:14.344 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-04T08:45:14.373 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-03-04T08:45:14.378 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-04T08:45:14.462 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-03-04T08:45:14.493 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-03-04T08:45:14.672 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-03-04T08:45:14.699 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-03-04T08:45:14.776 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-03-04T08:45:14.920 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-04T08:45:14.957 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 132/202 2024-03-04T08:45:15.045 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 132/202 2024-03-04T08:45:15.046 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-04T08:45:15.046 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-04T08:45:15.046 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:45:15.083 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 133/202 2024-03-04T08:45:15.420 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-03-04T08:45:15.530 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-03-04T08:45:16.758 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-03-04T08:45:20.723 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 133/202 2024-03-04T08:45:20.723 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /sys 2024-03-04T08:45:20.723 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /proc 2024-03-04T08:45:20.723 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /mnt 2024-03-04T08:45:20.723 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /var/tmp 2024-03-04T08:45:20.723 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /home 2024-03-04T08:45:20.724 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /root 2024-03-04T08:45:20.724 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /tmp 2024-03-04T08:45:20.724 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-04T08:45:20.890 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 134/202 2024-03-04T08:45:20.908 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 134/202 2024-03-04T08:45:20.908 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:20.908 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-04T08:45:20.908 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-04T08:45:22.392 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 135/202 2024-03-04T08:45:22.418 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 135/202 2024-03-04T08:45:22.418 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:22.418 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-04T08:45:22.418 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-04T08:45:22.419 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-04T08:45:22.419 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-04T08:45:22.643 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 136/202 2024-03-04T08:45:22.666 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 136/202 2024-03-04T08:45:22.666 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:22.666 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-04T08:45:22.666 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-04T08:45:22.666 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-04T08:45:22.666 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-04T08:45:23.110 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 137/202 2024-03-04T08:45:23.133 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 137/202 2024-03-04T08:45:23.133 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:23.133 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-04T08:45:23.134 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-04T08:45:23.134 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-04T08:45:23.134 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-04T08:45:23.220 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-03-04T08:45:23.713 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 139/202 2024-03-04T08:45:23.744 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 139/202 2024-03-04T08:45:23.775 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.e 140/202 2024-03-04T08:45:23.838 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7. 141/202 2024-03-04T08:45:23.866 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 142/202 2024-03-04T08:45:24.096 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 142/202 2024-03-04T08:45:24.126 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 143/202 2024-03-04T08:45:24.847 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-03-04T08:45:24.975 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-03-04T08:45:25.096 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 143/202 2024-03-04T08:45:25.117 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 144/202 2024-03-04T08:45:25.221 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 144/202 2024-03-04T08:45:25.381 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el 145/202 2024-03-04T08:45:25.408 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-03-04T08:45:25.414 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 146/202 2024-03-04T08:45:25.434 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 146/202 2024-03-04T08:45:25.435 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:25.435 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-04T08:45:25.435 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-04T08:45:25.435 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-04T08:45:25.435 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-04T08:45:25.439 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 125/202 2024-03-04T08:45:25.479 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 147/202 2024-03-04T08:45:25.491 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 147/202 2024-03-04T08:45:25.513 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 125/202 2024-03-04T08:45:25.536 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 148/202 2024-03-04T08:45:25.544 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 126/202 2024-03-04T08:45:25.589 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-03-04T08:45:26.052 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-03-04T08:45:26.083 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 129/202 2024-03-04T08:45:26.137 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 129/202 2024-03-04T08:45:26.168 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 130/202 2024-03-04T08:45:26.389 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-03-04T08:45:26.422 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 150/202 2024-03-04T08:45:26.445 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 150/202 2024-03-04T08:45:26.445 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:26.445 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-04T08:45:26.446 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-04T08:45:26.446 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-04T08:45:26.446 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-04T08:45:26.481 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 151/202 2024-03-04T08:45:26.504 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 151/202 2024-03-04T08:45:26.504 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:26.504 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-04T08:45:26.504 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-04T08:45:26.557 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 152/202 2024-03-04T08:45:26.580 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 152/202 2024-03-04T08:45:26.580 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:26.580 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-04T08:45:26.580 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-04T08:45:26.580 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-04T08:45:26.580 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-04T08:45:26.701 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 133/202 2024-03-04T08:45:26.701 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /sys 2024-03-04T08:45:26.701 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /proc 2024-03-04T08:45:26.702 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /mnt 2024-03-04T08:45:26.702 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /var/tmp 2024-03-04T08:45:26.702 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /home 2024-03-04T08:45:26.702 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /root 2024-03-04T08:45:26.702 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /tmp 2024-03-04T08:45:26.702 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:45:26.869 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 134/202 2024-03-04T08:45:26.884 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 134/202 2024-03-04T08:45:26.884 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:26.884 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-04T08:45:26.884 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:45:26.891 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-04T08:45:26.914 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-04T08:45:26.914 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:26.914 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-04T08:45:26.914 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-04T08:45:26.914 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-04T08:45:26.914 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-04T08:45:28.356 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 135/202 2024-03-04T08:45:28.370 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-04T08:45:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 135/202 2024-03-04T08:45:28.381 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:28.381 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-04T08:45:28.382 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-04T08:45:28.382 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-04T08:45:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:45:28.415 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-04T08:45:28.606 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 136/202 2024-03-04T08:45:28.628 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 136/202 2024-03-04T08:45:28.628 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:28.628 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-04T08:45:28.628 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-04T08:45:28.628 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-04T08:45:28.628 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:45:28.948 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-04T08:45:28.985 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 132/202 2024-03-04T08:45:29.066 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 132/202 2024-03-04T08:45:29.067 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-04T08:45:29.067 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-04T08:45:29.067 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-03-04T08:45:29.073 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 137/202 2024-03-04T08:45:29.096 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 137/202 2024-03-04T08:45:29.096 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:29.096 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-04T08:45:29.096 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-04T08:45:29.096 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-04T08:45:29.096 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:45:29.103 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 133/202 2024-03-04T08:45:29.183 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-03-04T08:45:29.633 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 139/202 2024-03-04T08:45:29.665 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 139/202 2024-03-04T08:45:29.696 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.e 140/202 2024-03-04T08:45:29.756 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7. 141/202 2024-03-04T08:45:29.779 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 142/202 2024-03-04T08:45:30.007 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 142/202 2024-03-04T08:45:30.039 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 143/202 2024-03-04T08:45:30.993 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 143/202 2024-03-04T08:45:31.014 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 144/202 2024-03-04T08:45:31.114 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 144/202 2024-03-04T08:45:31.275 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el 145/202 2024-03-04T08:45:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 146/202 2024-03-04T08:45:31.329 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 154/202 2024-03-04T08:45:31.330 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 146/202 2024-03-04T08:45:31.330 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:31.330 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-04T08:45:31.331 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-04T08:45:31.331 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-04T08:45:31.331 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:45:31.370 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 155/202 2024-03-04T08:45:31.374 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 147/202 2024-03-04T08:45:31.385 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 147/202 2024-03-04T08:45:31.400 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 156/202 2024-03-04T08:45:31.427 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-03-04T08:45:31.434 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 148/202 2024-03-04T08:45:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 158/202 2024-03-04T08:45:31.718 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-03-04T08:45:31.759 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 160/202 2024-03-04T08:45:31.807 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-03-04T08:45:31.850 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-03-04T08:45:31.924 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-03-04T08:45:31.969 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-03-04T08:45:32.012 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-03-04T08:45:32.043 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_ 166/202 2024-03-04T08:45:32.073 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 167/202 2024-03-04T08:45:32.135 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 168/202 2024-03-04T08:45:32.180 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-03-04T08:45:32.223 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-03-04T08:45:32.312 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-03-04T08:45:32.343 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 150/202 2024-03-04T08:45:32.366 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 150/202 2024-03-04T08:45:32.366 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:32.366 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-04T08:45:32.366 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-04T08:45:32.367 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-04T08:45:32.367 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:45:32.419 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 151/202 2024-03-04T08:45:32.441 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 151/202 2024-03-04T08:45:32.441 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:32.441 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-04T08:45:32.441 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:45:32.486 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 152/202 2024-03-04T08:45:32.508 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 152/202 2024-03-04T08:45:32.508 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:32.508 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-04T08:45:32.508 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-04T08:45:32.509 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-04T08:45:32.509 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:45:32.824 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-04T08:45:32.829 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-04T08:45:32.851 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-04T08:45:32.851 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:32.851 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-04T08:45:32.851 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-04T08:45:32.851 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-04T08:45:32.851 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:45:32.855 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-04T08:45:32.875 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-04T08:45:33.713 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 172/202 2024-03-04T08:45:33.782 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 173/202 2024-03-04T08:45:33.848 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-03-04T08:45:33.930 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-03-04T08:45:33.969 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-03-04T08:45:33.994 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-03-04T08:45:34.020 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-03-04T08:45:34.046 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-03-04T08:45:34.046 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-03-04T08:45:34.131 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-03-04T08:45:34.158 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-03-04T08:45:34.189 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-03-04T08:45:34.189 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-03-04T08:45:34.257 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-03-04T08:45:34.283 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-03-04T08:45:34.317 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-03-04T08:45:34.341 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-03-04T08:45:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-03-04T08:45:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-03-04T08:45:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-03-04T08:45:34.473 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-03-04T08:45:34.506 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-03-04T08:45:34.531 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-03-04T08:45:34.557 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-03-04T08:45:34.582 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-03-04T08:45:34.607 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-03-04T08:45:34.614 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-04T08:45:34.614 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-04T08:45:34.801 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-04T08:45:34.802 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-03-04T08:45:34.893 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-03-04T08:45:34.924 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-03-04T08:45:34.951 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-03-04T08:45:34.976 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-03-04T08:45:35.002 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-03-04T08:45:35.065 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-03-04T08:45:37.257 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 154/202 2024-03-04T08:45:37.298 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 155/202 2024-03-04T08:45:37.329 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 156/202 2024-03-04T08:45:37.356 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-03-04T08:45:37.545 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 158/202 2024-03-04T08:45:37.663 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-03-04T08:45:37.704 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 160/202 2024-03-04T08:45:37.764 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-03-04T08:45:37.812 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-03-04T08:45:37.869 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-03-04T08:45:37.913 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-03-04T08:45:37.949 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-03-04T08:45:37.997 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_ 166/202 2024-03-04T08:45:38.026 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 167/202 2024-03-04T08:45:38.095 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 168/202 2024-03-04T08:45:38.152 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-03-04T08:45:38.191 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-03-04T08:45:38.753 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-04T08:45:38.789 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-04T08:45:38.808 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-04T08:45:39.617 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 172/202 2024-03-04T08:45:39.706 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 173/202 2024-03-04T08:45:39.750 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-03-04T08:45:39.831 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-03-04T08:45:39.862 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-03-04T08:45:39.886 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-03-04T08:45:39.916 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-03-04T08:45:39.923 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-03-04T08:45:39.923 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 1/202 2024-03-04T08:45:39.923 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 2/202 2024-03-04T08:45:39.923 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 3/202 2024-03-04T08:45:39.923 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 4/202 2024-03-04T08:45:39.923 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 5/202 2024-03-04T08:45:39.923 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 6/202 2024-03-04T08:45:39.923 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 7/202 2024-03-04T08:45:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 8/202 2024-03-04T08:45:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 9/202 2024-03-04T08:45:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 10/202 2024-03-04T08:45:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-04T08:45:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-04T08:45:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 13/202 2024-03-04T08:45:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 14/202 2024-03-04T08:45:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_ 15/202 2024-03-04T08:45:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-04T08:45:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 17/202 2024-03-04T08:45:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_6 18/202 2024-03-04T08:45:39.924 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 19/202 2024-03-04T08:45:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 20/202 2024-03-04T08:45:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el 21/202 2024-03-04T08:45:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9. 22/202 2024-03-04T08:45:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_6 23/202 2024-03-04T08:45:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 24/202 2024-03-04T08:45:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 25/202 2024-03-04T08:45:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 26/202 2024-03-04T08:45:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 27/202 2024-03-04T08:45:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 28/202 2024-03-04T08:45:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 29/202 2024-03-04T08:45:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7. 30/202 2024-03-04T08:45:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 31/202 2024-03-04T08:45:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 32/202 2024-03-04T08:45:39.925 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 33/202 2024-03-04T08:45:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el 34/202 2024-03-04T08:45:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 35/202 2024-03-04T08:45:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.e 36/202 2024-03-04T08:45:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 37/202 2024-03-04T08:45:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 38/202 2024-03-04T08:45:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 39/202 2024-03-04T08:45:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-03-04T08:45:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-03-04T08:45:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-03-04T08:45:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-03-04T08:45:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-03-04T08:45:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-03-04T08:45:39.926 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-03-04T08:45:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-03-04T08:45:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-03-04T08:45:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-03-04T08:45:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-03-04T08:45:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-03-04T08:45:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-03-04T08:45:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-03-04T08:45:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-03-04T08:45:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-03-04T08:45:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-03-04T08:45:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-03-04T08:45:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-03-04T08:45:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 59/202 2024-03-04T08:45:39.927 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-03-04T08:45:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-03-04T08:45:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-03-04T08:45:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-03-04T08:45:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-03-04T08:45:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-03-04T08:45:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-03-04T08:45:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-03-04T08:45:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-03-04T08:45:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-03-04T08:45:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-03-04T08:45:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-03-04T08:45:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-03-04T08:45:39.928 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-03-04T08:45:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-03-04T08:45:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-03-04T08:45:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-03-04T08:45:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-03-04T08:45:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-03-04T08:45:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-03-04T08:45:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-03-04T08:45:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-03-04T08:45:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-03-04T08:45:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-03-04T08:45:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-03-04T08:45:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-03-04T08:45:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-03-04T08:45:39.929 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-03-04T08:45:39.930 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-03-04T08:45:39.930 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-03-04T08:45:39.930 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-03-04T08:45:39.930 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-03-04T08:45:39.930 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-03-04T08:45:39.930 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-03-04T08:45:39.930 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-03-04T08:45:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-03-04T08:45:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-03-04T08:45:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-03-04T08:45:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-03-04T08:45:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-03-04T08:45:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-03-04T08:45:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-03-04T08:45:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-03-04T08:45:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-03-04T08:45:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-03-04T08:45:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-03-04T08:45:39.931 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-03-04T08:45:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-03-04T08:45:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-03-04T08:45:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-03-04T08:45:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-03-04T08:45:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-03-04T08:45:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-03-04T08:45:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-03-04T08:45:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-03-04T08:45:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-03-04T08:45:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-03-04T08:45:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-03-04T08:45:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-03-04T08:45:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-03-04T08:45:39.932 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-03-04T08:45:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-03-04T08:45:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-03-04T08:45:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-03-04T08:45:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/202 2024-03-04T08:45:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-03-04T08:45:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-03-04T08:45:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-03-04T08:45:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-03-04T08:45:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-03-04T08:45:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-03-04T08:45:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-03-04T08:45:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-03-04T08:45:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-03-04T08:45:39.933 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-03-04T08:45:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-03-04T08:45:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-03-04T08:45:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-03-04T08:45:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-03-04T08:45:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-03-04T08:45:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-03-04T08:45:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-03-04T08:45:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-03-04T08:45:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-03-04T08:45:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-03-04T08:45:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-03-04T08:45:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-03-04T08:45:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-03-04T08:45:39.934 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-03-04T08:45:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-03-04T08:45:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-03-04T08:45:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 151/202 2024-03-04T08:45:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-03-04T08:45:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-04T08:45:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-03-04T08:45:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-03-04T08:45:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-03-04T08:45:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/202 2024-03-04T08:45:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-03-04T08:45:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/202 2024-03-04T08:45:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-03-04T08:45:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/202 2024-03-04T08:45:39.935 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-03-04T08:45:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/202 2024-03-04T08:45:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-03-04T08:45:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-03-04T08:45:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-03-04T08:45:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-03-04T08:45:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-03-04T08:45:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-03-04T08:45:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-03-04T08:45:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-03-04T08:45:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-03-04T08:45:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-03-04T08:45:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-03-04T08:45:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-03-04T08:45:39.936 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-03-04T08:45:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-03-04T08:45:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-03-04T08:45:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-03-04T08:45:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-03-04T08:45:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-03-04T08:45:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-03-04T08:45:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-03-04T08:45:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-03-04T08:45:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-03-04T08:45:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-03-04T08:45:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-03-04T08:45:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-03-04T08:45:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 189/202 2024-03-04T08:45:39.937 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-03-04T08:45:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 191/202 2024-03-04T08:45:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-03-04T08:45:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-03-04T08:45:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-03-04T08:45:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-03-04T08:45:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-03-04T08:45:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-03-04T08:45:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-03-04T08:45:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-03-04T08:45:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-03-04T08:45:39.938 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-03-04T08:45:39.939 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-03-04T08:45:39.939 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-03-04T08:45:40.015 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-03-04T08:45:40.042 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-03-04T08:45:40.073 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-03-04T08:45:40.073 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-03-04T08:45:40.141 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-03-04T08:45:40.167 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-03-04T08:45:40.201 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-03-04T08:45:40.234 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-03-04T08:45:40.281 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-03-04T08:45:40.310 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-03-04T08:45:40.351 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-03-04T08:45:40.407 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-03-04T08:45:40.442 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-03-04T08:45:40.467 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-03-04T08:45:40.492 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-03-04T08:45:40.517 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-03-04T08:45:40.543 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-03-04T08:45:40.549 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-04T08:45:40.549 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-04T08:45:40.729 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-04T08:45:40.729 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-03-04T08:45:40.811 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-03-04T08:45:40.844 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-03-04T08:45:40.870 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-03-04T08:45:40.895 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-03-04T08:45:40.921 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-03-04T08:45:40.987 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-03-04T08:45:42.756 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-03-04T08:45:42.756 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-04T08:45:42.757 INFO:teuthology.orchestra.run.smithi150.stdout:Upgraded: 2024-03-04T08:45:42.757 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-04T08:45:42.757 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-04T08:45:42.757 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-04T08:45:42.757 INFO:teuthology.orchestra.run.smithi150.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-04T08:45:42.757 INFO:teuthology.orchestra.run.smithi150.stdout: acl-2.3.1-4.el9.x86_64 2024-03-04T08:45:42.757 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-04T08:45:42.757 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-04T08:45:42.757 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-04T08:45:42.757 INFO:teuthology.orchestra.run.smithi150.stdout: iproute-6.7.0-2.el9.x86_64 2024-03-04T08:45:42.757 INFO:teuthology.orchestra.run.smithi150.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-03-04T08:45:42.757 INFO:teuthology.orchestra.run.smithi150.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-04T08:45:42.757 INFO:teuthology.orchestra.run.smithi150.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-04T08:45:42.758 INFO:teuthology.orchestra.run.smithi150.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-04T08:45:42.758 INFO:teuthology.orchestra.run.smithi150.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-04T08:45:42.758 INFO:teuthology.orchestra.run.smithi150.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-04T08:45:42.758 INFO:teuthology.orchestra.run.smithi150.stdout: librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.758 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.758 INFO:teuthology.orchestra.run.smithi150.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-04T08:45:42.758 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-04T08:45:42.758 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-04T08:45:42.758 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-04T08:45:42.758 INFO:teuthology.orchestra.run.smithi150.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-04T08:45:42.758 INFO:teuthology.orchestra.run.smithi150.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-04T08:45:42.758 INFO:teuthology.orchestra.run.smithi150.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-04T08:45:42.758 INFO:teuthology.orchestra.run.smithi150.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-04T08:45:42.759 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-04T08:45:42.759 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-03-04T08:45:42.759 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-04T08:45:42.759 INFO:teuthology.orchestra.run.smithi150.stdout: automake-1.16.2-8.el9.noarch 2024-03-04T08:45:42.759 INFO:teuthology.orchestra.run.smithi150.stdout: bison-3.7.4-5.el9.x86_64 2024-03-04T08:45:42.759 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-04T08:45:42.759 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.759 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.759 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.759 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.759 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:45:42.759 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.759 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.759 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.760 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:45:42.760 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:45:42.760 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:45:42.760 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:45:42.760 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:45:42.760 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.760 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.760 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:45:42.760 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.760 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.760 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.760 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:45:42.761 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:45:42.761 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.761 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.761 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:45:42.761 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-04T08:45:42.761 INFO:teuthology.orchestra.run.smithi150.stdout: flex-2.6.4-9.el9.x86_64 2024-03-04T08:45:42.761 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-04T08:45:42.761 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-04T08:45:42.761 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-04T08:45:42.761 INFO:teuthology.orchestra.run.smithi150.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-04T08:45:42.761 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-04T08:45:42.761 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-04T08:45:42.761 INFO:teuthology.orchestra.run.smithi150.stdout: jq-1.6-16.el9.x86_64 2024-03-04T08:45:42.761 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-04T08:45:42.762 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-04T08:45:42.762 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-04T08:45:42.762 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.762 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.762 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.762 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-04T08:45:42.762 INFO:teuthology.orchestra.run.smithi150.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-04T08:45:42.762 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-04T08:45:42.762 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-04T08:45:42.762 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-04T08:45:42.762 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.762 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.762 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-04T08:45:42.763 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.763 INFO:teuthology.orchestra.run.smithi150.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-04T08:45:42.763 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-04T08:45:42.763 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-04T08:45:42.763 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-04T08:45:42.763 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-04T08:45:42.763 INFO:teuthology.orchestra.run.smithi150.stdout: lua-5.4.4-4.el9.x86_64 2024-03-04T08:45:42.763 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-04T08:45:42.763 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-04T08:45:42.763 INFO:teuthology.orchestra.run.smithi150.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-04T08:45:42.763 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-04T08:45:42.763 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-04T08:45:42.763 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-04T08:45:42.763 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-04T08:45:42.764 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-13.11-1.el9.x86_64 2024-03-04T08:45:42.764 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-04T08:45:42.764 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-04T08:45:42.764 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-04T08:45:42.764 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-04T08:45:42.764 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-04T08:45:42.764 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-04T08:45:42.764 INFO:teuthology.orchestra.run.smithi150.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-04T08:45:42.764 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-04T08:45:42.764 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-04T08:45:42.764 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-04T08:45:42.764 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.764 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.765 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.765 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-04T08:45:42.765 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-04T08:45:42.765 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-04T08:45:42.765 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-04T08:45:42.765 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-04T08:45:42.765 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-04T08:45:42.765 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-04T08:45:42.765 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-04T08:45:42.765 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-04T08:45:42.765 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-04T08:45:42.765 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-04T08:45:42.765 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-04T08:45:42.766 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-04T08:45:42.766 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-04T08:45:42.766 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-04T08:45:42.766 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-04T08:45:42.766 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-04T08:45:42.766 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-04T08:45:42.766 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-04T08:45:42.766 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-04T08:45:42.766 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-04T08:45:42.766 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-04T08:45:42.766 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-04T08:45:42.766 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-04T08:45:42.766 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-04T08:45:42.767 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-04T08:45:42.767 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-04T08:45:42.767 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-04T08:45:42.767 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-04T08:45:42.767 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-04T08:45:42.767 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply-3.11-14.el9.noarch 2024-03-04T08:45:42.767 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-04T08:45:42.767 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-04T08:45:42.767 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-04T08:45:42.767 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-04T08:45:42.767 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-04T08:45:42.767 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-04T08:45:42.767 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-04T08:45:42.767 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-04T08:45:42.768 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-04T08:45:42.768 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.768 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.768 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-04T08:45:42.768 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-04T08:45:42.768 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-04T08:45:42.768 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.768 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-04T08:45:42.768 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-04T08:45:42.768 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-04T08:45:42.768 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-04T08:45:42.768 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-04T08:45:42.768 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-04T08:45:42.769 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-04T08:45:42.769 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-04T08:45:42.769 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-04T08:45:42.769 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-04T08:45:42.769 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-04T08:45:42.769 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-04T08:45:42.769 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-04T08:45:42.769 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-04T08:45:42.769 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-04T08:45:42.769 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.769 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.769 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:42.769 INFO:teuthology.orchestra.run.smithi150.stdout: re2-1:20211101-3.el9.x86_64 2024-03-04T08:45:42.769 INFO:teuthology.orchestra.run.smithi150.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-04T08:45:42.770 INFO:teuthology.orchestra.run.smithi150.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-04T08:45:42.770 INFO:teuthology.orchestra.run.smithi150.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-04T08:45:42.770 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-04T08:45:42.770 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-04T08:45:42.770 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-04T08:45:42.770 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-03-04T08:45:42.770 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-03-04T08:45:43.437 DEBUG:teuthology.parallel:result is None 2024-03-04T08:45:44.650 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 133/202 2024-03-04T08:45:44.650 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /sys 2024-03-04T08:45:44.650 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /proc 2024-03-04T08:45:44.651 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /mnt 2024-03-04T08:45:44.651 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /var/tmp 2024-03-04T08:45:44.651 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /home 2024-03-04T08:45:44.651 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /root 2024-03-04T08:45:44.651 INFO:teuthology.orchestra.run.smithi158.stdout:skipping the directory /tmp 2024-03-04T08:45:44.651 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-03-04T08:45:44.828 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 134/202 2024-03-04T08:45:44.843 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 134/202 2024-03-04T08:45:44.843 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:44.843 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-04T08:45:44.843 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-03-04T08:45:45.812 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-03-04T08:45:45.812 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 1/202 2024-03-04T08:45:45.812 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 2/202 2024-03-04T08:45:45.812 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 3/202 2024-03-04T08:45:45.812 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 4/202 2024-03-04T08:45:45.812 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 5/202 2024-03-04T08:45:45.812 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 6/202 2024-03-04T08:45:45.812 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 7/202 2024-03-04T08:45:45.812 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 8/202 2024-03-04T08:45:45.812 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 9/202 2024-03-04T08:45:45.813 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 10/202 2024-03-04T08:45:45.813 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-04T08:45:45.813 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-04T08:45:45.813 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 13/202 2024-03-04T08:45:45.813 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 14/202 2024-03-04T08:45:45.813 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_ 15/202 2024-03-04T08:45:45.813 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-04T08:45:45.813 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 17/202 2024-03-04T08:45:45.813 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_6 18/202 2024-03-04T08:45:45.813 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 19/202 2024-03-04T08:45:45.813 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 20/202 2024-03-04T08:45:45.813 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el 21/202 2024-03-04T08:45:45.813 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9. 22/202 2024-03-04T08:45:45.813 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_6 23/202 2024-03-04T08:45:45.814 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 24/202 2024-03-04T08:45:45.814 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 25/202 2024-03-04T08:45:45.814 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 26/202 2024-03-04T08:45:45.832 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 27/202 2024-03-04T08:45:45.832 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 28/202 2024-03-04T08:45:45.832 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 29/202 2024-03-04T08:45:45.832 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7. 30/202 2024-03-04T08:45:45.833 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 31/202 2024-03-04T08:45:45.833 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 32/202 2024-03-04T08:45:45.833 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 33/202 2024-03-04T08:45:45.833 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el 34/202 2024-03-04T08:45:45.833 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 35/202 2024-03-04T08:45:45.833 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.e 36/202 2024-03-04T08:45:45.833 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 37/202 2024-03-04T08:45:45.833 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 38/202 2024-03-04T08:45:45.833 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 39/202 2024-03-04T08:45:45.834 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-03-04T08:45:45.834 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-03-04T08:45:45.834 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-03-04T08:45:45.834 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-03-04T08:45:45.834 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-03-04T08:45:45.834 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-03-04T08:45:45.834 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-03-04T08:45:45.834 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-03-04T08:45:45.834 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-03-04T08:45:45.835 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-03-04T08:45:45.835 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-03-04T08:45:45.835 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-03-04T08:45:45.835 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-03-04T08:45:45.835 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-03-04T08:45:45.835 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-03-04T08:45:45.835 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-03-04T08:45:45.835 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-03-04T08:45:45.836 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-03-04T08:45:45.836 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-03-04T08:45:45.836 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 59/202 2024-03-04T08:45:45.836 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-03-04T08:45:45.836 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-03-04T08:45:45.836 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-03-04T08:45:45.836 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-03-04T08:45:45.836 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-03-04T08:45:45.836 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-03-04T08:45:45.836 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-03-04T08:45:45.836 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-03-04T08:45:45.837 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-03-04T08:45:45.837 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-03-04T08:45:45.837 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-03-04T08:45:45.837 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-03-04T08:45:45.837 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-03-04T08:45:45.837 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-03-04T08:45:45.837 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-03-04T08:45:45.837 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-03-04T08:45:45.837 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-03-04T08:45:45.837 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-03-04T08:45:45.837 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-03-04T08:45:45.837 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-03-04T08:45:45.837 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-03-04T08:45:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-03-04T08:45:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-03-04T08:45:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-03-04T08:45:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-03-04T08:45:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-03-04T08:45:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-03-04T08:45:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-03-04T08:45:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-03-04T08:45:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-03-04T08:45:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-03-04T08:45:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-03-04T08:45:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-03-04T08:45:45.838 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-03-04T08:45:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-03-04T08:45:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-03-04T08:45:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-03-04T08:45:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-03-04T08:45:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-03-04T08:45:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-03-04T08:45:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-03-04T08:45:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-03-04T08:45:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-03-04T08:45:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-03-04T08:45:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-03-04T08:45:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-03-04T08:45:45.839 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-03-04T08:45:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-03-04T08:45:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-03-04T08:45:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-03-04T08:45:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-03-04T08:45:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-03-04T08:45:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-03-04T08:45:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-03-04T08:45:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-03-04T08:45:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-03-04T08:45:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-03-04T08:45:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-03-04T08:45:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-03-04T08:45:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-03-04T08:45:45.840 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-03-04T08:45:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-03-04T08:45:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-03-04T08:45:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-03-04T08:45:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/202 2024-03-04T08:45:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-03-04T08:45:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-03-04T08:45:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-03-04T08:45:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-03-04T08:45:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-03-04T08:45:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-03-04T08:45:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-03-04T08:45:45.841 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-03-04T08:45:45.842 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-03-04T08:45:45.842 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-03-04T08:45:45.842 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-03-04T08:45:45.842 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-03-04T08:45:45.842 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-03-04T08:45:45.842 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-03-04T08:45:45.842 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-03-04T08:45:45.842 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-03-04T08:45:45.842 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-03-04T08:45:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-03-04T08:45:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-03-04T08:45:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-03-04T08:45:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-03-04T08:45:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-03-04T08:45:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-03-04T08:45:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-03-04T08:45:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-03-04T08:45:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-03-04T08:45:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 151/202 2024-03-04T08:45:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-03-04T08:45:45.843 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-04T08:45:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-03-04T08:45:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-03-04T08:45:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-03-04T08:45:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/202 2024-03-04T08:45:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-03-04T08:45:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/202 2024-03-04T08:45:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-03-04T08:45:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/202 2024-03-04T08:45:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-03-04T08:45:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/202 2024-03-04T08:45:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-03-04T08:45:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-03-04T08:45:45.844 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-03-04T08:45:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-03-04T08:45:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-03-04T08:45:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-03-04T08:45:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-03-04T08:45:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-03-04T08:45:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-03-04T08:45:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-03-04T08:45:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-03-04T08:45:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-03-04T08:45:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-03-04T08:45:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-03-04T08:45:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-03-04T08:45:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-03-04T08:45:45.845 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-03-04T08:45:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-03-04T08:45:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-03-04T08:45:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-03-04T08:45:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-03-04T08:45:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-03-04T08:45:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-03-04T08:45:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-03-04T08:45:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-03-04T08:45:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 189/202 2024-03-04T08:45:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-03-04T08:45:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 191/202 2024-03-04T08:45:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-03-04T08:45:45.846 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-03-04T08:45:45.847 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-03-04T08:45:45.847 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-03-04T08:45:45.847 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-03-04T08:45:45.847 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-03-04T08:45:45.847 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-03-04T08:45:45.847 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-03-04T08:45:45.847 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-03-04T08:45:45.847 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-03-04T08:45:48.810 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-03-04T08:45:48.810 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:45:48.810 INFO:teuthology.orchestra.run.smithi071.stdout:Upgraded: 2024-03-04T08:45:48.810 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-04T08:45:48.810 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-04T08:45:48.810 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-04T08:45:48.810 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-04T08:45:48.810 INFO:teuthology.orchestra.run.smithi071.stdout: acl-2.3.1-4.el9.x86_64 2024-03-04T08:45:48.810 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-04T08:45:48.811 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-04T08:45:48.811 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-04T08:45:48.811 INFO:teuthology.orchestra.run.smithi071.stdout: iproute-6.7.0-2.el9.x86_64 2024-03-04T08:45:48.811 INFO:teuthology.orchestra.run.smithi071.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-03-04T08:45:48.811 INFO:teuthology.orchestra.run.smithi071.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-04T08:45:48.811 INFO:teuthology.orchestra.run.smithi071.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-04T08:45:48.811 INFO:teuthology.orchestra.run.smithi071.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-04T08:45:48.811 INFO:teuthology.orchestra.run.smithi071.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-04T08:45:48.811 INFO:teuthology.orchestra.run.smithi071.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-04T08:45:48.811 INFO:teuthology.orchestra.run.smithi071.stdout: librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.811 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.811 INFO:teuthology.orchestra.run.smithi071.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-04T08:45:48.811 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-04T08:45:48.812 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-04T08:45:48.812 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-04T08:45:48.812 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-04T08:45:48.812 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-04T08:45:48.812 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-04T08:45:48.812 INFO:teuthology.orchestra.run.smithi071.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-04T08:45:48.812 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-04T08:45:48.812 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-03-04T08:45:48.812 INFO:teuthology.orchestra.run.smithi071.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-04T08:45:48.812 INFO:teuthology.orchestra.run.smithi071.stdout: automake-1.16.2-8.el9.noarch 2024-03-04T08:45:48.812 INFO:teuthology.orchestra.run.smithi071.stdout: bison-3.7.4-5.el9.x86_64 2024-03-04T08:45:48.812 INFO:teuthology.orchestra.run.smithi071.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-04T08:45:48.813 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.813 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.813 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.813 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.813 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:45:48.813 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.813 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.813 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.815 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:45:48.815 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:45:48.816 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:45:48.816 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:45:48.816 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:45:48.816 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.816 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.816 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:45:48.816 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.816 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.816 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.816 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:45:48.816 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:45:48.816 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.816 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.817 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:45:48.817 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-04T08:45:48.817 INFO:teuthology.orchestra.run.smithi071.stdout: flex-2.6.4-9.el9.x86_64 2024-03-04T08:45:48.817 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-04T08:45:48.817 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-04T08:45:48.817 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-04T08:45:48.817 INFO:teuthology.orchestra.run.smithi071.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-04T08:45:48.817 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-04T08:45:48.817 INFO:teuthology.orchestra.run.smithi071.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-04T08:45:48.817 INFO:teuthology.orchestra.run.smithi071.stdout: jq-1.6-16.el9.x86_64 2024-03-04T08:45:48.817 INFO:teuthology.orchestra.run.smithi071.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-04T08:45:48.817 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-04T08:45:48.817 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-04T08:45:48.818 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.818 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.818 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.818 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-04T08:45:48.818 INFO:teuthology.orchestra.run.smithi071.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-04T08:45:48.818 INFO:teuthology.orchestra.run.smithi071.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-04T08:45:48.818 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-04T08:45:48.818 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-04T08:45:48.818 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.818 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-04T08:45:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-04T08:45:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-04T08:45:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-04T08:45:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-04T08:45:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-04T08:45:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: lua-5.4.4-4.el9.x86_64 2024-03-04T08:45:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-04T08:45:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-04T08:45:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-04T08:45:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-04T08:45:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-04T08:45:48.819 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-04T08:45:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-04T08:45:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-13.11-1.el9.x86_64 2024-03-04T08:45:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-04T08:45:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-04T08:45:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-04T08:45:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-04T08:45:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-04T08:45:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-04T08:45:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-04T08:45:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-04T08:45:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-04T08:45:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-04T08:45:48.820 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-04T08:45:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-04T08:45:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-04T08:45:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-04T08:45:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-04T08:45:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-04T08:45:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-04T08:45:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-04T08:45:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-04T08:45:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-04T08:45:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-04T08:45:48.821 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-04T08:45:48.822 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-04T08:45:48.822 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-04T08:45:48.822 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-04T08:45:48.822 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-04T08:45:48.822 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-04T08:45:48.822 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-04T08:45:48.822 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-04T08:45:48.822 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-04T08:45:48.822 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-04T08:45:48.822 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-04T08:45:48.822 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-04T08:45:48.822 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-04T08:45:48.822 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-04T08:45:48.822 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-04T08:45:48.823 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-04T08:45:48.823 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-04T08:45:48.823 INFO:teuthology.orchestra.run.smithi071.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-04T08:45:48.823 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-04T08:45:48.823 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ply-3.11-14.el9.noarch 2024-03-04T08:45:48.823 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-04T08:45:48.823 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-04T08:45:48.823 INFO:teuthology.orchestra.run.smithi071.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-04T08:45:48.823 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-04T08:45:48.823 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-04T08:45:48.823 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-04T08:45:48.823 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-04T08:45:48.823 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-04T08:45:48.824 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-04T08:45:48.824 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.824 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.824 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-04T08:45:48.824 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-04T08:45:48.824 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-04T08:45:48.824 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.824 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-04T08:45:48.824 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-04T08:45:48.824 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-04T08:45:48.824 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-04T08:45:48.824 INFO:teuthology.orchestra.run.smithi071.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-04T08:45:48.824 INFO:teuthology.orchestra.run.smithi071.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-04T08:45:48.825 INFO:teuthology.orchestra.run.smithi071.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-04T08:45:48.825 INFO:teuthology.orchestra.run.smithi071.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-04T08:45:48.825 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-04T08:45:48.825 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-04T08:45:48.825 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-04T08:45:48.825 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-04T08:45:48.825 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-04T08:45:48.825 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-04T08:45:48.825 INFO:teuthology.orchestra.run.smithi071.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-04T08:45:48.825 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.825 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.825 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:45:48.825 INFO:teuthology.orchestra.run.smithi071.stdout: re2-1:20211101-3.el9.x86_64 2024-03-04T08:45:48.825 INFO:teuthology.orchestra.run.smithi071.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-04T08:45:48.826 INFO:teuthology.orchestra.run.smithi071.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-04T08:45:48.826 INFO:teuthology.orchestra.run.smithi071.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-04T08:45:48.826 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-04T08:45:48.826 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-04T08:45:48.826 INFO:teuthology.orchestra.run.smithi071.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-04T08:45:48.826 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:45:48.826 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-03-04T08:45:49.364 DEBUG:teuthology.parallel:result is None 2024-03-04T08:45:53.081 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 135/202 2024-03-04T08:45:53.112 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 135/202 2024-03-04T08:45:53.112 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:53.112 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-04T08:45:53.112 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-04T08:45:53.112 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-04T08:45:53.112 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-03-04T08:45:53.332 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 136/202 2024-03-04T08:45:53.353 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 136/202 2024-03-04T08:45:53.353 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:53.354 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-04T08:45:53.354 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-04T08:45:53.354 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-04T08:45:53.354 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-03-04T08:45:53.791 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 137/202 2024-03-04T08:45:53.813 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 137/202 2024-03-04T08:45:53.813 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:53.813 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-04T08:45:53.813 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-04T08:45:53.813 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-04T08:45:53.813 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-03-04T08:45:53.917 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-03-04T08:45:54.336 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 139/202 2024-03-04T08:45:54.366 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 139/202 2024-03-04T08:45:54.398 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.e 140/202 2024-03-04T08:45:54.459 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7. 141/202 2024-03-04T08:45:54.489 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 142/202 2024-03-04T08:45:54.725 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 142/202 2024-03-04T08:45:54.757 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 143/202 2024-03-04T08:45:55.728 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 143/202 2024-03-04T08:45:55.759 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 144/202 2024-03-04T08:45:55.861 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 144/202 2024-03-04T08:45:56.030 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el 145/202 2024-03-04T08:45:56.070 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 146/202 2024-03-04T08:45:56.095 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 146/202 2024-03-04T08:45:56.096 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:56.096 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-04T08:45:56.096 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-04T08:45:56.096 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-04T08:45:56.096 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-03-04T08:45:56.145 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 147/202 2024-03-04T08:45:56.156 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 147/202 2024-03-04T08:45:56.202 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 148/202 2024-03-04T08:45:57.055 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-03-04T08:45:57.089 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 150/202 2024-03-04T08:45:57.112 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 150/202 2024-03-04T08:45:57.112 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:57.112 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-04T08:45:57.112 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-04T08:45:57.112 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-04T08:45:57.112 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-03-04T08:45:57.157 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 151/202 2024-03-04T08:45:57.178 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 151/202 2024-03-04T08:45:57.178 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:57.179 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-04T08:45:57.179 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-03-04T08:45:57.241 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 152/202 2024-03-04T08:45:57.263 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 152/202 2024-03-04T08:45:57.263 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:57.263 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-04T08:45:57.263 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-04T08:45:57.263 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-04T08:45:57.263 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-03-04T08:45:57.597 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-04T08:45:57.619 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-04T08:45:57.620 INFO:teuthology.orchestra.run.smithi158.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T08:45:57.620 INFO:teuthology.orchestra.run.smithi158.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-04T08:45:57.620 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-04T08:45:57.620 INFO:teuthology.orchestra.run.smithi158.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-04T08:45:57.620 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-03-04T08:46:02.022 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 154/202 2024-03-04T08:46:02.070 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 155/202 2024-03-04T08:46:02.101 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 156/202 2024-03-04T08:46:02.136 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-03-04T08:46:02.346 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 158/202 2024-03-04T08:46:02.460 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-03-04T08:46:02.505 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 160/202 2024-03-04T08:46:02.540 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-03-04T08:46:02.593 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-03-04T08:46:02.666 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-03-04T08:46:02.712 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-03-04T08:46:02.755 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-03-04T08:46:02.785 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_ 166/202 2024-03-04T08:46:02.815 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 167/202 2024-03-04T08:46:02.887 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 168/202 2024-03-04T08:46:02.929 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-03-04T08:46:02.972 INFO:teuthology.orchestra.run.smithi158.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-03-04T08:46:03.577 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-04T08:46:11.631 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-04T08:46:11.655 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-04T08:46:12.501 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 172/202 2024-03-04T08:46:12.576 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 173/202 2024-03-04T08:46:12.617 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-03-04T08:46:12.707 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-03-04T08:46:12.754 INFO:teuthology.orchestra.run.smithi158.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-03-04T08:46:12.779 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-03-04T08:46:12.804 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-03-04T08:46:12.831 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-03-04T08:46:12.831 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-03-04T08:46:12.916 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-03-04T08:46:12.943 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-03-04T08:46:12.982 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-03-04T08:46:12.982 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-03-04T08:46:13.067 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-03-04T08:46:13.110 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-03-04T08:46:13.144 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-03-04T08:46:13.176 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-03-04T08:46:13.204 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-03-04T08:46:13.245 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-03-04T08:46:13.278 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-03-04T08:46:13.317 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-03-04T08:46:13.349 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-03-04T08:46:13.374 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-03-04T08:46:13.400 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-03-04T08:46:13.426 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-03-04T08:46:13.459 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-03-04T08:46:13.465 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-04T08:46:13.466 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-04T08:46:13.645 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-04T08:46:13.645 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-03-04T08:46:13.753 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-03-04T08:46:13.784 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-03-04T08:46:13.812 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-03-04T08:46:13.837 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-03-04T08:46:13.863 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-03-04T08:46:13.935 INFO:teuthology.orchestra.run.smithi158.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-03-04T08:46:28.129 INFO:teuthology.orchestra.run.smithi158.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-03-04T08:46:28.129 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 1/202 2024-03-04T08:46:28.129 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 2/202 2024-03-04T08:46:28.129 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 3/202 2024-03-04T08:46:28.129 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 4/202 2024-03-04T08:46:28.129 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 5/202 2024-03-04T08:46:28.130 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 6/202 2024-03-04T08:46:28.130 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 7/202 2024-03-04T08:46:28.130 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 8/202 2024-03-04T08:46:28.130 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 9/202 2024-03-04T08:46:28.130 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 10/202 2024-03-04T08:46:28.130 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-04T08:46:28.130 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-04T08:46:28.130 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 13/202 2024-03-04T08:46:28.130 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 14/202 2024-03-04T08:46:28.130 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_ 15/202 2024-03-04T08:46:28.148 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-04T08:46:28.148 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 17/202 2024-03-04T08:46:28.148 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_6 18/202 2024-03-04T08:46:28.148 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 19/202 2024-03-04T08:46:28.148 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 20/202 2024-03-04T08:46:28.149 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el 21/202 2024-03-04T08:46:28.149 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9. 22/202 2024-03-04T08:46:28.149 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_6 23/202 2024-03-04T08:46:28.149 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 24/202 2024-03-04T08:46:28.149 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 25/202 2024-03-04T08:46:28.149 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 26/202 2024-03-04T08:46:28.149 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 27/202 2024-03-04T08:46:28.149 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 28/202 2024-03-04T08:46:28.149 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 29/202 2024-03-04T08:46:28.149 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7. 30/202 2024-03-04T08:46:28.149 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 31/202 2024-03-04T08:46:28.149 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 32/202 2024-03-04T08:46:28.149 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 33/202 2024-03-04T08:46:28.150 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el 34/202 2024-03-04T08:46:28.150 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 35/202 2024-03-04T08:46:28.150 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.e 36/202 2024-03-04T08:46:28.150 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 37/202 2024-03-04T08:46:28.150 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 38/202 2024-03-04T08:46:28.150 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 39/202 2024-03-04T08:46:28.150 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-03-04T08:46:28.150 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-03-04T08:46:28.150 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-03-04T08:46:28.150 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-03-04T08:46:28.150 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-03-04T08:46:28.150 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-03-04T08:46:28.150 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-03-04T08:46:28.151 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-03-04T08:46:28.151 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-03-04T08:46:28.151 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-03-04T08:46:28.151 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-03-04T08:46:28.151 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-03-04T08:46:28.151 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-03-04T08:46:28.151 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-03-04T08:46:28.151 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-03-04T08:46:28.151 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-03-04T08:46:28.151 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-03-04T08:46:28.151 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-03-04T08:46:28.151 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-03-04T08:46:28.151 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 59/202 2024-03-04T08:46:28.152 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-03-04T08:46:28.152 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-03-04T08:46:28.152 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-03-04T08:46:28.152 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-03-04T08:46:28.152 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-03-04T08:46:28.152 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-03-04T08:46:28.152 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-03-04T08:46:28.152 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-03-04T08:46:28.152 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-03-04T08:46:28.152 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-03-04T08:46:28.152 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-03-04T08:46:28.152 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-03-04T08:46:28.153 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-03-04T08:46:28.153 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-03-04T08:46:28.153 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-03-04T08:46:28.153 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-03-04T08:46:28.153 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-03-04T08:46:28.153 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-03-04T08:46:28.153 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-03-04T08:46:28.153 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-03-04T08:46:28.153 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-03-04T08:46:28.153 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-03-04T08:46:28.153 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-03-04T08:46:28.153 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-03-04T08:46:28.153 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-03-04T08:46:28.153 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-03-04T08:46:28.154 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-03-04T08:46:28.154 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-03-04T08:46:28.154 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-03-04T08:46:28.154 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-03-04T08:46:28.154 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-03-04T08:46:28.154 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-03-04T08:46:28.154 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-03-04T08:46:28.154 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-03-04T08:46:28.154 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-03-04T08:46:28.154 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-03-04T08:46:28.154 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-03-04T08:46:28.154 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-03-04T08:46:28.155 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-03-04T08:46:28.155 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-03-04T08:46:28.155 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-03-04T08:46:28.155 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-03-04T08:46:28.155 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-03-04T08:46:28.155 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-03-04T08:46:28.155 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-03-04T08:46:28.155 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-03-04T08:46:28.155 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-03-04T08:46:28.155 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-03-04T08:46:28.156 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-03-04T08:46:28.156 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-03-04T08:46:28.156 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-03-04T08:46:28.156 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-03-04T08:46:28.156 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-03-04T08:46:28.156 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-03-04T08:46:28.156 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-03-04T08:46:28.156 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-03-04T08:46:28.156 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-03-04T08:46:28.156 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-03-04T08:46:28.156 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-03-04T08:46:28.157 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-03-04T08:46:28.157 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-03-04T08:46:28.157 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-03-04T08:46:28.157 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-03-04T08:46:28.157 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-03-04T08:46:28.157 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/202 2024-03-04T08:46:28.157 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-03-04T08:46:28.157 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-03-04T08:46:28.157 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-03-04T08:46:28.157 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-03-04T08:46:28.157 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-03-04T08:46:28.157 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-03-04T08:46:28.157 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-03-04T08:46:28.157 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-03-04T08:46:28.158 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-03-04T08:46:28.158 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-03-04T08:46:28.158 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-03-04T08:46:28.158 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-03-04T08:46:28.158 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-03-04T08:46:28.158 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-03-04T08:46:28.158 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-03-04T08:46:28.158 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-03-04T08:46:28.158 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-03-04T08:46:28.158 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-03-04T08:46:28.158 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-03-04T08:46:28.158 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-03-04T08:46:28.158 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-03-04T08:46:28.159 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-03-04T08:46:28.159 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-03-04T08:46:28.159 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-03-04T08:46:28.159 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-03-04T08:46:28.159 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-03-04T08:46:28.159 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 151/202 2024-03-04T08:46:28.159 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-03-04T08:46:28.159 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-04T08:46:28.159 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-03-04T08:46:28.159 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-03-04T08:46:28.159 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-03-04T08:46:28.159 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/202 2024-03-04T08:46:28.159 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-03-04T08:46:28.159 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/202 2024-03-04T08:46:28.160 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-03-04T08:46:28.160 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/202 2024-03-04T08:46:28.160 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-03-04T08:46:28.160 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/202 2024-03-04T08:46:28.160 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-03-04T08:46:28.160 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-03-04T08:46:28.160 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-03-04T08:46:28.160 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-03-04T08:46:28.160 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-03-04T08:46:28.160 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-03-04T08:46:28.160 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-03-04T08:46:28.160 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-03-04T08:46:28.160 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-03-04T08:46:28.161 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-03-04T08:46:28.161 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-03-04T08:46:28.161 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-03-04T08:46:28.161 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-03-04T08:46:28.161 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-03-04T08:46:28.161 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-03-04T08:46:28.161 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-03-04T08:46:28.161 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-03-04T08:46:28.161 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-03-04T08:46:28.161 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-03-04T08:46:28.161 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-03-04T08:46:28.161 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-03-04T08:46:28.161 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-03-04T08:46:28.161 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-03-04T08:46:28.161 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-03-04T08:46:28.162 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-03-04T08:46:28.162 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 189/202 2024-03-04T08:46:28.162 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-03-04T08:46:28.162 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 191/202 2024-03-04T08:46:28.162 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-03-04T08:46:28.162 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-03-04T08:46:28.162 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-03-04T08:46:28.162 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-03-04T08:46:28.162 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-03-04T08:46:28.162 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-03-04T08:46:28.162 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-03-04T08:46:28.162 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-03-04T08:46:28.162 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-03-04T08:46:28.162 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-03-04T08:46:31.020 INFO:teuthology.orchestra.run.smithi158.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-03-04T08:46:31.020 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-03-04T08:46:31.020 INFO:teuthology.orchestra.run.smithi158.stdout:Upgraded: 2024-03-04T08:46:31.020 INFO:teuthology.orchestra.run.smithi158.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-04T08:46:31.020 INFO:teuthology.orchestra.run.smithi158.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-04T08:46:31.020 INFO:teuthology.orchestra.run.smithi158.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-04T08:46:31.021 INFO:teuthology.orchestra.run.smithi158.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-04T08:46:31.021 INFO:teuthology.orchestra.run.smithi158.stdout: acl-2.3.1-4.el9.x86_64 2024-03-04T08:46:31.021 INFO:teuthology.orchestra.run.smithi158.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-04T08:46:31.021 INFO:teuthology.orchestra.run.smithi158.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-04T08:46:31.021 INFO:teuthology.orchestra.run.smithi158.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-04T08:46:31.021 INFO:teuthology.orchestra.run.smithi158.stdout: iproute-6.7.0-2.el9.x86_64 2024-03-04T08:46:31.021 INFO:teuthology.orchestra.run.smithi158.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-03-04T08:46:31.021 INFO:teuthology.orchestra.run.smithi158.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-04T08:46:31.021 INFO:teuthology.orchestra.run.smithi158.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-04T08:46:31.021 INFO:teuthology.orchestra.run.smithi158.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-04T08:46:31.021 INFO:teuthology.orchestra.run.smithi158.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-04T08:46:31.021 INFO:teuthology.orchestra.run.smithi158.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-04T08:46:31.021 INFO:teuthology.orchestra.run.smithi158.stdout: librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.022 INFO:teuthology.orchestra.run.smithi158.stdout: librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.022 INFO:teuthology.orchestra.run.smithi158.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-04T08:46:31.022 INFO:teuthology.orchestra.run.smithi158.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-04T08:46:31.022 INFO:teuthology.orchestra.run.smithi158.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-04T08:46:31.022 INFO:teuthology.orchestra.run.smithi158.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-04T08:46:31.022 INFO:teuthology.orchestra.run.smithi158.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-04T08:46:31.022 INFO:teuthology.orchestra.run.smithi158.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-04T08:46:31.022 INFO:teuthology.orchestra.run.smithi158.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-04T08:46:31.022 INFO:teuthology.orchestra.run.smithi158.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-04T08:46:31.022 INFO:teuthology.orchestra.run.smithi158.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-04T08:46:31.022 INFO:teuthology.orchestra.run.smithi158.stdout:Installed: 2024-03-04T08:46:31.022 INFO:teuthology.orchestra.run.smithi158.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-04T08:46:31.022 INFO:teuthology.orchestra.run.smithi158.stdout: automake-1.16.2-8.el9.noarch 2024-03-04T08:46:31.023 INFO:teuthology.orchestra.run.smithi158.stdout: bison-3.7.4-5.el9.x86_64 2024-03-04T08:46:31.023 INFO:teuthology.orchestra.run.smithi158.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-04T08:46:31.023 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.023 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.023 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.023 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.023 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:46:31.023 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-immutable-object-cache-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.023 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.023 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.023 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:46:31.023 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:46:31.023 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:46:31.024 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:46:31.024 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:46:31.024 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.024 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.024 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:46:31.024 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.024 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.024 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.024 INFO:teuthology.orchestra.run.smithi158.stdout: ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:46:31.024 INFO:teuthology.orchestra.run.smithi158.stdout: cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:46:31.024 INFO:teuthology.orchestra.run.smithi158.stdout: cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.024 INFO:teuthology.orchestra.run.smithi158.stdout: cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.024 INFO:teuthology.orchestra.run.smithi158.stdout: cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T08:46:31.025 INFO:teuthology.orchestra.run.smithi158.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-04T08:46:31.025 INFO:teuthology.orchestra.run.smithi158.stdout: flex-2.6.4-9.el9.x86_64 2024-03-04T08:46:31.025 INFO:teuthology.orchestra.run.smithi158.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-04T08:46:31.025 INFO:teuthology.orchestra.run.smithi158.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-04T08:46:31.025 INFO:teuthology.orchestra.run.smithi158.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-04T08:46:31.025 INFO:teuthology.orchestra.run.smithi158.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-04T08:46:31.025 INFO:teuthology.orchestra.run.smithi158.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-04T08:46:31.025 INFO:teuthology.orchestra.run.smithi158.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-04T08:46:31.025 INFO:teuthology.orchestra.run.smithi158.stdout: jq-1.6-16.el9.x86_64 2024-03-04T08:46:31.025 INFO:teuthology.orchestra.run.smithi158.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-04T08:46:31.025 INFO:teuthology.orchestra.run.smithi158.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-04T08:46:31.025 INFO:teuthology.orchestra.run.smithi158.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-04T08:46:31.025 INFO:teuthology.orchestra.run.smithi158.stdout: libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.025 INFO:teuthology.orchestra.run.smithi158.stdout: libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.025 INFO:teuthology.orchestra.run.smithi158.stdout: libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.026 INFO:teuthology.orchestra.run.smithi158.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-04T08:46:31.026 INFO:teuthology.orchestra.run.smithi158.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-04T08:46:31.026 INFO:teuthology.orchestra.run.smithi158.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-04T08:46:31.026 INFO:teuthology.orchestra.run.smithi158.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-04T08:46:31.026 INFO:teuthology.orchestra.run.smithi158.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-04T08:46:31.026 INFO:teuthology.orchestra.run.smithi158.stdout: librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.026 INFO:teuthology.orchestra.run.smithi158.stdout: libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.026 INFO:teuthology.orchestra.run.smithi158.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-04T08:46:31.026 INFO:teuthology.orchestra.run.smithi158.stdout: librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.026 INFO:teuthology.orchestra.run.smithi158.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-04T08:46:31.026 INFO:teuthology.orchestra.run.smithi158.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-04T08:46:31.026 INFO:teuthology.orchestra.run.smithi158.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-04T08:46:31.026 INFO:teuthology.orchestra.run.smithi158.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-04T08:46:31.026 INFO:teuthology.orchestra.run.smithi158.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-04T08:46:31.027 INFO:teuthology.orchestra.run.smithi158.stdout: lua-5.4.4-4.el9.x86_64 2024-03-04T08:46:31.027 INFO:teuthology.orchestra.run.smithi158.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-04T08:46:31.027 INFO:teuthology.orchestra.run.smithi158.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-04T08:46:31.027 INFO:teuthology.orchestra.run.smithi158.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-04T08:46:31.027 INFO:teuthology.orchestra.run.smithi158.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-04T08:46:31.027 INFO:teuthology.orchestra.run.smithi158.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-04T08:46:31.027 INFO:teuthology.orchestra.run.smithi158.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-04T08:46:31.027 INFO:teuthology.orchestra.run.smithi158.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-04T08:46:31.027 INFO:teuthology.orchestra.run.smithi158.stdout: postgresql-13.11-1.el9.x86_64 2024-03-04T08:46:31.027 INFO:teuthology.orchestra.run.smithi158.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-04T08:46:31.027 INFO:teuthology.orchestra.run.smithi158.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-04T08:46:31.027 INFO:teuthology.orchestra.run.smithi158.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-04T08:46:31.027 INFO:teuthology.orchestra.run.smithi158.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-04T08:46:31.028 INFO:teuthology.orchestra.run.smithi158.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-04T08:46:31.028 INFO:teuthology.orchestra.run.smithi158.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-04T08:46:31.028 INFO:teuthology.orchestra.run.smithi158.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-04T08:46:31.028 INFO:teuthology.orchestra.run.smithi158.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-04T08:46:31.028 INFO:teuthology.orchestra.run.smithi158.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-04T08:46:31.028 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-04T08:46:31.028 INFO:teuthology.orchestra.run.smithi158.stdout: python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.028 INFO:teuthology.orchestra.run.smithi158.stdout: python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.028 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.028 INFO:teuthology.orchestra.run.smithi158.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-04T08:46:31.028 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-04T08:46:31.028 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-04T08:46:31.028 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-04T08:46:31.028 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-04T08:46:31.029 INFO:teuthology.orchestra.run.smithi158.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-04T08:46:31.029 INFO:teuthology.orchestra.run.smithi158.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-04T08:46:31.029 INFO:teuthology.orchestra.run.smithi158.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-04T08:46:31.029 INFO:teuthology.orchestra.run.smithi158.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-04T08:46:31.029 INFO:teuthology.orchestra.run.smithi158.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-04T08:46:31.029 INFO:teuthology.orchestra.run.smithi158.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-04T08:46:31.029 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-04T08:46:31.029 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-04T08:46:31.029 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-04T08:46:31.029 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-04T08:46:31.029 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-04T08:46:31.029 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-04T08:46:31.029 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-04T08:46:31.029 INFO:teuthology.orchestra.run.smithi158.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-04T08:46:31.030 INFO:teuthology.orchestra.run.smithi158.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-04T08:46:31.030 INFO:teuthology.orchestra.run.smithi158.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-04T08:46:31.030 INFO:teuthology.orchestra.run.smithi158.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-04T08:46:31.030 INFO:teuthology.orchestra.run.smithi158.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-04T08:46:31.030 INFO:teuthology.orchestra.run.smithi158.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-04T08:46:31.030 INFO:teuthology.orchestra.run.smithi158.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-04T08:46:31.030 INFO:teuthology.orchestra.run.smithi158.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-04T08:46:31.030 INFO:teuthology.orchestra.run.smithi158.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-04T08:46:31.030 INFO:teuthology.orchestra.run.smithi158.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-04T08:46:31.030 INFO:teuthology.orchestra.run.smithi158.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-04T08:46:31.030 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-04T08:46:31.030 INFO:teuthology.orchestra.run.smithi158.stdout: python3-ply-3.11-14.el9.noarch 2024-03-04T08:46:31.030 INFO:teuthology.orchestra.run.smithi158.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-04T08:46:31.030 INFO:teuthology.orchestra.run.smithi158.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-04T08:46:31.031 INFO:teuthology.orchestra.run.smithi158.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-04T08:46:31.031 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-04T08:46:31.031 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-04T08:46:31.031 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-04T08:46:31.031 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-04T08:46:31.031 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-04T08:46:31.031 INFO:teuthology.orchestra.run.smithi158.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-04T08:46:31.031 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.031 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.031 INFO:teuthology.orchestra.run.smithi158.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-04T08:46:31.031 INFO:teuthology.orchestra.run.smithi158.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-04T08:46:31.031 INFO:teuthology.orchestra.run.smithi158.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-04T08:46:31.031 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.031 INFO:teuthology.orchestra.run.smithi158.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-04T08:46:31.031 INFO:teuthology.orchestra.run.smithi158.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-04T08:46:31.032 INFO:teuthology.orchestra.run.smithi158.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-04T08:46:31.032 INFO:teuthology.orchestra.run.smithi158.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-04T08:46:31.032 INFO:teuthology.orchestra.run.smithi158.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-04T08:46:31.032 INFO:teuthology.orchestra.run.smithi158.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-04T08:46:31.032 INFO:teuthology.orchestra.run.smithi158.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-04T08:46:31.032 INFO:teuthology.orchestra.run.smithi158.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-04T08:46:31.032 INFO:teuthology.orchestra.run.smithi158.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-04T08:46:31.032 INFO:teuthology.orchestra.run.smithi158.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-04T08:46:31.032 INFO:teuthology.orchestra.run.smithi158.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-04T08:46:31.032 INFO:teuthology.orchestra.run.smithi158.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-04T08:46:31.032 INFO:teuthology.orchestra.run.smithi158.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-04T08:46:31.032 INFO:teuthology.orchestra.run.smithi158.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-04T08:46:31.032 INFO:teuthology.orchestra.run.smithi158.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-04T08:46:31.033 INFO:teuthology.orchestra.run.smithi158.stdout: rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.033 INFO:teuthology.orchestra.run.smithi158.stdout: rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.033 INFO:teuthology.orchestra.run.smithi158.stdout: rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T08:46:31.033 INFO:teuthology.orchestra.run.smithi158.stdout: re2-1:20211101-3.el9.x86_64 2024-03-04T08:46:31.033 INFO:teuthology.orchestra.run.smithi158.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-04T08:46:31.033 INFO:teuthology.orchestra.run.smithi158.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-04T08:46:31.033 INFO:teuthology.orchestra.run.smithi158.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-04T08:46:31.033 INFO:teuthology.orchestra.run.smithi158.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-04T08:46:31.033 INFO:teuthology.orchestra.run.smithi158.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-04T08:46:31.034 INFO:teuthology.orchestra.run.smithi158.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-04T08:46:31.034 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-03-04T08:46:31.034 INFO:teuthology.orchestra.run.smithi158.stdout:Complete! 2024-03-04T08:46:31.505 DEBUG:teuthology.parallel:result is None 2024-03-04T08:46:31.505 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T08:46:31.645 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-04T08:46:31.671 INFO:teuthology.orchestra.run.smithi071.stdout:19.0.0-1739.g1bc250c7.el9 2024-03-04T08:46:31.671 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1739.g1bc250c7.el9 2024-03-04T08:46:31.671 INFO:teuthology.task.install:The correct ceph version 19.0.0-1739.g1bc250c7 is installed. 2024-03-04T08:46:31.673 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T08:46:31.813 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-04T08:46:31.839 INFO:teuthology.orchestra.run.smithi150.stdout:19.0.0-1739.g1bc250c7.el9 2024-03-04T08:46:31.839 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1739.g1bc250c7.el9 2024-03-04T08:46:31.839 INFO:teuthology.task.install:The correct ceph version 19.0.0-1739.g1bc250c7 is installed. 2024-03-04T08:46:31.841 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T08:46:31.983 DEBUG:teuthology.orchestra.run.smithi158:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-04T08:46:32.008 INFO:teuthology.orchestra.run.smithi158.stdout:19.0.0-1739.g1bc250c7.el9 2024-03-04T08:46:32.008 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1739.g1bc250c7.el9 2024-03-04T08:46:32.008 INFO:teuthology.task.install:The correct ceph version 19.0.0-1739.g1bc250c7 is installed. 2024-03-04T08:46:32.010 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-04T08:46:32.010 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-04T08:46:32.010 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-04T08:46:32.044 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-03-04T08:46:32.045 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-04T08:46:32.080 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2024-03-04T08:46:32.080 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-04T08:46:32.114 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-04T08:46:32.114 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-04T08:46:32.114 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/daemon-helper 2024-03-04T08:46:32.145 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-04T08:46:32.214 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-03-04T08:46:32.214 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2024-03-04T08:46:32.246 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-04T08:46:32.316 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2024-03-04T08:46:32.316 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/usr/bin/daemon-helper 2024-03-04T08:46:32.348 DEBUG:teuthology.orchestra.run.smithi158:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-04T08:46:32.418 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-04T08:46:32.418 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-04T08:46:32.418 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-04T08:46:32.447 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-04T08:46:32.515 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-03-04T08:46:32.516 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-04T08:46:32.544 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-04T08:46:32.611 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2024-03-04T08:46:32.611 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-04T08:46:32.642 DEBUG:teuthology.orchestra.run.smithi158:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-04T08:46:32.712 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-04T08:46:32.712 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-04T08:46:32.712 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/stdin-killer 2024-03-04T08:46:32.741 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-04T08:46:32.810 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-03-04T08:46:32.810 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2024-03-04T08:46:32.837 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-04T08:46:32.906 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2024-03-04T08:46:32.906 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/usr/bin/stdin-killer 2024-03-04T08:46:32.936 DEBUG:teuthology.orchestra.run.smithi158:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-04T08:46:33.006 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-04T08:46:33.106 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-04T08:46:33.106 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T08:46:33.107 INFO:tasks.cephadm:Cluster fsid is b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T08:46:33.107 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-04T08:46:33.107 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.71', 'mon.b': '172.21.15.150', 'mon.c': '172.21.15.158'} 2024-03-04T08:46:33.107 INFO:tasks.cephadm:First mon is mon.a on smithi071 2024-03-04T08:46:33.108 INFO:tasks.cephadm:First mgr is x 2024-03-04T08:46:33.108 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-04T08:46:33.108 DEBUG:teuthology.orchestra.run.smithi071:> sudo hostname $(hostname -s) 2024-03-04T08:46:33.137 DEBUG:teuthology.orchestra.run.smithi150:> sudo hostname $(hostname -s) 2024-03-04T08:46:33.164 DEBUG:teuthology.orchestra.run.smithi158:> sudo hostname $(hostname -s) 2024-03-04T08:46:33.195 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-04T08:46:33.196 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T08:46:33.502 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77279/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1739-g1bc250c7', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1739.g1bc250c7'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-04 00:06:26.635980', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-qdb', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-03-04T08:46:33.651 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-lusov-qdb, sha1 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T08:46:33.653 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/x86_64/flavors/default/cephadm 2024-03-04T08:46:33.653 DEBUG:teuthology.orchestra.run.smithi071:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-04T08:46:34.655 INFO:teuthology.orchestra.run.smithi071.stdout:-rw-r--r--. 1 ubuntu ubuntu 766895 Mar 4 08:46 /home/ubuntu/cephtest/cephadm 2024-03-04T08:46:34.656 DEBUG:teuthology.orchestra.run.smithi150:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-04T08:46:36.045 INFO:teuthology.orchestra.run.smithi150.stdout:-rw-r--r--. 1 ubuntu ubuntu 766895 Mar 4 08:46 /home/ubuntu/cephtest/cephadm 2024-03-04T08:46:36.045 DEBUG:teuthology.orchestra.run.smithi158:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-04T08:46:37.357 INFO:teuthology.orchestra.run.smithi158.stdout:-rw-r--r--. 1 ubuntu ubuntu 766895 Mar 4 08:46 /home/ubuntu/cephtest/cephadm 2024-03-04T08:46:37.357 DEBUG:teuthology.orchestra.run.smithi071:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-04T08:46:37.376 DEBUG:teuthology.orchestra.run.smithi150:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-04T08:46:37.395 DEBUG:teuthology.orchestra.run.smithi158:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-04T08:46:37.431 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd on all hosts... 2024-03-04T08:46:37.431 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd pull 2024-03-04T08:46:37.433 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd pull 2024-03-04T08:46:37.438 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd pull 2024-03-04T08:46:38.255 INFO:teuthology.orchestra.run.smithi150.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd... 2024-03-04T08:46:38.282 INFO:teuthology.orchestra.run.smithi071.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd... 2024-03-04T08:46:38.286 INFO:teuthology.orchestra.run.smithi158.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd... 2024-03-04T08:47:05.742 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-03-04T08:47:05.742 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_version": "ceph version 19.0.0-1739-g1bc250c7 (1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd) squid (dev)", 2024-03-04T08:47:05.742 INFO:teuthology.orchestra.run.smithi150.stdout: "image_id": "f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba", 2024-03-04T08:47:05.742 INFO:teuthology.orchestra.run.smithi150.stdout: "repo_digests": [ 2024-03-04T08:47:05.742 INFO:teuthology.orchestra.run.smithi150.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5" 2024-03-04T08:47:05.742 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-03-04T08:47:05.742 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-03-04T08:47:08.585 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-03-04T08:47:08.585 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 19.0.0-1739-g1bc250c7 (1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd) squid (dev)", 2024-03-04T08:47:08.585 INFO:teuthology.orchestra.run.smithi071.stdout: "image_id": "f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba", 2024-03-04T08:47:08.585 INFO:teuthology.orchestra.run.smithi071.stdout: "repo_digests": [ 2024-03-04T08:47:08.585 INFO:teuthology.orchestra.run.smithi071.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5" 2024-03-04T08:47:08.586 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-03-04T08:47:08.586 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-03-04T08:47:13.825 INFO:teuthology.orchestra.run.smithi158.stdout:{ 2024-03-04T08:47:13.825 INFO:teuthology.orchestra.run.smithi158.stdout: "ceph_version": "ceph version 19.0.0-1739-g1bc250c7 (1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd) squid (dev)", 2024-03-04T08:47:13.825 INFO:teuthology.orchestra.run.smithi158.stdout: "image_id": "f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba", 2024-03-04T08:47:13.826 INFO:teuthology.orchestra.run.smithi158.stdout: "repo_digests": [ 2024-03-04T08:47:13.826 INFO:teuthology.orchestra.run.smithi158.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5" 2024-03-04T08:47:13.826 INFO:teuthology.orchestra.run.smithi158.stdout: ] 2024-03-04T08:47:13.826 INFO:teuthology.orchestra.run.smithi158.stdout:} 2024-03-04T08:47:13.850 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /etc/ceph 2024-03-04T08:47:13.884 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph 2024-03-04T08:47:13.919 DEBUG:teuthology.orchestra.run.smithi158:> sudo mkdir -p /etc/ceph 2024-03-04T08:47:13.949 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 777 /etc/ceph 2024-03-04T08:47:13.978 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /etc/ceph 2024-03-04T08:47:14.008 DEBUG:teuthology.orchestra.run.smithi158:> sudo chmod 777 /etc/ceph 2024-03-04T08:47:14.034 INFO:tasks.cephadm:Writing seed config... 2024-03-04T08:47:14.035 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-04T08:47:14.035 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-04T08:47:14.035 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-04T08:47:14.035 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-04T08:47:14.035 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-04T08:47:14.036 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-04T08:47:14.036 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-04T08:47:14.036 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-04T08:47:14.036 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-04T08:47:14.036 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-04T08:47:14.036 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-04T08:47:14.036 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-04T08:47:14.036 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-04T08:47:14.036 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-04T08:47:14.036 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-04T08:47:14.036 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-04T08:47:14.036 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-04T08:47:14.036 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-04T08:47:14.036 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-04T08:47:14.036 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-04T08:47:14.037 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-04T08:47:14.037 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-04T08:47:14.037 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-04T08:47:14.037 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-04T08:47:14.037 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-04T08:47:14.037 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-04T08:47:14.037 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-04T08:47:14.037 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-04T08:47:14.037 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-04T08:47:14.037 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-04T08:47:14.037 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-04T08:47:14.037 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-04T08:47:14.037 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-04T08:47:14.037 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-04T08:47:14.037 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-04T08:47:14.037 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-04T08:47:14.038 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-04T08:47:14.038 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-04T08:47:14.038 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-04T08:47:14.038 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-04T08:47:14.038 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-04T08:47:14.038 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-04T08:47:14.038 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-03-04T08:47:14.039 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-04T08:47:14.039 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-04T08:47:14.055 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b3c85c40-da03-11ee-95c6-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-04T08:47:14.055 DEBUG:teuthology.orchestra.run.smithi071:mon.a> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mon.a.service 2024-03-04T08:47:14.097 DEBUG:teuthology.orchestra.run.smithi071:mgr.x> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mgr.x.service 2024-03-04T08:47:14.140 INFO:tasks.cephadm:Bootstrapping... 2024-03-04T08:47:14.140 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd -v bootstrap --fsid b3c85c40-da03-11ee-95c6-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.71 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-04T08:47:14.333 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-03-04T08:47:14.333 INFO:teuthology.orchestra.run.smithi071.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', '-v', 'bootstrap', '--fsid', 'b3c85c40-da03-11ee-95c6-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.71', '--skip-admin-label'] 2024-03-04T08:47:14.359 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-04T08:47:14.360 INFO:teuthology.orchestra.run.smithi071.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-04T08:47:14.360 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying podman|docker is present... 2024-03-04T08:47:14.383 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-04T08:47:14.383 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying lvm2 is present... 2024-03-04T08:47:14.383 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying time synchronization is in place... 2024-03-04T08:47:14.391 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-04T08:47:14.391 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-04T08:47:14.397 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-04T08:47:14.397 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-03-04T08:47:14.403 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout enabled 2024-03-04T08:47:14.409 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout active 2024-03-04T08:47:14.409 INFO:teuthology.orchestra.run.smithi071.stdout:Unit chronyd.service is enabled and running 2024-03-04T08:47:14.409 INFO:teuthology.orchestra.run.smithi071.stdout:Repeating the final host check... 2024-03-04T08:47:14.432 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-04T08:47:14.432 INFO:teuthology.orchestra.run.smithi071.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-04T08:47:14.432 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl is present 2024-03-04T08:47:14.432 INFO:teuthology.orchestra.run.smithi071.stdout:lvcreate is present 2024-03-04T08:47:14.438 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-04T08:47:14.438 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-04T08:47:14.443 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-04T08:47:14.444 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-03-04T08:47:14.450 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout enabled 2024-03-04T08:47:14.456 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout active 2024-03-04T08:47:14.456 INFO:teuthology.orchestra.run.smithi071.stdout:Unit chronyd.service is enabled and running 2024-03-04T08:47:14.456 INFO:teuthology.orchestra.run.smithi071.stdout:Host looks OK 2024-03-04T08:47:14.456 INFO:teuthology.orchestra.run.smithi071.stdout:Cluster fsid: b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T08:47:14.456 INFO:teuthology.orchestra.run.smithi071.stdout:Acquiring lock 140511472443344 on /run/cephadm/b3c85c40-da03-11ee-95c6-87774f69a715.lock 2024-03-04T08:47:14.456 INFO:teuthology.orchestra.run.smithi071.stdout:Lock 140511472443344 acquired on /run/cephadm/b3c85c40-da03-11ee-95c6-87774f69a715.lock 2024-03-04T08:47:14.457 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying IP 172.21.15.71 port 3300 ... 2024-03-04T08:47:14.457 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying IP 172.21.15.71 port 6789 ... 2024-03-04T08:47:14.457 INFO:teuthology.orchestra.run.smithi071.stdout:Base mon IP(s) is [172.21.15.71:3300, 172.21.15.71:6789], mon addrv is [v2:172.21.15.71:3300,v1:172.21.15.71:6789] 2024-03-04T08:47:14.460 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.71 metric 100 2024-03-04T08:47:14.460 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.71 metric 100 2024-03-04T08:47:14.462 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-04T08:47:14.462 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-04T08:47:14.463 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1797sec hoplimit 64 pref medium 2024-03-04T08:47:14.465 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-04T08:47:14.465 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-04T08:47:14.465 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-04T08:47:14.465 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-04T08:47:14.465 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d765/64 scope link 2024-03-04T08:47:14.466 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-04T08:47:14.466 INFO:teuthology.orchestra.run.smithi071.stdout:Mon IP `172.21.15.71` is in CIDR network `172.21.0.0/20` 2024-03-04T08:47:14.466 INFO:teuthology.orchestra.run.smithi071.stdout:Mon IP `172.21.15.71` is in CIDR network `172.21.0.0/20` 2024-03-04T08:47:14.466 INFO:teuthology.orchestra.run.smithi071.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-04T08:47:14.467 INFO:teuthology.orchestra.run.smithi071.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-04T08:47:14.467 INFO:teuthology.orchestra.run.smithi071.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd... 2024-03-04T08:47:17.789 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stdout f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba 2024-03-04T08:47:17.789 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd... 2024-03-04T08:47:17.790 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Getting image source signatures 2024-03-04T08:47:17.790 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Copying blob sha256:5d74734bfa4d1f8e0a72f8c221d93ca0d3a68fb40764a44763479cacf5cffcb4 2024-03-04T08:47:17.790 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Copying blob sha256:7feca07754707458c3945cf0062cf4dabc512f6d90fe1a9a1370b362b6011124 2024-03-04T08:47:17.790 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Copying config sha256:f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba 2024-03-04T08:47:17.790 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-04T08:47:19.163 INFO:teuthology.orchestra.run.smithi071.stdout:ceph: stdout ceph version 19.0.0-1739-g1bc250c7 (1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd) squid (dev) 2024-03-04T08:47:19.163 INFO:teuthology.orchestra.run.smithi071.stdout:Ceph version: ceph version 19.0.0-1739-g1bc250c7 (1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd) squid (dev) 2024-03-04T08:47:19.163 INFO:teuthology.orchestra.run.smithi071.stdout:Extracting ceph user uid/gid from container image... 2024-03-04T08:47:20.069 INFO:teuthology.orchestra.run.smithi071.stdout:stat: stdout 167 167 2024-03-04T08:47:20.069 INFO:teuthology.orchestra.run.smithi071.stdout:Creating initial keys... 2024-03-04T08:47:20.801 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph-authtool: stdout AQCYiuVlSztXFRAA84YUAYGoRBIHSsmIfeomhw== 2024-03-04T08:47:21.730 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph-authtool: stdout AQCZiuVl4A1BCBAALpyhwsaWabBD6lJu/dQW3A== 2024-03-04T08:47:22.526 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph-authtool: stdout AQCZiuVldkctOxAAf2FYWqZPv/pUYakQwRRiFQ== 2024-03-04T08:47:22.527 INFO:teuthology.orchestra.run.smithi071.stdout:Creating initial monmap... 2024-03-04T08:47:25.594 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-04T08:47:25.594 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-04T08:47:25.594 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T08:47:25.595 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-04T08:47:25.595 INFO:teuthology.orchestra.run.smithi071.stdout:monmaptool for a [v2:172.21.15.71:3300,v1:172.21.15.71:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-04T08:47:25.595 INFO:teuthology.orchestra.run.smithi071.stdout:setting min_mon_release = quincy 2024-03-04T08:47:25.595 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: set fsid to b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T08:47:25.595 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-04T08:47:25.595 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:47:25.595 INFO:teuthology.orchestra.run.smithi071.stdout:Creating mon... 2024-03-04T08:47:27.060 INFO:teuthology.orchestra.run.smithi071.stdout:create mon.a on 2024-03-04T08:47:27.263 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-04T08:47:27.409 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-04T08:47:27.592 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b3c85c40-da03-11ee-95c6-87774f69a715.target → /etc/systemd/system/ceph-b3c85c40-da03-11ee-95c6-87774f69a715.target. 2024-03-04T08:47:27.592 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b3c85c40-da03-11ee-95c6-87774f69a715.target → /etc/systemd/system/ceph-b3c85c40-da03-11ee-95c6-87774f69a715.target. 2024-03-04T08:47:27.838 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mon.a 2024-03-04T08:47:27.838 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mon.a.service: Unit ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mon.a.service not loaded. 2024-03-04T08:47:28.005 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b3c85c40-da03-11ee-95c6-87774f69a715.target.wants/ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mon.a.service → /etc/systemd/system/ceph-b3c85c40-da03-11ee-95c6-87774f69a715@.service. 2024-03-04T08:47:28.559 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-04T08:47:28.559 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout disabled 2024-03-04T08:47:28.566 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-04T08:47:28.567 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-03-04T08:47:28.567 INFO:teuthology.orchestra.run.smithi071.stdout:firewalld.service is not enabled 2024-03-04T08:47:28.567 INFO:teuthology.orchestra.run.smithi071.stdout:Not possible to enable service . firewalld.service is not available 2024-03-04T08:47:28.567 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mon to start... 2024-03-04T08:47:28.567 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mon... 2024-03-04T08:47:29.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:29 smithi071 ceph-mon[32416]: from='client.? 172.21.15.71:0/3885710148' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-04T08:47:29.736 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout cluster: 2024-03-04T08:47:29.736 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout id: b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T08:47:29.736 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-04T08:47:29.736 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-03-04T08:47:29.736 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout services: 2024-03-04T08:47:29.736 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.571861s) 2024-03-04T08:47:29.736 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-04T08:47:29.736 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-04T08:47:29.737 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-03-04T08:47:29.737 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout data: 2024-03-04T08:47:29.737 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-04T08:47:29.737 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-04T08:47:29.737 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-04T08:47:29.737 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout pgs: 2024-03-04T08:47:29.737 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-03-04T08:47:29.737 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.033+0000 7ff46489f700 1 Processor -- start 2024-03-04T08:47:29.737 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.034+0000 7ff46489f700 1 -- start start 2024-03-04T08:47:29.737 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.034+0000 7ff46489f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff460108460 0x7ff46010a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:29.737 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.034+0000 7ff46489f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46007ae80 con 0x7ff460108460 2024-03-04T08:47:29.737 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.034+0000 7ff45e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff460108460 0x7ff46010a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:29.737 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.034+0000 7ff45e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff460108460 0x7ff46010a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39368/0 (socket says 172.21.15.71:39368) 2024-03-04T08:47:29.737 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.034+0000 7ff45e59c700 1 -- 172.21.15.71:0/2157635221 learned_addr learned my addr 172.21.15.71:0/2157635221 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:29.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.035+0000 7ff45e59c700 1 -- 172.21.15.71:0/2157635221 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46010ad90 con 0x7ff460108460 2024-03-04T08:47:29.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.035+0000 7ff45e59c700 1 --2- 172.21.15.71:0/2157635221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff460108460 0x7ff46010a850 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff4540098a0 tx=0x7ff454009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3679d02615abe9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:29.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.035+0000 7ff45d59a700 1 -- 172.21.15.71:0/2157635221 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff454004160 con 0x7ff460108460 2024-03-04T08:47:29.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.035+0000 7ff45d59a700 1 -- 172.21.15.71:0/2157635221 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff454004300 con 0x7ff460108460 2024-03-04T08:47:29.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.035+0000 7ff45d59a700 1 -- 172.21.15.71:0/2157635221 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4540036a0 con 0x7ff460108460 2024-03-04T08:47:29.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.035+0000 7ff46489f700 1 -- 172.21.15.71:0/2157635221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff460108460 msgr2=0x7ff46010a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:29.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.035+0000 7ff46489f700 1 --2- 172.21.15.71:0/2157635221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff460108460 0x7ff46010a850 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff4540098a0 tx=0x7ff454009bb0 comp rx=0 tx=0).stop 2024-03-04T08:47:29.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.036+0000 7ff46489f700 1 -- 172.21.15.71:0/2157635221 shutdown_connections 2024-03-04T08:47:29.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.036+0000 7ff46489f700 1 --2- 172.21.15.71:0/2157635221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff460108460 0x7ff46010a850 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:29.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.036+0000 7ff46489f700 1 -- 172.21.15.71:0/2157635221 >> 172.21.15.71:0/2157635221 conn(0x7ff460101f50 msgr2=0x7ff460104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:29.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.036+0000 7ff46489f700 1 -- 172.21.15.71:0/2157635221 shutdown_connections 2024-03-04T08:47:29.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.036+0000 7ff46489f700 1 -- 172.21.15.71:0/2157635221 wait complete. 2024-03-04T08:47:29.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.037+0000 7ff46489f700 1 Processor -- start 2024-03-04T08:47:29.738 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.037+0000 7ff46489f700 1 -- start start 2024-03-04T08:47:29.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.037+0000 7ff46489f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff460108460 0x7ff46019b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:29.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.037+0000 7ff46489f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46010b200 con 0x7ff460108460 2024-03-04T08:47:29.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.037+0000 7ff45e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff460108460 0x7ff46019b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:29.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.037+0000 7ff45e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff460108460 0x7ff46019b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39370/0 (socket says 172.21.15.71:39370) 2024-03-04T08:47:29.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.037+0000 7ff45e59c700 1 -- 172.21.15.71:0/3885710148 learned_addr learned my addr 172.21.15.71:0/3885710148 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:29.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.037+0000 7ff45e59c700 1 -- 172.21.15.71:0/3885710148 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46019b580 con 0x7ff460108460 2024-03-04T08:47:29.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.037+0000 7ff45e59c700 1 --2- 172.21.15.71:0/3885710148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff460108460 0x7ff46019b040 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff454000c00 tx=0x7ff454004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:29.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.037+0000 7ff44f7fe700 1 -- 172.21.15.71:0/3885710148 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff454003db0 con 0x7ff460108460 2024-03-04T08:47:29.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.038+0000 7ff44f7fe700 1 -- 172.21.15.71:0/3885710148 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff454011630 con 0x7ff460108460 2024-03-04T08:47:29.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.038+0000 7ff44f7fe700 1 -- 172.21.15.71:0/3885710148 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff454011950 con 0x7ff460108460 2024-03-04T08:47:29.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.038+0000 7ff46489f700 1 -- 172.21.15.71:0/3885710148 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff46019b870 con 0x7ff460108460 2024-03-04T08:47:29.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.038+0000 7ff46489f700 1 -- 172.21.15.71:0/3885710148 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff46019e500 con 0x7ff460108460 2024-03-04T08:47:29.739 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.038+0000 7ff44f7fe700 1 -- 172.21.15.71:0/3885710148 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff45401a460 con 0x7ff460108460 2024-03-04T08:47:29.740 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.038+0000 7ff44f7fe700 1 -- 172.21.15.71:0/3885710148 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff45401bb40 con 0x7ff460108460 2024-03-04T08:47:29.740 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.039+0000 7ff46489f700 1 -- 172.21.15.71:0/3885710148 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff460069150 con 0x7ff460108460 2024-03-04T08:47:29.740 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.041+0000 7ff44f7fe700 1 -- 172.21.15.71:0/3885710148 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ff45401bde0 con 0x7ff460108460 2024-03-04T08:47:29.740 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.115+0000 7ff46489f700 1 -- 172.21.15.71:0/3885710148 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7ff46002db70 con 0x7ff460108460 2024-03-04T08:47:29.740 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.116+0000 7ff44f7fe700 1 -- 172.21.15.71:0/3885710148 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7ff454004260 con 0x7ff460108460 2024-03-04T08:47:29.740 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.117+0000 7ff46489f700 1 -- 172.21.15.71:0/3885710148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff460108460 msgr2=0x7ff46019b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:29.740 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.117+0000 7ff46489f700 1 --2- 172.21.15.71:0/3885710148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff460108460 0x7ff46019b040 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff454000c00 tx=0x7ff454004750 comp rx=0 tx=0).stop 2024-03-04T08:47:29.740 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.117+0000 7ff46489f700 1 -- 172.21.15.71:0/3885710148 shutdown_connections 2024-03-04T08:47:29.740 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.117+0000 7ff46489f700 1 --2- 172.21.15.71:0/3885710148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff460108460 0x7ff46019b040 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:29.740 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.117+0000 7ff46489f700 1 -- 172.21.15.71:0/3885710148 >> 172.21.15.71:0/3885710148 conn(0x7ff460101f50 msgr2=0x7ff460104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:29.740 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.117+0000 7ff46489f700 1 -- 172.21.15.71:0/3885710148 shutdown_connections 2024-03-04T08:47:29.740 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:29.117+0000 7ff46489f700 1 -- 172.21.15.71:0/3885710148 wait complete. 2024-03-04T08:47:29.740 INFO:teuthology.orchestra.run.smithi071.stdout:mon is available 2024-03-04T08:47:29.741 INFO:teuthology.orchestra.run.smithi071.stdout:Assimilating anything we can from ceph.conf... 2024-03-04T08:47:30.808 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:30 smithi071 ceph-mon[32416]: from='client.? 172.21.15.71:0/2160485498' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-04T08:47:30.808 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:30 smithi071 ceph-mon[32416]: from='client.? 172.21.15.71:0/2160485498' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-04T08:47:30.817 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-03-04T08:47:30.817 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [global] 2024-03-04T08:47:30.817 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout fsid = b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T08:47:30.817 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.71:3300,v1:172.21.15.71:6789] 2024-03-04T08:47:30.817 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-04T08:47:30.818 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-04T08:47:30.818 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-04T08:47:30.818 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-04T08:47:30.818 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-03-04T08:47:30.818 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-04T08:47:30.818 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-04T08:47:30.818 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-03-04T08:47:30.818 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [osd] 2024-03-04T08:47:30.818 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-04T08:47:30.818 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-04T08:47:30.818 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-04T08:47:30.818 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-04T08:47:30.818 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-04T08:47:30.819 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.173+0000 7fd256434700 1 Processor -- start 2024-03-04T08:47:30.819 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.173+0000 7fd256434700 1 -- start start 2024-03-04T08:47:30.819 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.173+0000 7fd256434700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd250107a10 0x7fd250107e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:30.819 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.173+0000 7fd256434700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd250108400 con 0x7fd250107a10 2024-03-04T08:47:30.819 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.173+0000 7fd24ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd250107a10 0x7fd250107e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:30.819 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.173+0000 7fd24ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd250107a10 0x7fd250107e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39374/0 (socket says 172.21.15.71:39374) 2024-03-04T08:47:30.819 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.173+0000 7fd24ffff700 1 -- 172.21.15.71:0/4025521417 learned_addr learned my addr 172.21.15.71:0/4025521417 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:30.819 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.173+0000 7fd24ffff700 1 -- 172.21.15.71:0/4025521417 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd250108c10 con 0x7fd250107a10 2024-03-04T08:47:30.819 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.173+0000 7fd24ffff700 1 --2- 172.21.15.71:0/4025521417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd250107a10 0x7fd250107e30 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd2400098a0 tx=0x7fd240009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6328375a3e06f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:30.819 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.174+0000 7fd24effd700 1 -- 172.21.15.71:0/4025521417 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd240004160 con 0x7fd250107a10 2024-03-04T08:47:30.819 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.174+0000 7fd24effd700 1 -- 172.21.15.71:0/4025521417 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd240004300 con 0x7fd250107a10 2024-03-04T08:47:30.819 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.174+0000 7fd24effd700 1 -- 172.21.15.71:0/4025521417 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2400036a0 con 0x7fd250107a10 2024-03-04T08:47:30.819 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.174+0000 7fd256434700 1 -- 172.21.15.71:0/4025521417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd250107a10 msgr2=0x7fd250107e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:30.820 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.174+0000 7fd256434700 1 --2- 172.21.15.71:0/4025521417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd250107a10 0x7fd250107e30 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd2400098a0 tx=0x7fd240009bb0 comp rx=0 tx=0).stop 2024-03-04T08:47:30.820 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.174+0000 7fd256434700 1 -- 172.21.15.71:0/4025521417 shutdown_connections 2024-03-04T08:47:30.820 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.174+0000 7fd256434700 1 --2- 172.21.15.71:0/4025521417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd250107a10 0x7fd250107e30 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:30.820 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.174+0000 7fd256434700 1 -- 172.21.15.71:0/4025521417 >> 172.21.15.71:0/4025521417 conn(0x7fd250103070 msgr2=0x7fd250105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:30.820 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.174+0000 7fd256434700 1 -- 172.21.15.71:0/4025521417 shutdown_connections 2024-03-04T08:47:30.820 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.174+0000 7fd256434700 1 -- 172.21.15.71:0/4025521417 wait complete. 2024-03-04T08:47:30.820 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.175+0000 7fd256434700 1 Processor -- start 2024-03-04T08:47:30.820 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.175+0000 7fd256434700 1 -- start start 2024-03-04T08:47:30.820 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.175+0000 7fd256434700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd250107a10 0x7fd25019b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:30.820 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.175+0000 7fd256434700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd250109140 con 0x7fd250107a10 2024-03-04T08:47:30.820 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.175+0000 7fd24ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd250107a10 0x7fd25019b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:30.820 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.175+0000 7fd24ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd250107a10 0x7fd25019b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39376/0 (socket says 172.21.15.71:39376) 2024-03-04T08:47:30.820 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.175+0000 7fd24ffff700 1 -- 172.21.15.71:0/2160485498 learned_addr learned my addr 172.21.15.71:0/2160485498 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:30.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.175+0000 7fd24ffff700 1 -- 172.21.15.71:0/2160485498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25019b680 con 0x7fd250107a10 2024-03-04T08:47:30.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.176+0000 7fd24ffff700 1 --2- 172.21.15.71:0/2160485498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd250107a10 0x7fd25019b140 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd240000c00 tx=0x7fd240004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:30.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.176+0000 7fd24d7fa700 1 -- 172.21.15.71:0/2160485498 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd240003db0 con 0x7fd250107a10 2024-03-04T08:47:30.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.176+0000 7fd24d7fa700 1 -- 172.21.15.71:0/2160485498 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd240011630 con 0x7fd250107a10 2024-03-04T08:47:30.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.176+0000 7fd24d7fa700 1 -- 172.21.15.71:0/2160485498 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd240011950 con 0x7fd250107a10 2024-03-04T08:47:30.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.176+0000 7fd256434700 1 -- 172.21.15.71:0/2160485498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25019b910 con 0x7fd250107a10 2024-03-04T08:47:30.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.176+0000 7fd256434700 1 -- 172.21.15.71:0/2160485498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25019e600 con 0x7fd250107a10 2024-03-04T08:47:30.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.176+0000 7fd24d7fa700 1 -- 172.21.15.71:0/2160485498 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd24001a460 con 0x7fd250107a10 2024-03-04T08:47:30.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.176+0000 7fd24d7fa700 1 -- 172.21.15.71:0/2160485498 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd24001b780 con 0x7fd250107a10 2024-03-04T08:47:30.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.177+0000 7fd256434700 1 -- 172.21.15.71:0/2160485498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd250069150 con 0x7fd250107a10 2024-03-04T08:47:30.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.179+0000 7fd24d7fa700 1 -- 172.21.15.71:0/2160485498 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd240031020 con 0x7fd250107a10 2024-03-04T08:47:30.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.246+0000 7fd256434700 1 -- 172.21.15.71:0/2160485498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fd25002dc00 con 0x7fd250107a10 2024-03-04T08:47:30.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.249+0000 7fd24d7fa700 1 -- 172.21.15.71:0/2160485498 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fd24001b9d0 con 0x7fd250107a10 2024-03-04T08:47:30.822 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.250+0000 7fd24d7fa700 1 -- 172.21.15.71:0/2160485498 <== mon.0 v2:172.21.15.71:3300/0 8 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd24001ad50 con 0x7fd250107a10 2024-03-04T08:47:30.822 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.250+0000 7fd256434700 1 -- 172.21.15.71:0/2160485498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd250107a10 msgr2=0x7fd25019b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:30.822 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.250+0000 7fd256434700 1 --2- 172.21.15.71:0/2160485498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd250107a10 0x7fd25019b140 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd240000c00 tx=0x7fd240004750 comp rx=0 tx=0).stop 2024-03-04T08:47:30.822 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.251+0000 7fd256434700 1 -- 172.21.15.71:0/2160485498 shutdown_connections 2024-03-04T08:47:30.822 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.251+0000 7fd256434700 1 --2- 172.21.15.71:0/2160485498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd250107a10 0x7fd25019b140 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:30.822 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.251+0000 7fd256434700 1 -- 172.21.15.71:0/2160485498 >> 172.21.15.71:0/2160485498 conn(0x7fd250103070 msgr2=0x7fd250191d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:30.822 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.251+0000 7fd256434700 1 -- 172.21.15.71:0/2160485498 shutdown_connections 2024-03-04T08:47:30.822 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:30.251+0000 7fd256434700 1 -- 172.21.15.71:0/2160485498 wait complete. 2024-03-04T08:47:30.822 INFO:teuthology.orchestra.run.smithi071.stdout:Generating new minimal ceph.conf... 2024-03-04T08:47:32.097 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:31 smithi071 ceph-mon[32416]: from='client.? 172.21.15.71:0/1417829589' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:47:32.358 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.730+0000 7f035eed8700 1 Processor -- start 2024-03-04T08:47:32.364 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.731+0000 7f035eed8700 1 -- start start 2024-03-04T08:47:32.364 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.731+0000 7f035eed8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03581068f0 0x7f0358106d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:32.364 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.731+0000 7f035eed8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03581072e0 con 0x7f03581068f0 2024-03-04T08:47:32.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.731+0000 7f035cc74700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03581068f0 0x7f0358106d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:32.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.731+0000 7f035cc74700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03581068f0 0x7f0358106d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39388/0 (socket says 172.21.15.71:39388) 2024-03-04T08:47:32.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.731+0000 7f035cc74700 1 -- 172.21.15.71:0/4119367169 learned_addr learned my addr 172.21.15.71:0/4119367169 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:32.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.731+0000 7f035cc74700 1 -- 172.21.15.71:0/4119367169 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0358107af0 con 0x7f03581068f0 2024-03-04T08:47:32.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.731+0000 7f035cc74700 1 --2- 172.21.15.71:0/4119367169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03581068f0 0x7f0358106d10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f034c0098a0 tx=0x7f034c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e622b97e4b1bf88 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:32.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.732+0000 7f03577fe700 1 -- 172.21.15.71:0/4119367169 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f034c004160 con 0x7f03581068f0 2024-03-04T08:47:32.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.732+0000 7f03577fe700 1 -- 172.21.15.71:0/4119367169 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f034c004300 con 0x7f03581068f0 2024-03-04T08:47:32.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.732+0000 7f03577fe700 1 -- 172.21.15.71:0/4119367169 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f034c011b50 con 0x7f03581068f0 2024-03-04T08:47:32.365 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.732+0000 7f035eed8700 1 -- 172.21.15.71:0/4119367169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03581068f0 msgr2=0x7f0358106d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:32.366 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.732+0000 7f035eed8700 1 --2- 172.21.15.71:0/4119367169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03581068f0 0x7f0358106d10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f034c0098a0 tx=0x7f034c009bb0 comp rx=0 tx=0).stop 2024-03-04T08:47:32.366 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.732+0000 7f035eed8700 1 -- 172.21.15.71:0/4119367169 shutdown_connections 2024-03-04T08:47:32.366 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.732+0000 7f035eed8700 1 --2- 172.21.15.71:0/4119367169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03581068f0 0x7f0358106d10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:32.366 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.732+0000 7f035eed8700 1 -- 172.21.15.71:0/4119367169 >> 172.21.15.71:0/4119367169 conn(0x7f0358101f30 msgr2=0x7f0358104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:32.366 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.732+0000 7f035eed8700 1 -- 172.21.15.71:0/4119367169 shutdown_connections 2024-03-04T08:47:32.366 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.732+0000 7f035eed8700 1 -- 172.21.15.71:0/4119367169 wait complete. 2024-03-04T08:47:32.366 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.733+0000 7f035eed8700 1 Processor -- start 2024-03-04T08:47:32.367 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.733+0000 7f035eed8700 1 -- start start 2024-03-04T08:47:32.367 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.733+0000 7f035eed8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03581068f0 0x7f035819ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:32.367 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.733+0000 7f035eed8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035810aea0 con 0x7f03581068f0 2024-03-04T08:47:32.367 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.733+0000 7f035cc74700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03581068f0 0x7f035819ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:32.367 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.733+0000 7f035cc74700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03581068f0 0x7f035819ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39402/0 (socket says 172.21.15.71:39402) 2024-03-04T08:47:32.367 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.733+0000 7f035cc74700 1 -- 172.21.15.71:0/1417829589 learned_addr learned my addr 172.21.15.71:0/1417829589 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:32.367 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.733+0000 7f035cc74700 1 -- 172.21.15.71:0/1417829589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035819bfa0 con 0x7f03581068f0 2024-03-04T08:47:32.367 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.734+0000 7f035cc74700 1 --2- 172.21.15.71:0/1417829589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03581068f0 0x7f035819ba60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f034c011850 tx=0x7f034c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:32.367 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.734+0000 7f0355ffb700 1 -- 172.21.15.71:0/1417829589 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f034c003af0 con 0x7f03581068f0 2024-03-04T08:47:32.367 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.734+0000 7f0355ffb700 1 -- 172.21.15.71:0/1417829589 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f034c020070 con 0x7f03581068f0 2024-03-04T08:47:32.368 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.734+0000 7f035eed8700 1 -- 172.21.15.71:0/1417829589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f035819c230 con 0x7f03581068f0 2024-03-04T08:47:32.368 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.734+0000 7f0355ffb700 1 -- 172.21.15.71:0/1417829589 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f034c0195a0 con 0x7f03581068f0 2024-03-04T08:47:32.368 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.734+0000 7f035eed8700 1 -- 172.21.15.71:0/1417829589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f035819ef20 con 0x7f03581068f0 2024-03-04T08:47:32.368 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.734+0000 7f0355ffb700 1 -- 172.21.15.71:0/1417829589 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f034c019be0 con 0x7f03581068f0 2024-03-04T08:47:32.368 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.734+0000 7f0355ffb700 1 -- 172.21.15.71:0/1417829589 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f034c011e00 con 0x7f03581068f0 2024-03-04T08:47:32.368 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.735+0000 7f035eed8700 1 -- 172.21.15.71:0/1417829589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0358069150 con 0x7f03581068f0 2024-03-04T08:47:32.368 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.737+0000 7f0355ffb700 1 -- 172.21.15.71:0/1417829589 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f034c003c90 con 0x7f03581068f0 2024-03-04T08:47:32.368 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.804+0000 7f035eed8700 1 -- 172.21.15.71:0/1417829589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f035802dc20 con 0x7f03581068f0 2024-03-04T08:47:32.368 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.804+0000 7f0355ffb700 1 -- 172.21.15.71:0/1417829589 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f034c025460 con 0x7f03581068f0 2024-03-04T08:47:32.368 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.805+0000 7f035eed8700 1 -- 172.21.15.71:0/1417829589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03581068f0 msgr2=0x7f035819ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:32.368 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.805+0000 7f035eed8700 1 --2- 172.21.15.71:0/1417829589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03581068f0 0x7f035819ba60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f034c011850 tx=0x7f034c011c10 comp rx=0 tx=0).stop 2024-03-04T08:47:32.368 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.805+0000 7f035eed8700 1 -- 172.21.15.71:0/1417829589 shutdown_connections 2024-03-04T08:47:32.368 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.805+0000 7f035eed8700 1 --2- 172.21.15.71:0/1417829589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03581068f0 0x7f035819ba60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:32.369 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.805+0000 7f035eed8700 1 -- 172.21.15.71:0/1417829589 >> 172.21.15.71:0/1417829589 conn(0x7f0358101f30 msgr2=0x7f0358102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:32.369 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.805+0000 7f035eed8700 1 -- 172.21.15.71:0/1417829589 shutdown_connections 2024-03-04T08:47:32.369 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:31.805+0000 7f035eed8700 1 -- 172.21.15.71:0/1417829589 wait complete. 2024-03-04T08:47:32.369 INFO:teuthology.orchestra.run.smithi071.stdout:Restarting the monitor... 2024-03-04T08:47:32.619 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:32 smithi071 systemd[1]: Stopping Ceph mon.a for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T08:47:32.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:32 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-a[32407]: 2024-03-04T08:47:32.621+0000 7fc787fcc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T08:47:32.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:32 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-a[32407]: 2024-03-04T08:47:32.621+0000 7fc787fcc700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-04T08:47:32.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:32 smithi071 podman[32662]: 2024-03-04 08:47:32.724778755 +0000 UTC m=+0.195502295 container died 6f06e7b94b9d7710946fad0301ee58924d8522284365b91da72cf279ae39cac1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-a, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-04T08:47:33.218 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 podman[32662]: 2024-03-04 08:47:33.030315679 +0000 UTC m=+0.501039221 container cleanup 6f06e7b94b9d7710946fad0301ee58924d8522284365b91da72cf279ae39cac1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-a, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-03-04T08:47:33.219 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 bash[32662]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-a 2024-03-04T08:47:33.644 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 podman[32676]: 2024-03-04 08:47:33.223319122 +0000 UTC m=+0.499207406 container remove 6f06e7b94b9d7710946fad0301ee58924d8522284365b91da72cf279ae39cac1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-03-04T08:47:33.644 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 systemd[1]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-04T08:47:33.647 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 systemd[1]: Stopped Ceph mon.a for b3c85c40-da03-11ee-95c6-87774f69a715. 2024-03-04T08:47:33.647 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 systemd[1]: Starting Ceph mon.a for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T08:47:33.876 INFO:teuthology.orchestra.run.smithi071.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-04T08:47:33.919 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 podman[32771]: 2024-03-04 08:47:33.649262454 +0000 UTC m=+0.096465786 container create c1582a68dbdaef2eeb47857e48d2011d0c4578c625f6ad27d898f5c5c8f3d9ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.schema-version=1.0) 2024-03-04T08:47:33.920 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 podman[32771]: 2024-03-04 08:47:33.570422377 +0000 UTC m=+0.017625709 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T08:47:33.920 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 podman[32771]: 2024-03-04 08:47:33.818138444 +0000 UTC m=+0.265341776 container init c1582a68dbdaef2eeb47857e48d2011d0c4578c625f6ad27d898f5c5c8f3d9ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T08:47:33.920 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 podman[32771]: 2024-03-04 08:47:33.820876469 +0000 UTC m=+0.268079806 container start c1582a68dbdaef2eeb47857e48d2011d0c4578c625f6ad27d898f5c5c8f3d9ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-a, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T08:47:33.920 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: set uid:gid to 167:167 (ceph:ceph) 2024-03-04T08:47:33.920 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: ceph version 19.0.0-1739-g1bc250c7 (1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd) squid (dev), process ceph-mon, pid 2 2024-03-04T08:47:33.920 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: pidfile_write: ignore empty --pid-file 2024-03-04T08:47:33.920 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: load: jerasure load: lrc 2024-03-04T08:47:33.920 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: RocksDB version: 7.9.2 2024-03-04T08:47:33.920 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Git sha 0 2024-03-04T08:47:33.920 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Compile date 2024-03-03 23:16:35 2024-03-04T08:47:33.920 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: DB SUMMARY 2024-03-04T08:47:33.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: DB Session ID: JMTBSXY07391KSBPUA9F 2024-03-04T08:47:33.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: CURRENT file: CURRENT 2024-03-04T08:47:33.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: IDENTITY file: IDENTITY 2024-03-04T08:47:33.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-04T08:47:33.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-04T08:47:33.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99150 ; 2024-03-04T08:47:33.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.error_if_exists: 0 2024-03-04T08:47:33.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.create_if_missing: 0 2024-03-04T08:47:33.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.paranoid_checks: 1 2024-03-04T08:47:33.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-04T08:47:33.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-04T08:47:33.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-04T08:47:33.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.env: 0x5619e49e1b20 2024-03-04T08:47:33.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.fs: PosixFileSystem 2024-03-04T08:47:33.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.info_log: 0x5619e62ef920 2024-03-04T08:47:33.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_file_opening_threads: 16 2024-03-04T08:47:33.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.statistics: (nil) 2024-03-04T08:47:33.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.use_fsync: 0 2024-03-04T08:47:33.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_log_file_size: 0 2024-03-04T08:47:33.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-04T08:47:33.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-04T08:47:33.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.keep_log_file_num: 1000 2024-03-04T08:47:33.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.recycle_log_file_num: 0 2024-03-04T08:47:33.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.allow_fallocate: 1 2024-03-04T08:47:33.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.allow_mmap_reads: 0 2024-03-04T08:47:33.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.allow_mmap_writes: 0 2024-03-04T08:47:33.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.use_direct_reads: 0 2024-03-04T08:47:33.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-04T08:47:33.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.create_missing_column_families: 0 2024-03-04T08:47:33.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.db_log_dir: 2024-03-04T08:47:33.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.wal_dir: 2024-03-04T08:47:33.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-04T08:47:33.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-04T08:47:33.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-04T08:47:33.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-04T08:47:33.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-04T08:47:33.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-04T08:47:33.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.advise_random_on_open: 1 2024-03-04T08:47:33.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.db_write_buffer_size: 0 2024-03-04T08:47:33.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.write_buffer_manager: 0x5619e62c3400 2024-03-04T08:47:33.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-04T08:47:33.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-04T08:47:33.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-04T08:47:33.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.rate_limiter: (nil) 2024-03-04T08:47:33.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-04T08:47:33.925 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.wal_recovery_mode: 2 2024-03-04T08:47:33.925 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.enable_thread_tracking: 0 2024-03-04T08:47:33.925 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.enable_pipelined_write: 0 2024-03-04T08:47:33.925 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.unordered_write: 0 2024-03-04T08:47:33.925 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-04T08:47:33.926 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-04T08:47:33.926 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-04T08:47:33.926 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-04T08:47:33.926 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.row_cache: None 2024-03-04T08:47:33.926 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.wal_filter: None 2024-03-04T08:47:33.926 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-04T08:47:33.926 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.allow_ingest_behind: 0 2024-03-04T08:47:33.926 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.two_write_queues: 0 2024-03-04T08:47:33.926 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.manual_wal_flush: 0 2024-03-04T08:47:33.926 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.wal_compression: 0 2024-03-04T08:47:33.926 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.atomic_flush: 0 2024-03-04T08:47:33.926 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-04T08:47:33.926 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-04T08:47:33.926 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-04T08:47:33.927 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.log_readahead_size: 0 2024-03-04T08:47:33.927 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-04T08:47:33.927 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.best_efforts_recovery: 0 2024-03-04T08:47:33.927 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-04T08:47:33.927 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-04T08:47:33.927 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.allow_data_in_errors: 0 2024-03-04T08:47:33.927 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.db_host_id: __hostname__ 2024-03-04T08:47:33.927 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-04T08:47:33.927 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_background_jobs: 2 2024-03-04T08:47:33.927 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_background_compactions: -1 2024-03-04T08:47:33.927 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_subcompactions: 1 2024-03-04T08:47:33.927 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-04T08:47:33.927 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-04T08:47:33.927 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-04T08:47:33.928 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_total_wal_size: 0 2024-03-04T08:47:33.928 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-04T08:47:33.928 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-04T08:47:33.928 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-04T08:47:33.928 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-04T08:47:33.928 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_open_files: -1 2024-03-04T08:47:33.928 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.bytes_per_sync: 0 2024-03-04T08:47:33.928 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-04T08:47:33.928 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-04T08:47:33.928 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.compaction_readahead_size: 0 2024-03-04T08:47:33.928 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_background_flushes: -1 2024-03-04T08:47:33.928 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Compression algorithms supported: 2024-03-04T08:47:33.928 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-04T08:47:33.929 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: kZSTD supported: 0 2024-03-04T08:47:33.929 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: kXpressCompression supported: 0 2024-03-04T08:47:33.929 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: kLZ4HCCompression supported: 1 2024-03-04T08:47:33.929 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: kZlibCompression supported: 1 2024-03-04T08:47:33.929 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: kSnappyCompression supported: 1 2024-03-04T08:47:33.929 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: kLZ4Compression supported: 1 2024-03-04T08:47:33.929 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: kBZip2Compression supported: 0 2024-03-04T08:47:33.929 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-04T08:47:33.929 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-04T08:47:33.929 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:47:33.929 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-04T08:47:33.929 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-04T08:47:33.929 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.merge_operator: 2024-03-04T08:47:33.929 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.compaction_filter: None 2024-03-04T08:47:33.929 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.compaction_filter_factory: None 2024-03-04T08:47:33.930 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.sst_partitioner_factory: None 2024-03-04T08:47:33.930 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-04T08:47:33.930 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-04T08:47:33.930 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5619e62efa20) 2024-03-04T08:47:33.930 INFO:journalctl@ceph.mon.a.smithi071.stdout: cache_index_and_filter_blocks: 1 2024-03-04T08:47:33.930 INFO:journalctl@ceph.mon.a.smithi071.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-04T08:47:33.930 INFO:journalctl@ceph.mon.a.smithi071.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-04T08:47:33.930 INFO:journalctl@ceph.mon.a.smithi071.stdout: pin_top_level_index_and_filter: 1 2024-03-04T08:47:33.930 INFO:journalctl@ceph.mon.a.smithi071.stdout: index_type: 0 2024-03-04T08:47:33.930 INFO:journalctl@ceph.mon.a.smithi071.stdout: data_block_index_type: 0 2024-03-04T08:47:33.930 INFO:journalctl@ceph.mon.a.smithi071.stdout: index_shortening: 1 2024-03-04T08:47:33.930 INFO:journalctl@ceph.mon.a.smithi071.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-04T08:47:33.930 INFO:journalctl@ceph.mon.a.smithi071.stdout: checksum: 4 2024-03-04T08:47:33.930 INFO:journalctl@ceph.mon.a.smithi071.stdout: no_block_cache: 0 2024-03-04T08:47:33.930 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_cache: 0x5619e62f3090 2024-03-04T08:47:33.931 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_cache_name: BinnedLRUCache 2024-03-04T08:47:33.931 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_cache_options: 2024-03-04T08:47:33.931 INFO:journalctl@ceph.mon.a.smithi071.stdout: capacity : 536870912 2024-03-04T08:47:33.931 INFO:journalctl@ceph.mon.a.smithi071.stdout: num_shard_bits : 4 2024-03-04T08:47:33.931 INFO:journalctl@ceph.mon.a.smithi071.stdout: strict_capacity_limit : 0 2024-03-04T08:47:33.931 INFO:journalctl@ceph.mon.a.smithi071.stdout: high_pri_pool_ratio: 0.000 2024-03-04T08:47:33.931 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_cache_compressed: (nil) 2024-03-04T08:47:33.931 INFO:journalctl@ceph.mon.a.smithi071.stdout: persistent_cache: (nil) 2024-03-04T08:47:33.931 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_size: 4096 2024-03-04T08:47:33.931 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_size_deviation: 10 2024-03-04T08:47:33.931 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_restart_interval: 16 2024-03-04T08:47:33.931 INFO:journalctl@ceph.mon.a.smithi071.stdout: index_block_restart_interval: 1 2024-03-04T08:47:33.932 INFO:journalctl@ceph.mon.a.smithi071.stdout: metadata_block_size: 4096 2024-03-04T08:47:33.932 INFO:journalctl@ceph.mon.a.smithi071.stdout: partition_filters: 0 2024-03-04T08:47:33.932 INFO:journalctl@ceph.mon.a.smithi071.stdout: use_delta_encoding: 1 2024-03-04T08:47:33.932 INFO:journalctl@ceph.mon.a.smithi071.stdout: filter_policy: bloomfilter 2024-03-04T08:47:33.932 INFO:journalctl@ceph.mon.a.smithi071.stdout: whole_key_filtering: 1 2024-03-04T08:47:33.932 INFO:journalctl@ceph.mon.a.smithi071.stdout: verify_compression: 0 2024-03-04T08:47:33.932 INFO:journalctl@ceph.mon.a.smithi071.stdout: read_amp_bytes_per_bit: 0 2024-03-04T08:47:33.932 INFO:journalctl@ceph.mon.a.smithi071.stdout: format_version: 5 2024-03-04T08:47:33.932 INFO:journalctl@ceph.mon.a.smithi071.stdout: enable_index_compression: 1 2024-03-04T08:47:33.932 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_align: 0 2024-03-04T08:47:33.932 INFO:journalctl@ceph.mon.a.smithi071.stdout: max_auto_readahead_size: 262144 2024-03-04T08:47:33.932 INFO:journalctl@ceph.mon.a.smithi071.stdout: prepopulate_block_cache: 0 2024-03-04T08:47:33.932 INFO:journalctl@ceph.mon.a.smithi071.stdout: initial_auto_readahead_size: 8192 2024-03-04T08:47:33.933 INFO:journalctl@ceph.mon.a.smithi071.stdout: num_file_reads_for_auto_readahead: 2 2024-03-04T08:47:33.933 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.write_buffer_size: 33554432 2024-03-04T08:47:33.933 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_write_buffer_number: 2 2024-03-04T08:47:33.933 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.compression: NoCompression 2024-03-04T08:47:33.933 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.bottommost_compression: Disabled 2024-03-04T08:47:33.933 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.prefix_extractor: nullptr 2024-03-04T08:47:33.933 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-04T08:47:33.933 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.num_levels: 7 2024-03-04T08:47:33.933 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-04T08:47:33.933 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-04T08:47:33.933 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-04T08:47:33.933 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-04T08:47:33.933 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-04T08:47:33.933 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-04T08:47:33.934 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-04T08:47:33.934 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-04T08:47:33.934 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-04T08:47:33.934 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-04T08:47:33.934 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-04T08:47:33.934 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-04T08:47:33.934 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-04T08:47:33.934 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.compression_opts.level: 32767 2024-03-04T08:47:33.934 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.compression_opts.strategy: 0 2024-03-04T08:47:33.934 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-04T08:47:33.934 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-04T08:47:33.934 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-04T08:47:33.934 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-04T08:47:33.934 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.compression_opts.enabled: false 2024-03-04T08:47:33.935 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-04T08:47:33.935 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-04T08:47:33.935 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-04T08:47:33.935 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-04T08:47:33.935 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.target_file_size_base: 67108864 2024-03-04T08:47:33.935 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-04T08:47:33.935 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-04T08:47:33.935 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-04T08:47:33.935 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-04T08:47:33.935 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-04T08:47:33.935 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-04T08:47:33.935 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-04T08:47:33.935 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-04T08:47:33.935 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-04T08:47:33.936 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-04T08:47:33.936 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-04T08:47:33.936 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-04T08:47:33.936 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-04T08:47:33.936 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-04T08:47:33.936 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.arena_block_size: 1048576 2024-03-04T08:47:33.936 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-04T08:47:33.936 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-04T08:47:33.936 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.disable_auto_compactions: 0 2024-03-04T08:47:33.936 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-04T08:47:33.936 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-04T08:47:33.936 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-04T08:47:33.936 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-04T08:47:33.936 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-04T08:47:33.937 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-04T08:47:33.937 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-04T08:47:33.937 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-04T08:47:33.937 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-04T08:47:33.937 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-04T08:47:33.937 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.table_properties_collectors: 2024-03-04T08:47:33.937 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.inplace_update_support: 0 2024-03-04T08:47:33.937 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-04T08:47:33.937 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-04T08:47:33.937 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-04T08:47:33.937 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-04T08:47:33.937 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.bloom_locality: 0 2024-03-04T08:47:33.937 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.max_successive_merges: 0 2024-03-04T08:47:33.937 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-04T08:47:33.938 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.paranoid_file_checks: 0 2024-03-04T08:47:33.938 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.force_consistency_checks: 1 2024-03-04T08:47:33.938 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.report_bg_io_stats: 0 2024-03-04T08:47:33.938 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.ttl: 2592000 2024-03-04T08:47:33.938 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-04T08:47:33.938 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-04T08:47:33.938 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-04T08:47:33.938 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.enable_blob_files: false 2024-03-04T08:47:33.938 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.min_blob_size: 0 2024-03-04T08:47:33.938 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.blob_file_size: 268435456 2024-03-04T08:47:33.938 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-04T08:47:33.938 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-04T08:47:33.938 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-04T08:47:33.938 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-04T08:47:33.938 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-04T08:47:33.939 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.blob_file_starting_level: 0 2024-03-04T08:47:33.939 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-04T08:47:33.939 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-04T08:47:33.939 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-04T08:47:33.939 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7cfcacda-d955-441f-856a-27e7867c3992 2024-03-04T08:47:33.939 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709542053849244, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-04T08:47:33.939 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-04T08:47:33.939 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709542053850184, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94424, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 314, "table_properties": {"data_size": 92424, "index_size": 282, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13640, "raw_average_key_size": 49, "raw_value_size": 84961, "raw_average_value_size": 311, "num_data_blocks": 11, "num_entries": 273, "num_filter_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1709542053, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7cfcacda-d955-441f-856a-27e7867c3992", "db_session_id": "JMTBSXY07391KSBPUA9F", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-04T08:47:33.939 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709542053850264, "job": 1, "event": "recovery_finished"} 2024-03-04T08:47:33.939 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-04T08:47:33.939 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-04T08:47:33.939 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5619e63e4000 2024-03-04T08:47:33.939 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: DB pointer 0x5619e63ce000 2024-03-04T08:47:33.939 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-04T08:47:33.940 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-04T08:47:33.940 INFO:journalctl@ceph.mon.a.smithi071.stdout: ** DB Stats ** 2024-03-04T08:47:33.940 INFO:journalctl@ceph.mon.a.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-04T08:47:33.940 INFO:journalctl@ceph.mon.a.smithi071.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-04T08:47:33.940 INFO:journalctl@ceph.mon.a.smithi071.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-04T08:47:33.940 INFO:journalctl@ceph.mon.a.smithi071.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-04T08:47:33.940 INFO:journalctl@ceph.mon.a.smithi071.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-04T08:47:33.940 INFO:journalctl@ceph.mon.a.smithi071.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-04T08:47:33.940 INFO:journalctl@ceph.mon.a.smithi071.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-04T08:47:33.940 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-03-04T08:47:33.940 INFO:journalctl@ceph.mon.a.smithi071.stdout: ** Compaction Stats [default] ** 2024-03-04T08:47:33.940 INFO:journalctl@ceph.mon.a.smithi071.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-04T08:47:33.940 INFO:journalctl@ceph.mon.a.smithi071.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-04T08:47:33.940 INFO:journalctl@ceph.mon.a.smithi071.stdout: L0 2/0 94.05 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 153.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-04T08:47:33.941 INFO:journalctl@ceph.mon.a.smithi071.stdout: Sum 2/0 94.05 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 153.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-04T08:47:33.941 INFO:journalctl@ceph.mon.a.smithi071.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 153.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-04T08:47:33.941 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-03-04T08:47:33.941 INFO:journalctl@ceph.mon.a.smithi071.stdout: ** Compaction Stats [default] ** 2024-03-04T08:47:33.941 INFO:journalctl@ceph.mon.a.smithi071.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-04T08:47:33.941 INFO:journalctl@ceph.mon.a.smithi071.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-04T08:47:33.941 INFO:journalctl@ceph.mon.a.smithi071.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 153.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-04T08:47:33.941 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-03-04T08:47:33.941 INFO:journalctl@ceph.mon.a.smithi071.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-04T08:47:33.941 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-03-04T08:47:33.941 INFO:journalctl@ceph.mon.a.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-04T08:47:33.941 INFO:journalctl@ceph.mon.a.smithi071.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-04T08:47:33.941 INFO:journalctl@ceph.mon.a.smithi071.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-04T08:47:33.941 INFO:journalctl@ceph.mon.a.smithi071.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-04T08:47:33.942 INFO:journalctl@ceph.mon.a.smithi071.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-04T08:47:33.942 INFO:journalctl@ceph.mon.a.smithi071.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-04T08:47:33.942 INFO:journalctl@ceph.mon.a.smithi071.stdout: Cumulative compaction: 0.00 GB write, 31.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-04T08:47:33.942 INFO:journalctl@ceph.mon.a.smithi071.stdout: Interval compaction: 0.00 GB write, 31.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-04T08:47:33.942 INFO:journalctl@ceph.mon.a.smithi071.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-04T08:47:33.942 INFO:journalctl@ceph.mon.a.smithi071.stdout: Block cache BinnedLRUCache@0x5619e62f3090#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-03-04T08:47:33.942 INFO:journalctl@ceph.mon.a.smithi071.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-03-04T08:47:33.942 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-03-04T08:47:33.942 INFO:journalctl@ceph.mon.a.smithi071.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-04T08:47:33.942 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: starting mon.a rank 0 at public addrs [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] at bind addrs [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T08:47:33.942 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: mon.a@-1(???) e1 preinit fsid b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T08:47:33.942 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: mon.a@-1(???).mds e1 new map 2024-03-04T08:47:33.942 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: mon.a@-1(???).mds e1 print_map 2024-03-04T08:47:33.942 INFO:journalctl@ceph.mon.a.smithi071.stdout: e1 2024-03-04T08:47:33.942 INFO:journalctl@ceph.mon.a.smithi071.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T08:47:33.943 INFO:journalctl@ceph.mon.a.smithi071.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-04T08:47:33.943 INFO:journalctl@ceph.mon.a.smithi071.stdout: legacy client fscid: -1 2024-03-04T08:47:33.943 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-03-04T08:47:33.943 INFO:journalctl@ceph.mon.a.smithi071.stdout: No filesystems configured 2024-03-04T08:47:33.943 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-04T08:47:33.943 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-04T08:47:33.943 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-04T08:47:33.943 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-04T08:47:33.943 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-04T08:47:33.943 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 bash[32771]: c1582a68dbdaef2eeb47857e48d2011d0c4578c625f6ad27d898f5c5c8f3d9ef 2024-03-04T08:47:33.943 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 systemd[1]: Started Ceph mon.a for b3c85c40-da03-11ee-95c6-87774f69a715. 2024-03-04T08:47:33.943 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-04T08:47:33.943 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: monmap e1: 1 mons at {a=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T08:47:33.943 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: fsmap 2024-03-04T08:47:33.943 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: osdmap e1: 0 total, 0 up, 0 in 2024-03-04T08:47:33.944 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:33 smithi071 ceph-mon[32789]: mgrmap e1: no daemons active 2024-03-04T08:47:34.954 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.331+0000 7f59a8b42700 1 Processor -- start 2024-03-04T08:47:34.954 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.331+0000 7f59a8b42700 1 -- start start 2024-03-04T08:47:34.954 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.331+0000 7f59a8b42700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59a4106240 0x7f59a4106660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:34.954 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.331+0000 7f59a8b42700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a407ae00 con 0x7f59a4106240 2024-03-04T08:47:34.954 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.332+0000 7f59a259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59a4106240 0x7f59a4106660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:34.954 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.332+0000 7f59a259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59a4106240 0x7f59a4106660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39418/0 (socket says 172.21.15.71:39418) 2024-03-04T08:47:34.954 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.332+0000 7f59a259c700 1 -- 172.21.15.71:0/1776957392 learned_addr learned my addr 172.21.15.71:0/1776957392 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:34.954 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.332+0000 7f59a259c700 1 -- 172.21.15.71:0/1776957392 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a4106ba0 con 0x7f59a4106240 2024-03-04T08:47:34.954 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.332+0000 7f59a259c700 1 --2- 172.21.15.71:0/1776957392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59a4106240 0x7f59a4106660 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f599c009e80 tx=0x7f599c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35679ac1745fb75 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:34.954 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.332+0000 7f59a159a700 1 -- 172.21.15.71:0/1776957392 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f599c017070 con 0x7f59a4106240 2024-03-04T08:47:34.954 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.332+0000 7f59a159a700 1 -- 172.21.15.71:0/1776957392 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f599c00b8c0 con 0x7f59a4106240 2024-03-04T08:47:34.955 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.332+0000 7f59a159a700 1 -- 172.21.15.71:0/1776957392 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f599c0045c0 con 0x7f59a4106240 2024-03-04T08:47:34.955 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.333+0000 7f59a8b42700 1 -- 172.21.15.71:0/1776957392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59a4106240 msgr2=0x7f59a4106660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:34.955 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.333+0000 7f59a8b42700 1 --2- 172.21.15.71:0/1776957392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59a4106240 0x7f59a4106660 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f599c009e80 tx=0x7f599c00b330 comp rx=0 tx=0).stop 2024-03-04T08:47:34.955 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.333+0000 7f59a8b42700 1 -- 172.21.15.71:0/1776957392 shutdown_connections 2024-03-04T08:47:34.955 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.333+0000 7f59a8b42700 1 --2- 172.21.15.71:0/1776957392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59a4106240 0x7f59a4106660 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:34.955 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.333+0000 7f59a8b42700 1 -- 172.21.15.71:0/1776957392 >> 172.21.15.71:0/1776957392 conn(0x7f59a4101e80 msgr2=0x7f59a41042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:34.955 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.333+0000 7f59a8b42700 1 -- 172.21.15.71:0/1776957392 shutdown_connections 2024-03-04T08:47:34.955 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.333+0000 7f59a8b42700 1 -- 172.21.15.71:0/1776957392 wait complete. 2024-03-04T08:47:34.955 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.334+0000 7f59a8b42700 1 Processor -- start 2024-03-04T08:47:34.955 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.334+0000 7f59a8b42700 1 -- start start 2024-03-04T08:47:34.955 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.334+0000 7f59a8b42700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59a4106240 0x7f59a419da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:34.955 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.334+0000 7f59a8b42700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a4107bc0 con 0x7f59a4106240 2024-03-04T08:47:34.955 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.334+0000 7f59a259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59a4106240 0x7f59a419da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:34.955 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.334+0000 7f59a259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59a4106240 0x7f59a419da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39420/0 (socket says 172.21.15.71:39420) 2024-03-04T08:47:34.956 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.334+0000 7f59a259c700 1 -- 172.21.15.71:0/1737312041 learned_addr learned my addr 172.21.15.71:0/1737312041 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:34.956 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.334+0000 7f59a259c700 1 -- 172.21.15.71:0/1737312041 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a419dfb0 con 0x7f59a4106240 2024-03-04T08:47:34.956 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.335+0000 7f59a259c700 1 --2- 172.21.15.71:0/1737312041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59a4106240 0x7f59a419da70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f599c009530 tx=0x7f599c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:34.956 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.335+0000 7f59937fe700 1 -- 172.21.15.71:0/1737312041 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f599c020020 con 0x7f59a4106240 2024-03-04T08:47:34.956 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.335+0000 7f59937fe700 1 -- 172.21.15.71:0/1737312041 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f599c003b50 con 0x7f59a4106240 2024-03-04T08:47:34.956 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.335+0000 7f59a8b42700 1 -- 172.21.15.71:0/1737312041 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59a419e240 con 0x7f59a4106240 2024-03-04T08:47:34.956 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.335+0000 7f59937fe700 1 -- 172.21.15.71:0/1737312041 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f599c017040 con 0x7f59a4106240 2024-03-04T08:47:34.956 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.335+0000 7f59a8b42700 1 -- 172.21.15.71:0/1737312041 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a41a0f30 con 0x7f59a4106240 2024-03-04T08:47:34.956 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.335+0000 7f59937fe700 1 -- 172.21.15.71:0/1737312041 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f599c023020 con 0x7f59a4106240 2024-03-04T08:47:34.956 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.335+0000 7f59937fe700 1 -- 172.21.15.71:0/1737312041 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f599c00f050 con 0x7f59a4106240 2024-03-04T08:47:34.956 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.336+0000 7f59a8b42700 1 -- 172.21.15.71:0/1737312041 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59a40636f0 con 0x7f59a4106240 2024-03-04T08:47:34.956 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.338+0000 7f59937fe700 1 -- 172.21.15.71:0/1737312041 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f599c02a020 con 0x7f59a4106240 2024-03-04T08:47:34.956 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.404+0000 7f59a8b42700 1 -- 172.21.15.71:0/1737312041 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f59a402dc40 con 0x7f59a4106240 2024-03-04T08:47:34.957 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.408+0000 7f59937fe700 1 -- 172.21.15.71:0/1737312041 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f599c00b860 con 0x7f59a4106240 2024-03-04T08:47:34.957 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.409+0000 7f59a8b42700 1 -- 172.21.15.71:0/1737312041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59a4106240 msgr2=0x7f59a419da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:34.957 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.409+0000 7f59a8b42700 1 --2- 172.21.15.71:0/1737312041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59a4106240 0x7f59a419da70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f599c009530 tx=0x7f599c00bd50 comp rx=0 tx=0).stop 2024-03-04T08:47:34.957 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.409+0000 7f59a8b42700 1 -- 172.21.15.71:0/1737312041 shutdown_connections 2024-03-04T08:47:34.957 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.409+0000 7f59a8b42700 1 --2- 172.21.15.71:0/1737312041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59a4106240 0x7f59a419da70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:34.957 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.409+0000 7f59a8b42700 1 -- 172.21.15.71:0/1737312041 >> 172.21.15.71:0/1737312041 conn(0x7f59a4101e80 msgr2=0x7f59a41946b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:34.957 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.409+0000 7f59a8b42700 1 -- 172.21.15.71:0/1737312041 shutdown_connections 2024-03-04T08:47:34.957 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:34.409+0000 7f59a8b42700 1 -- 172.21.15.71:0/1737312041 wait complete. 2024-03-04T08:47:34.957 INFO:teuthology.orchestra.run.smithi071.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-04T08:47:34.979 INFO:teuthology.orchestra.run.smithi071.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-04T08:47:34.979 INFO:teuthology.orchestra.run.smithi071.stdout:Creating mgr... 2024-03-04T08:47:34.979 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-04T08:47:34.980 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-04T08:47:35.224 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mgr.x 2024-03-04T08:47:35.224 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mgr.x.service: Unit ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mgr.x.service not loaded. 2024-03-04T08:47:35.379 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b3c85c40-da03-11ee-95c6-87774f69a715.target.wants/ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-b3c85c40-da03-11ee-95c6-87774f69a715@.service. 2024-03-04T08:47:35.610 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:35 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1737312041' entity='client.admin' 2024-03-04T08:47:35.875 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:35 smithi071 podman[33003]: 2024-03-04 08:47:35.832390492 +0000 UTC m=+0.332503378 container start ba61387fd7a74c3136ce81a40d45ecb0a69bf18a1bd9a8d36b64da68c6ba3452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222) 2024-03-04T08:47:35.889 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-04T08:47:35.889 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout disabled 2024-03-04T08:47:35.896 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-04T08:47:35.897 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-03-04T08:47:35.897 INFO:teuthology.orchestra.run.smithi071.stdout:firewalld.service is not enabled 2024-03-04T08:47:35.897 INFO:teuthology.orchestra.run.smithi071.stdout:Not possible to enable service . firewalld.service is not available 2024-03-04T08:47:35.904 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-04T08:47:35.904 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout disabled 2024-03-04T08:47:35.912 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-04T08:47:35.912 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-03-04T08:47:35.912 INFO:teuthology.orchestra.run.smithi071.stdout:firewalld.service is not enabled 2024-03-04T08:47:35.913 INFO:teuthology.orchestra.run.smithi071.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-04T08:47:35.913 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mgr to start... 2024-03-04T08:47:35.913 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mgr... 2024-03-04T08:47:36.163 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:35 smithi071 bash[33003]: ba61387fd7a74c3136ce81a40d45ecb0a69bf18a1bd9a8d36b64da68c6ba3452 2024-03-04T08:47:36.164 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:35 smithi071 systemd[1]: Started Ceph mgr.x for b3c85c40-da03-11ee-95c6-87774f69a715. 2024-03-04T08:47:36.421 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:36 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:36.387+0000 7f253c5bc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T08:47:36.756 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:36 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3824298686' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-04T08:47:36.756 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:36 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:36.485+0000 7f253c5bc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T08:47:37.051 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-03-04T08:47:37.062 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-03-04T08:47:37.062 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsid": "b3c85c40-da03-11ee-95c6-87774f69a715", 2024-03-04T08:47:37.062 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "health": { 2024-03-04T08:47:37.062 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-04T08:47:37.063 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-04T08:47:37.063 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-04T08:47:37.063 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:37.063 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-04T08:47:37.063 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-04T08:47:37.063 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 0 2024-03-04T08:47:37.063 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-03-04T08:47:37.063 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-04T08:47:37.063 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "a" 2024-03-04T08:47:37.063 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-03-04T08:47:37.063 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-04T08:47:37.063 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-04T08:47:37.063 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T08:47:37.064 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-04T08:47:37.064 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-04T08:47:37.064 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:37.064 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-04T08:47:37.064 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T08:47:37.064 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-04T08:47:37.064 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-04T08:47:37.064 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-04T08:47:37.064 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-04T08:47:37.064 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-04T08:47:37.064 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-04T08:47:37.064 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:37.064 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-04T08:47:37.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-04T08:47:37.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-04T08:47:37.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-04T08:47:37.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-04T08:47:37.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-04T08:47:37.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-04T08:47:37.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-04T08:47:37.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-04T08:47:37.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:37.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-04T08:47:37.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T08:47:37.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-04T08:47:37.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-04T08:47:37.065 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:37.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-04T08:47:37.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-04T08:47:37.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-04T08:47:37.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-04T08:47:37.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-04T08:47:37.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-04T08:47:37.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "restful" 2024-03-04T08:47:37.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-03-04T08:47:37.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T08:47:37.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:37.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-04T08:47:37.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T08:47:37.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modified": "2024-03-04T08:47:28.546614+0000", 2024-03-04T08:47:37.066 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T08:47:37.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:37.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-04T08:47:37.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-03-04T08:47:37.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.374+0000 7f8193bb6700 1 Processor -- start 2024-03-04T08:47:37.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.374+0000 7f8193bb6700 1 -- start start 2024-03-04T08:47:37.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.374+0000 7f8193bb6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f818c105d10 0x7f818c106130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:37.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.374+0000 7f8193bb6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c106700 con 0x7f818c105d10 2024-03-04T08:47:37.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.374+0000 7f8191952700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f818c105d10 0x7f818c106130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:37.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.374+0000 7f8191952700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f818c105d10 0x7f818c106130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55724/0 (socket says 172.21.15.71:55724) 2024-03-04T08:47:37.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.374+0000 7f8191952700 1 -- 172.21.15.71:0/3315109713 learned_addr learned my addr 172.21.15.71:0/3315109713 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:37.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.375+0000 7f8191952700 1 -- 172.21.15.71:0/3315109713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c106880 con 0x7f818c105d10 2024-03-04T08:47:37.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.375+0000 7f8191952700 1 --2- 172.21.15.71:0/3315109713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f818c105d10 0x7f818c106130 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f81880098a0 tx=0x7f8188009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f31f2177378883 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:37.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.375+0000 7f8190950700 1 -- 172.21.15.71:0/3315109713 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8188004160 con 0x7f818c105d10 2024-03-04T08:47:37.067 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.375+0000 7f8190950700 1 -- 172.21.15.71:0/3315109713 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8188004300 con 0x7f818c105d10 2024-03-04T08:47:37.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.375+0000 7f8190950700 1 -- 172.21.15.71:0/3315109713 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8188011b50 con 0x7f818c105d10 2024-03-04T08:47:37.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.376+0000 7f8193bb6700 1 -- 172.21.15.71:0/3315109713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f818c105d10 msgr2=0x7f818c106130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:37.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.376+0000 7f8193bb6700 1 --2- 172.21.15.71:0/3315109713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f818c105d10 0x7f818c106130 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f81880098a0 tx=0x7f8188009bb0 comp rx=0 tx=0).stop 2024-03-04T08:47:37.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.376+0000 7f8193bb6700 1 -- 172.21.15.71:0/3315109713 shutdown_connections 2024-03-04T08:47:37.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.376+0000 7f8193bb6700 1 --2- 172.21.15.71:0/3315109713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f818c105d10 0x7f818c106130 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:37.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.376+0000 7f8193bb6700 1 -- 172.21.15.71:0/3315109713 >> 172.21.15.71:0/3315109713 conn(0x7f818c101690 msgr2=0x7f818c103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:37.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.376+0000 7f8193bb6700 1 -- 172.21.15.71:0/3315109713 shutdown_connections 2024-03-04T08:47:37.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.376+0000 7f8193bb6700 1 -- 172.21.15.71:0/3315109713 wait complete. 2024-03-04T08:47:37.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.377+0000 7f8193bb6700 1 Processor -- start 2024-03-04T08:47:37.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.377+0000 7f8193bb6700 1 -- start start 2024-03-04T08:47:37.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.377+0000 7f8193bb6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f818c105d10 0x7f818c19b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:37.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.377+0000 7f8193bb6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c109be0 con 0x7f818c105d10 2024-03-04T08:47:37.068 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.377+0000 7f8191952700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f818c105d10 0x7f818c19b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:37.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.377+0000 7f8191952700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f818c105d10 0x7f818c19b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55738/0 (socket says 172.21.15.71:55738) 2024-03-04T08:47:37.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.377+0000 7f8191952700 1 -- 172.21.15.71:0/3824298686 learned_addr learned my addr 172.21.15.71:0/3824298686 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:37.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.377+0000 7f8191952700 1 -- 172.21.15.71:0/3824298686 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c19bf20 con 0x7f818c105d10 2024-03-04T08:47:37.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.377+0000 7f8191952700 1 --2- 172.21.15.71:0/3824298686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f818c105d10 0x7f818c19b9e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8188011850 tx=0x7f8188011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:37.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.377+0000 7f8182ffd700 1 -- 172.21.15.71:0/3824298686 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8188003af0 con 0x7f818c105d10 2024-03-04T08:47:37.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.377+0000 7f8182ffd700 1 -- 172.21.15.71:0/3824298686 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8188020070 con 0x7f818c105d10 2024-03-04T08:47:37.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.377+0000 7f8193bb6700 1 -- 172.21.15.71:0/3824298686 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818c19c1b0 con 0x7f818c105d10 2024-03-04T08:47:37.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.377+0000 7f8182ffd700 1 -- 172.21.15.71:0/3824298686 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81880195a0 con 0x7f818c105d10 2024-03-04T08:47:37.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.378+0000 7f8193bb6700 1 -- 172.21.15.71:0/3824298686 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818c19eea0 con 0x7f818c105d10 2024-03-04T08:47:37.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.378+0000 7f8182ffd700 1 -- 172.21.15.71:0/3824298686 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8188019c30 con 0x7f818c105d10 2024-03-04T08:47:37.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.378+0000 7f8182ffd700 1 -- 172.21.15.71:0/3824298686 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8188011e00 con 0x7f818c105d10 2024-03-04T08:47:37.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.378+0000 7f8193bb6700 1 -- 172.21.15.71:0/3824298686 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818c069150 con 0x7f818c105d10 2024-03-04T08:47:37.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.381+0000 7f8182ffd700 1 -- 172.21.15.71:0/3824298686 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8188003c90 con 0x7f818c105d10 2024-03-04T08:47:37.069 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.453+0000 7f8193bb6700 1 -- 172.21.15.71:0/3824298686 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f818c19f3a0 con 0x7f818c105d10 2024-03-04T08:47:37.070 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.453+0000 7f8182ffd700 1 -- 172.21.15.71:0/3824298686 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f818801a430 con 0x7f818c105d10 2024-03-04T08:47:37.070 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.454+0000 7f8193bb6700 1 -- 172.21.15.71:0/3824298686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f818c105d10 msgr2=0x7f818c19b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:37.070 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.454+0000 7f8193bb6700 1 --2- 172.21.15.71:0/3824298686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f818c105d10 0x7f818c19b9e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8188011850 tx=0x7f8188011c10 comp rx=0 tx=0).stop 2024-03-04T08:47:37.070 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.454+0000 7f8193bb6700 1 -- 172.21.15.71:0/3824298686 shutdown_connections 2024-03-04T08:47:37.070 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.454+0000 7f8193bb6700 1 --2- 172.21.15.71:0/3824298686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f818c105d10 0x7f818c19b9e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:37.070 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.454+0000 7f8193bb6700 1 -- 172.21.15.71:0/3824298686 >> 172.21.15.71:0/3824298686 conn(0x7f818c101690 msgr2=0x7f818c102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:37.070 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.454+0000 7f8193bb6700 1 -- 172.21.15.71:0/3824298686 shutdown_connections 2024-03-04T08:47:37.070 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:36.455+0000 7f8193bb6700 1 -- 172.21.15.71:0/3824298686 wait complete. 2024-03-04T08:47:37.070 INFO:teuthology.orchestra.run.smithi071.stdout:mgr not available, waiting (1/15)... 2024-03-04T08:47:37.354 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:37 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:37.131+0000 7f253c5bc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T08:47:37.354 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:37 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:37.220+0000 7f253c5bc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T08:47:37.354 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:37 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:37.358+0000 7f253c5bc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T08:47:37.671 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:37 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:37.453+0000 7f253c5bc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T08:47:37.671 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:37 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:37.533+0000 7f253c5bc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T08:47:38.069 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:37 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:37.790+0000 7f253c5bc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-04T08:47:38.069 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:37 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:37.897+0000 7f253c5bc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T08:47:38.069 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:37 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:37.981+0000 7f253c5bc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T08:47:38.070 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:38 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:38.073+0000 7f253c5bc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T08:47:38.364 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:38 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:38.243+0000 7f253c5bc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T08:47:39.443 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:39 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:39.203+0000 7f253c5bc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-04T08:47:39.443 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:39 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:39.338+0000 7f253c5bc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T08:47:39.761 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:39 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/283671356' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-04T08:47:39.761 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:39 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:39.677+0000 7f253c5bc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T08:47:39.761 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:39 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:39.764+0000 7f253c5bc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T08:47:40.141 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:39 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:39.961+0000 7f253c5bc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T08:47:40.141 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:40 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:40.144+0000 7f253c5bc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T08:47:40.159 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-03-04T08:47:40.159 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-03-04T08:47:40.159 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsid": "b3c85c40-da03-11ee-95c6-87774f69a715", 2024-03-04T08:47:40.159 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "health": { 2024-03-04T08:47:40.159 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-04T08:47:40.159 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-04T08:47:40.159 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-04T08:47:40.159 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:40.159 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-04T08:47:40.159 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-04T08:47:40.159 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 0 2024-03-04T08:47:40.159 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-03-04T08:47:40.160 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-04T08:47:40.160 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "a" 2024-03-04T08:47:40.160 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-03-04T08:47:40.160 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-04T08:47:40.160 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-04T08:47:40.160 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T08:47:40.160 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-04T08:47:40.160 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-04T08:47:40.160 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:40.160 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-04T08:47:40.160 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T08:47:40.160 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-04T08:47:40.160 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-04T08:47:40.160 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-04T08:47:40.168 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-04T08:47:40.168 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-04T08:47:40.168 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-04T08:47:40.168 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:40.169 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-04T08:47:40.169 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-04T08:47:40.169 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-04T08:47:40.169 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-04T08:47:40.169 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-04T08:47:40.169 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-04T08:47:40.169 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-04T08:47:40.169 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-04T08:47:40.169 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-04T08:47:40.169 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:40.169 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-04T08:47:40.169 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T08:47:40.169 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-04T08:47:40.170 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-04T08:47:40.170 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:40.170 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-04T08:47:40.170 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-04T08:47:40.170 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-04T08:47:40.170 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-04T08:47:40.170 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-04T08:47:40.170 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-04T08:47:40.170 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "restful" 2024-03-04T08:47:40.170 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-03-04T08:47:40.170 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T08:47:40.170 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:40.170 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-04T08:47:40.170 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T08:47:40.170 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modified": "2024-03-04T08:47:28.546614+0000", 2024-03-04T08:47:40.171 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T08:47:40.171 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:40.171 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-04T08:47:40.171 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-03-04T08:47:40.171 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.470+0000 7f7cad515700 1 Processor -- start 2024-03-04T08:47:40.171 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.470+0000 7f7cad515700 1 -- start start 2024-03-04T08:47:40.171 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.470+0000 7f7cad515700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ca8107aa0 0x7f7ca8107ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:40.171 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.470+0000 7f7cad515700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca8108490 con 0x7f7ca8107aa0 2024-03-04T08:47:40.171 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.470+0000 7f7ca6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ca8107aa0 0x7f7ca8107ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:40.171 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.470+0000 7f7ca6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ca8107aa0 0x7f7ca8107ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55742/0 (socket says 172.21.15.71:55742) 2024-03-04T08:47:40.171 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.470+0000 7f7ca6ffd700 1 -- 172.21.15.71:0/3499221818 learned_addr learned my addr 172.21.15.71:0/3499221818 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:40.171 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.471+0000 7f7ca6ffd700 1 -- 172.21.15.71:0/3499221818 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca8108ca0 con 0x7f7ca8107aa0 2024-03-04T08:47:40.171 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.471+0000 7f7ca6ffd700 1 --2- 172.21.15.71:0/3499221818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ca8107aa0 0x7f7ca8107ec0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0098a0 tx=0x7f7c9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489195b7f9b3c039 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:40.172 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.471+0000 7f7ca5ffb700 1 -- 172.21.15.71:0/3499221818 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c9c004160 con 0x7f7ca8107aa0 2024-03-04T08:47:40.172 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.471+0000 7f7ca5ffb700 1 -- 172.21.15.71:0/3499221818 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c9c004300 con 0x7f7ca8107aa0 2024-03-04T08:47:40.172 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.471+0000 7f7ca5ffb700 1 -- 172.21.15.71:0/3499221818 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c9c011b50 con 0x7f7ca8107aa0 2024-03-04T08:47:40.172 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.472+0000 7f7cad515700 1 -- 172.21.15.71:0/3499221818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ca8107aa0 msgr2=0x7f7ca8107ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:40.172 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.472+0000 7f7cad515700 1 --2- 172.21.15.71:0/3499221818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ca8107aa0 0x7f7ca8107ec0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0098a0 tx=0x7f7c9c009bb0 comp rx=0 tx=0).stop 2024-03-04T08:47:40.172 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.472+0000 7f7cad515700 1 -- 172.21.15.71:0/3499221818 shutdown_connections 2024-03-04T08:47:40.172 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.472+0000 7f7cad515700 1 --2- 172.21.15.71:0/3499221818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ca8107aa0 0x7f7ca8107ec0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:40.172 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.472+0000 7f7cad515700 1 -- 172.21.15.71:0/3499221818 >> 172.21.15.71:0/3499221818 conn(0x7f7ca81030a0 msgr2=0x7f7ca81054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:40.172 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.472+0000 7f7cad515700 1 -- 172.21.15.71:0/3499221818 shutdown_connections 2024-03-04T08:47:40.172 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.472+0000 7f7cad515700 1 -- 172.21.15.71:0/3499221818 wait complete. 2024-03-04T08:47:40.172 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.472+0000 7f7cad515700 1 Processor -- start 2024-03-04T08:47:40.172 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.473+0000 7f7cad515700 1 -- start start 2024-03-04T08:47:40.172 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.473+0000 7f7cad515700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ca8107aa0 0x7f7ca819ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:40.172 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.473+0000 7f7cad515700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca810bed0 con 0x7f7ca8107aa0 2024-03-04T08:47:40.173 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.473+0000 7f7ca6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ca8107aa0 0x7f7ca819ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:40.173 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.473+0000 7f7ca6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ca8107aa0 0x7f7ca819ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55748/0 (socket says 172.21.15.71:55748) 2024-03-04T08:47:40.173 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.473+0000 7f7ca6ffd700 1 -- 172.21.15.71:0/283671356 learned_addr learned my addr 172.21.15.71:0/283671356 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:40.173 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.473+0000 7f7ca6ffd700 1 -- 172.21.15.71:0/283671356 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca819bf80 con 0x7f7ca8107aa0 2024-03-04T08:47:40.173 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.473+0000 7f7ca6ffd700 1 --2- 172.21.15.71:0/283671356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ca8107aa0 0x7f7ca819ba40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009fb0 tx=0x7f7c9c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:40.173 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.473+0000 7f7c8ffff700 1 -- 172.21.15.71:0/283671356 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c9c0039e0 con 0x7f7ca8107aa0 2024-03-04T08:47:40.173 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.474+0000 7f7cad515700 1 -- 172.21.15.71:0/283671356 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca819c210 con 0x7f7ca8107aa0 2024-03-04T08:47:40.173 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.474+0000 7f7c8ffff700 1 -- 172.21.15.71:0/283671356 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c9c011e50 con 0x7f7ca8107aa0 2024-03-04T08:47:40.173 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.474+0000 7f7c8ffff700 1 -- 172.21.15.71:0/283671356 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c9c023550 con 0x7f7ca8107aa0 2024-03-04T08:47:40.173 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.474+0000 7f7cad515700 1 -- 172.21.15.71:0/283671356 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca819ef00 con 0x7f7ca8107aa0 2024-03-04T08:47:40.173 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.474+0000 7f7c8ffff700 1 -- 172.21.15.71:0/283671356 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7c9c022030 con 0x7f7ca8107aa0 2024-03-04T08:47:40.173 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.474+0000 7f7c8ffff700 1 -- 172.21.15.71:0/283671356 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7c9c0193c0 con 0x7f7ca8107aa0 2024-03-04T08:47:40.173 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.475+0000 7f7cad515700 1 -- 172.21.15.71:0/283671356 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca810c050 con 0x7f7ca8107aa0 2024-03-04T08:47:40.173 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.477+0000 7f7c8ffff700 1 -- 172.21.15.71:0/283671356 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7c9c023d70 con 0x7f7ca8107aa0 2024-03-04T08:47:40.174 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.547+0000 7f7cad515700 1 -- 172.21.15.71:0/283671356 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f7ca819f0d0 con 0x7f7ca8107aa0 2024-03-04T08:47:40.174 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.547+0000 7f7c8ffff700 1 -- 172.21.15.71:0/283671356 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f7c9c003b80 con 0x7f7ca8107aa0 2024-03-04T08:47:40.174 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.548+0000 7f7cad515700 1 -- 172.21.15.71:0/283671356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ca8107aa0 msgr2=0x7f7ca819ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:40.174 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.548+0000 7f7cad515700 1 --2- 172.21.15.71:0/283671356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ca8107aa0 0x7f7ca819ba40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009fb0 tx=0x7f7c9c011c10 comp rx=0 tx=0).stop 2024-03-04T08:47:40.174 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.548+0000 7f7cad515700 1 -- 172.21.15.71:0/283671356 shutdown_connections 2024-03-04T08:47:40.174 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.548+0000 7f7cad515700 1 --2- 172.21.15.71:0/283671356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ca8107aa0 0x7f7ca819ba40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:40.174 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.548+0000 7f7cad515700 1 -- 172.21.15.71:0/283671356 >> 172.21.15.71:0/283671356 conn(0x7f7ca81030a0 msgr2=0x7f7ca8103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:40.174 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.548+0000 7f7cad515700 1 -- 172.21.15.71:0/283671356 shutdown_connections 2024-03-04T08:47:40.174 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:39.548+0000 7f7cad515700 1 -- 172.21.15.71:0/283671356 wait complete. 2024-03-04T08:47:40.174 INFO:teuthology.orchestra.run.smithi071.stdout:mgr not available, waiting (2/15)... 2024-03-04T08:47:40.677 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:40 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:40.426+0000 7f253c5bc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T08:47:40.677 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:40 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:40.681+0000 7f253c5bc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T08:47:41.171 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:40 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:40.915+0000 7f253c5bc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T08:47:42.171 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:41 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:41.710+0000 7f253c5bc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T08:47:42.532 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:42 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:42.274+0000 7f253c5bc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T08:47:42.532 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:42 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:42.373+0000 7f253c5bc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T08:47:42.909 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:42 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:42.595+0000 7f253c5bc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T08:47:42.909 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:42 smithi071 ceph-mon[32789]: Activating manager daemon x 2024-03-04T08:47:42.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:42 smithi071 ceph-mon[32789]: mgrmap e2: x(active, starting, since 0.00349432s) 2024-03-04T08:47:42.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:42 smithi071 ceph-mon[32789]: from='mgr.14100 172.21.15.71:0/3724840919' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T08:47:42.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:42 smithi071 ceph-mon[32789]: from='mgr.14100 172.21.15.71:0/3724840919' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T08:47:42.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:42 smithi071 ceph-mon[32789]: from='mgr.14100 172.21.15.71:0/3724840919' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T08:47:42.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:42 smithi071 ceph-mon[32789]: from='mgr.14100 172.21.15.71:0/3724840919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T08:47:42.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:42 smithi071 ceph-mon[32789]: from='mgr.14100 172.21.15.71:0/3724840919' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-04T08:47:42.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:42 smithi071 ceph-mon[32789]: Manager daemon x is now available 2024-03-04T08:47:42.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:42 smithi071 ceph-mon[32789]: from='mgr.14100 172.21.15.71:0/3724840919' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-04T08:47:42.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:42 smithi071 ceph-mon[32789]: from='mgr.14100 172.21.15.71:0/3724840919' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-04T08:47:42.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:42 smithi071 ceph-mon[32789]: from='mgr.14100 172.21.15.71:0/3724840919' entity='mgr.x' 2024-03-04T08:47:42.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:42 smithi071 ceph-mon[32789]: from='mgr.14100 172.21.15.71:0/3724840919' entity='mgr.x' 2024-03-04T08:47:42.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:42 smithi071 ceph-mon[32789]: from='mgr.14100 172.21.15.71:0/3724840919' entity='mgr.x' 2024-03-04T08:47:43.265 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-03-04T08:47:43.266 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-03-04T08:47:43.266 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsid": "b3c85c40-da03-11ee-95c6-87774f69a715", 2024-03-04T08:47:43.266 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "health": { 2024-03-04T08:47:43.266 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-04T08:47:43.266 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-04T08:47:43.266 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-04T08:47:43.266 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:43.266 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-04T08:47:43.266 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-04T08:47:43.266 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 0 2024-03-04T08:47:43.266 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-03-04T08:47:43.266 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-04T08:47:43.267 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "a" 2024-03-04T08:47:43.267 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-03-04T08:47:43.267 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-03-04T08:47:43.267 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-04T08:47:43.267 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T08:47:43.267 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-04T08:47:43.267 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-04T08:47:43.267 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:43.267 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-04T08:47:43.267 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T08:47:43.267 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-04T08:47:43.267 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-04T08:47:43.267 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-04T08:47:43.267 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-04T08:47:43.268 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-04T08:47:43.268 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-04T08:47:43.268 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:43.268 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-04T08:47:43.268 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-04T08:47:43.268 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-04T08:47:43.268 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-04T08:47:43.268 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-04T08:47:43.268 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-04T08:47:43.268 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-04T08:47:43.268 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-04T08:47:43.268 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-04T08:47:43.268 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:43.268 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-04T08:47:43.269 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T08:47:43.269 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-04T08:47:43.269 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-04T08:47:43.269 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:43.269 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-04T08:47:43.269 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-04T08:47:43.269 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-04T08:47:43.269 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-04T08:47:43.269 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-04T08:47:43.269 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-04T08:47:43.269 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "restful" 2024-03-04T08:47:43.269 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-03-04T08:47:43.269 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T08:47:43.269 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:43.269 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-04T08:47:43.270 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T08:47:43.270 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modified": "2024-03-04T08:47:28.546614+0000", 2024-03-04T08:47:43.270 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T08:47:43.270 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:43.270 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-04T08:47:43.270 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-03-04T08:47:43.270 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.576+0000 7fd06b1c5700 1 Processor -- start 2024-03-04T08:47:43.270 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.576+0000 7fd06b1c5700 1 -- start start 2024-03-04T08:47:43.270 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.576+0000 7fd06b1c5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd064107fe0 0x7fd064108400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:43.270 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.576+0000 7fd06b1c5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0641089d0 con 0x7fd064107fe0 2024-03-04T08:47:43.270 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.576+0000 7fd068f61700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd064107fe0 0x7fd064108400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:43.270 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.576+0000 7fd068f61700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd064107fe0 0x7fd064108400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55758/0 (socket says 172.21.15.71:55758) 2024-03-04T08:47:43.270 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.576+0000 7fd068f61700 1 -- 172.21.15.71:0/286136095 learned_addr learned my addr 172.21.15.71:0/286136095 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:43.270 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.576+0000 7fd068f61700 1 -- 172.21.15.71:0/286136095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd064105ae0 con 0x7fd064107fe0 2024-03-04T08:47:43.271 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.576+0000 7fd068f61700 1 --2- 172.21.15.71:0/286136095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd064107fe0 0x7fd064108400 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd0580098a0 tx=0x7fd058009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a258b33c555096 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:43.271 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.577+0000 7fd0637fe700 1 -- 172.21.15.71:0/286136095 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd058004160 con 0x7fd064107fe0 2024-03-04T08:47:43.271 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.577+0000 7fd0637fe700 1 -- 172.21.15.71:0/286136095 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd058004300 con 0x7fd064107fe0 2024-03-04T08:47:43.271 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.577+0000 7fd0637fe700 1 -- 172.21.15.71:0/286136095 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd058011b50 con 0x7fd064107fe0 2024-03-04T08:47:43.271 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.577+0000 7fd06b1c5700 1 -- 172.21.15.71:0/286136095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd064107fe0 msgr2=0x7fd064108400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:43.271 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.577+0000 7fd06b1c5700 1 --2- 172.21.15.71:0/286136095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd064107fe0 0x7fd064108400 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd0580098a0 tx=0x7fd058009bb0 comp rx=0 tx=0).stop 2024-03-04T08:47:43.271 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.577+0000 7fd06b1c5700 1 -- 172.21.15.71:0/286136095 shutdown_connections 2024-03-04T08:47:43.271 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.577+0000 7fd06b1c5700 1 --2- 172.21.15.71:0/286136095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd064107fe0 0x7fd064108400 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:43.271 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.577+0000 7fd06b1c5700 1 -- 172.21.15.71:0/286136095 >> 172.21.15.71:0/286136095 conn(0x7fd0641016c0 msgr2=0x7fd064103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:43.271 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.577+0000 7fd06b1c5700 1 -- 172.21.15.71:0/286136095 shutdown_connections 2024-03-04T08:47:43.271 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.578+0000 7fd06b1c5700 1 -- 172.21.15.71:0/286136095 wait complete. 2024-03-04T08:47:43.271 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.578+0000 7fd06b1c5700 1 Processor -- start 2024-03-04T08:47:43.271 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.578+0000 7fd06b1c5700 1 -- start start 2024-03-04T08:47:43.271 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.578+0000 7fd06b1c5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd064107fe0 0x7fd06419dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:43.272 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.578+0000 7fd06b1c5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd064106af0 con 0x7fd064107fe0 2024-03-04T08:47:43.272 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.579+0000 7fd068f61700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd064107fe0 0x7fd06419dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:43.272 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.579+0000 7fd068f61700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd064107fe0 0x7fd06419dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55762/0 (socket says 172.21.15.71:55762) 2024-03-04T08:47:43.272 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.579+0000 7fd068f61700 1 -- 172.21.15.71:0/3169851288 learned_addr learned my addr 172.21.15.71:0/3169851288 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:43.272 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.579+0000 7fd068f61700 1 -- 172.21.15.71:0/3169851288 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06419e110 con 0x7fd064107fe0 2024-03-04T08:47:43.272 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.579+0000 7fd068f61700 1 --2- 172.21.15.71:0/3169851288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd064107fe0 0x7fd06419dbd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd058009fb0 tx=0x7fd058011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:43.272 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.579+0000 7fd061ffb700 1 -- 172.21.15.71:0/3169851288 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0580038b0 con 0x7fd064107fe0 2024-03-04T08:47:43.272 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.579+0000 7fd061ffb700 1 -- 172.21.15.71:0/3169851288 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd058011e50 con 0x7fd064107fe0 2024-03-04T08:47:43.272 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.579+0000 7fd06b1c5700 1 -- 172.21.15.71:0/3169851288 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd06419e3a0 con 0x7fd064107fe0 2024-03-04T08:47:43.272 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.579+0000 7fd061ffb700 1 -- 172.21.15.71:0/3169851288 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd058019420 con 0x7fd064107fe0 2024-03-04T08:47:43.272 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.579+0000 7fd06b1c5700 1 -- 172.21.15.71:0/3169851288 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0641a1090 con 0x7fd064107fe0 2024-03-04T08:47:43.272 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.580+0000 7fd061ffb700 1 -- 172.21.15.71:0/3169851288 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd058019a60 con 0x7fd064107fe0 2024-03-04T08:47:43.272 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.580+0000 7fd061ffb700 1 -- 172.21.15.71:0/3169851288 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd058024730 con 0x7fd064107fe0 2024-03-04T08:47:43.273 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.580+0000 7fd06b1c5700 1 -- 172.21.15.71:0/3169851288 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd064069150 con 0x7fd064107fe0 2024-03-04T08:47:43.273 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.582+0000 7fd061ffb700 1 -- 172.21.15.71:0/3169851288 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd058043dd0 con 0x7fd064107fe0 2024-03-04T08:47:43.273 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.611+0000 7fd061ffb700 1 -- 172.21.15.71:0/3169851288 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 2) v1 ==== 45684+0+0 (secure 0 0 0) 0x7fd058003a50 con 0x7fd064107fe0 2024-03-04T08:47:43.273 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.661+0000 7fd06b1c5700 1 -- 172.21.15.71:0/3169851288 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fd0641971d0 con 0x7fd064107fe0 2024-03-04T08:47:43.273 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.661+0000 7fd061ffb700 1 -- 172.21.15.71:0/3169851288 <== mon.0 v2:172.21.15.71:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fd05802b340 con 0x7fd064107fe0 2024-03-04T08:47:43.273 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.662+0000 7fd06b1c5700 1 -- 172.21.15.71:0/3169851288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd064107fe0 msgr2=0x7fd06419dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:43.273 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.662+0000 7fd06b1c5700 1 --2- 172.21.15.71:0/3169851288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd064107fe0 0x7fd06419dbd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd058009fb0 tx=0x7fd058011bf0 comp rx=0 tx=0).stop 2024-03-04T08:47:43.273 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.662+0000 7fd06b1c5700 1 -- 172.21.15.71:0/3169851288 shutdown_connections 2024-03-04T08:47:43.273 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.662+0000 7fd06b1c5700 1 --2- 172.21.15.71:0/3169851288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd064107fe0 0x7fd06419dbd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:43.273 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.662+0000 7fd06b1c5700 1 -- 172.21.15.71:0/3169851288 >> 172.21.15.71:0/3169851288 conn(0x7fd0641016c0 msgr2=0x7fd064194290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:43.273 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.662+0000 7fd06b1c5700 1 -- 172.21.15.71:0/3169851288 shutdown_connections 2024-03-04T08:47:43.273 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:42.662+0000 7fd06b1c5700 1 -- 172.21.15.71:0/3169851288 wait complete. 2024-03-04T08:47:43.273 INFO:teuthology.orchestra.run.smithi071.stdout:mgr not available, waiting (3/15)... 2024-03-04T08:47:43.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:43 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3169851288' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-04T08:47:43.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:43 smithi071 ceph-mon[32789]: mgrmap e3: x(active, since 1.00706s) 2024-03-04T08:47:45.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:45 smithi071 ceph-mon[32789]: mgrmap e4: x(active, since 2s) 2024-03-04T08:47:46.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-03-04T08:47:46.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-03-04T08:47:46.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsid": "b3c85c40-da03-11ee-95c6-87774f69a715", 2024-03-04T08:47:46.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "health": { 2024-03-04T08:47:46.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-04T08:47:46.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-04T08:47:46.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-04T08:47:46.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:46.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-04T08:47:46.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-04T08:47:46.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 0 2024-03-04T08:47:46.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-03-04T08:47:46.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-04T08:47:46.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "a" 2024-03-04T08:47:46.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-03-04T08:47:46.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-03-04T08:47:46.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-04T08:47:46.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T08:47:46.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-04T08:47:46.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-04T08:47:46.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:46.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-04T08:47:46.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T08:47:46.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-04T08:47:46.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-04T08:47:46.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-04T08:47:46.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-04T08:47:46.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-04T08:47:46.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-04T08:47:46.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:46.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-04T08:47:46.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-04T08:47:46.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-04T08:47:46.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-04T08:47:46.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-04T08:47:46.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-04T08:47:46.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-04T08:47:46.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-04T08:47:46.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-04T08:47:46.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:46.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-04T08:47:46.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T08:47:46.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-04T08:47:46.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-04T08:47:46.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:46.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-04T08:47:46.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-04T08:47:46.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-04T08:47:46.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-04T08:47:46.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-04T08:47:46.517 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-04T08:47:46.517 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "restful" 2024-03-04T08:47:46.517 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-03-04T08:47:46.517 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T08:47:46.517 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:46.517 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-04T08:47:46.517 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T08:47:46.517 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modified": "2024-03-04T08:47:28.546614+0000", 2024-03-04T08:47:46.517 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T08:47:46.517 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-03-04T08:47:46.517 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-04T08:47:46.517 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-03-04T08:47:46.517 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.694+0000 7fe63f91d700 1 Processor -- start 2024-03-04T08:47:46.517 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.694+0000 7fe63f91d700 1 -- start start 2024-03-04T08:47:46.518 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.695+0000 7fe63f91d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6381062b0 0x7fe6381066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:46.518 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.695+0000 7fe63f91d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63807ae20 con 0x7fe6381062b0 2024-03-04T08:47:46.518 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.695+0000 7fe63d6b9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6381062b0 0x7fe6381066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:46.518 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.695+0000 7fe63d6b9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6381062b0 0x7fe6381066d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43132/0 (socket says 172.21.15.71:43132) 2024-03-04T08:47:46.518 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.695+0000 7fe63d6b9700 1 -- 172.21.15.71:0/259132230 learned_addr learned my addr 172.21.15.71:0/259132230 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:46.518 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.695+0000 7fe63d6b9700 1 -- 172.21.15.71:0/259132230 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe638106c10 con 0x7fe6381062b0 2024-03-04T08:47:46.518 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.695+0000 7fe63d6b9700 1 --2- 172.21.15.71:0/259132230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6381062b0 0x7fe6381066d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe6280098a0 tx=0x7fe628009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a3f093d37fa9ea server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:46.518 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.695+0000 7fe62ffff700 1 -- 172.21.15.71:0/259132230 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe628004160 con 0x7fe6381062b0 2024-03-04T08:47:46.518 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.695+0000 7fe62ffff700 1 -- 172.21.15.71:0/259132230 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe628004300 con 0x7fe6381062b0 2024-03-04T08:47:46.518 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.695+0000 7fe62ffff700 1 -- 172.21.15.71:0/259132230 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe628011b50 con 0x7fe6381062b0 2024-03-04T08:47:46.518 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.696+0000 7fe63f91d700 1 -- 172.21.15.71:0/259132230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6381062b0 msgr2=0x7fe6381066d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:46.518 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.696+0000 7fe63f91d700 1 --2- 172.21.15.71:0/259132230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6381062b0 0x7fe6381066d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe6280098a0 tx=0x7fe628009bb0 comp rx=0 tx=0).stop 2024-03-04T08:47:46.518 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.696+0000 7fe63f91d700 1 -- 172.21.15.71:0/259132230 shutdown_connections 2024-03-04T08:47:46.518 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.696+0000 7fe63f91d700 1 --2- 172.21.15.71:0/259132230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6381062b0 0x7fe6381066d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:46.519 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.696+0000 7fe63f91d700 1 -- 172.21.15.71:0/259132230 >> 172.21.15.71:0/259132230 conn(0x7fe638101f50 msgr2=0x7fe638104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:46.519 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.696+0000 7fe63f91d700 1 -- 172.21.15.71:0/259132230 shutdown_connections 2024-03-04T08:47:46.519 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.696+0000 7fe63f91d700 1 -- 172.21.15.71:0/259132230 wait complete. 2024-03-04T08:47:46.519 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.697+0000 7fe63f91d700 1 Processor -- start 2024-03-04T08:47:46.519 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.697+0000 7fe63f91d700 1 -- start start 2024-03-04T08:47:46.519 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.697+0000 7fe63f91d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6381062b0 0x7fe63819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:46.519 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.697+0000 7fe63f91d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe638107c30 con 0x7fe6381062b0 2024-03-04T08:47:46.519 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.697+0000 7fe63d6b9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6381062b0 0x7fe63819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:46.519 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.697+0000 7fe63d6b9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6381062b0 0x7fe63819db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43136/0 (socket says 172.21.15.71:43136) 2024-03-04T08:47:46.519 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.697+0000 7fe63d6b9700 1 -- 172.21.15.71:0/3450223526 learned_addr learned my addr 172.21.15.71:0/3450223526 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:46.519 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.697+0000 7fe63d6b9700 1 -- 172.21.15.71:0/3450223526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63819e0b0 con 0x7fe6381062b0 2024-03-04T08:47:46.519 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.697+0000 7fe63d6b9700 1 --2- 172.21.15.71:0/3450223526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6381062b0 0x7fe63819db70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe628009fb0 tx=0x7fe628011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:46.519 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.697+0000 7fe62e7fc700 1 -- 172.21.15.71:0/3450223526 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6280038b0 con 0x7fe6381062b0 2024-03-04T08:47:46.519 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.698+0000 7fe63f91d700 1 -- 172.21.15.71:0/3450223526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe63819e340 con 0x7fe6381062b0 2024-03-04T08:47:46.519 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.698+0000 7fe62e7fc700 1 -- 172.21.15.71:0/3450223526 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe628011e50 con 0x7fe6381062b0 2024-03-04T08:47:46.520 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.698+0000 7fe62e7fc700 1 -- 172.21.15.71:0/3450223526 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe628019420 con 0x7fe6381062b0 2024-03-04T08:47:46.520 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.698+0000 7fe63f91d700 1 -- 172.21.15.71:0/3450223526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6381a1030 con 0x7fe6381062b0 2024-03-04T08:47:46.520 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.698+0000 7fe62e7fc700 1 -- 172.21.15.71:0/3450223526 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 4) v1 ==== 45917+0+0 (secure 0 0 0) 0x7fe62801a430 con 0x7fe6381062b0 2024-03-04T08:47:46.520 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.699+0000 7fe63f91d700 1 -- 172.21.15.71:0/3450223526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe638069150 con 0x7fe6381062b0 2024-03-04T08:47:46.520 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.699+0000 7fe62e7fc700 1 --2- 172.21.15.71:0/3450223526 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7fe624038b00 0x7fe62403afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:46.520 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.699+0000 7fe62e7fc700 1 -- 172.21.15.71:0/3450223526 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe62804cc70 con 0x7fe6381062b0 2024-03-04T08:47:46.520 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.699+0000 7fe63ceb8700 1 --2- 172.21.15.71:0/3450223526 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7fe624038b00 0x7fe62403afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:46.520 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.699+0000 7fe63ceb8700 1 --2- 172.21.15.71:0/3450223526 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7fe624038b00 0x7fe62403afc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe634006fd0 tx=0x7fe634006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:46.520 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.704+0000 7fe62e7fc700 1 -- 172.21.15.71:0/3450223526 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe628022c10 con 0x7fe6381062b0 2024-03-04T08:47:46.520 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.976+0000 7fe63f91d700 1 -- 172.21.15.71:0/3450223526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fe63802dbf0 con 0x7fe6381062b0 2024-03-04T08:47:46.520 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.976+0000 7fe62e7fc700 1 -- 172.21.15.71:0/3450223526 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fe62804c690 con 0x7fe6381062b0 2024-03-04T08:47:46.521 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.978+0000 7fe63f91d700 1 -- 172.21.15.71:0/3450223526 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7fe624038b00 msgr2=0x7fe62403afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:46.521 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.978+0000 7fe63f91d700 1 --2- 172.21.15.71:0/3450223526 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7fe624038b00 0x7fe62403afc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe634006fd0 tx=0x7fe634006e40 comp rx=0 tx=0).stop 2024-03-04T08:47:46.521 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.978+0000 7fe63f91d700 1 -- 172.21.15.71:0/3450223526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6381062b0 msgr2=0x7fe63819db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:46.521 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.978+0000 7fe63f91d700 1 --2- 172.21.15.71:0/3450223526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6381062b0 0x7fe63819db70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe628009fb0 tx=0x7fe628011bf0 comp rx=0 tx=0).stop 2024-03-04T08:47:46.521 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.978+0000 7fe63f91d700 1 -- 172.21.15.71:0/3450223526 shutdown_connections 2024-03-04T08:47:46.521 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.978+0000 7fe63f91d700 1 --2- 172.21.15.71:0/3450223526 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7fe624038b00 0x7fe62403afc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:46.521 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.978+0000 7fe63f91d700 1 --2- 172.21.15.71:0/3450223526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6381062b0 0x7fe63819db70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:46.521 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.978+0000 7fe63f91d700 1 -- 172.21.15.71:0/3450223526 >> 172.21.15.71:0/3450223526 conn(0x7fe638101f50 msgr2=0x7fe638194810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:46.521 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.978+0000 7fe63f91d700 1 -- 172.21.15.71:0/3450223526 shutdown_connections 2024-03-04T08:47:46.521 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:45.978+0000 7fe63f91d700 1 -- 172.21.15.71:0/3450223526 wait complete. 2024-03-04T08:47:46.521 INFO:teuthology.orchestra.run.smithi071.stdout:mgr is available 2024-03-04T08:47:46.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:46 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3450223526' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-04T08:47:47.673 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-03-04T08:47:47.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [global] 2024-03-04T08:47:47.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout fsid = b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T08:47:47.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-04T08:47:47.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-04T08:47:47.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-04T08:47:47.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-04T08:47:47.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-03-04T08:47:47.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-04T08:47:47.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-04T08:47:47.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-03-04T08:47:47.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [osd] 2024-03-04T08:47:47.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-04T08:47:47.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-04T08:47:47.675 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-04T08:47:47.675 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-04T08:47:47.675 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-04T08:47:47.675 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.929+0000 7f8f020ef700 1 Processor -- start 2024-03-04T08:47:47.675 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.929+0000 7f8f020ef700 1 -- start start 2024-03-04T08:47:47.675 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.929+0000 7f8f020ef700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8efc108540 0x7f8efc10a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:47.675 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.929+0000 7f8f020ef700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc07ade0 con 0x7f8efc108540 2024-03-04T08:47:47.675 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.929+0000 7f8efb7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8efc108540 0x7f8efc10a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:47.675 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.929+0000 7f8efb7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8efc108540 0x7f8efc10a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43152/0 (socket says 172.21.15.71:43152) 2024-03-04T08:47:47.675 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.929+0000 7f8efb7fe700 1 -- 172.21.15.71:0/2423346038 learned_addr learned my addr 172.21.15.71:0/2423346038 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:47.675 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.930+0000 7f8efb7fe700 1 -- 172.21.15.71:0/2423346038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8efc10ae70 con 0x7f8efc108540 2024-03-04T08:47:47.675 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.930+0000 7f8efb7fe700 1 --2- 172.21.15.71:0/2423346038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8efc108540 0x7f8efc10a930 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0098a0 tx=0x7f8eec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5557f6f4aece1ff6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:47.675 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.930+0000 7f8efa7fc700 1 -- 172.21.15.71:0/2423346038 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8eec004160 con 0x7f8efc108540 2024-03-04T08:47:47.675 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.930+0000 7f8efa7fc700 1 -- 172.21.15.71:0/2423346038 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8eec004300 con 0x7f8efc108540 2024-03-04T08:47:47.676 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.930+0000 7f8efa7fc700 1 -- 172.21.15.71:0/2423346038 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8eec011b50 con 0x7f8efc108540 2024-03-04T08:47:47.676 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.931+0000 7f8f020ef700 1 -- 172.21.15.71:0/2423346038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8efc108540 msgr2=0x7f8efc10a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:47.676 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.931+0000 7f8f020ef700 1 --2- 172.21.15.71:0/2423346038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8efc108540 0x7f8efc10a930 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0098a0 tx=0x7f8eec009bb0 comp rx=0 tx=0).stop 2024-03-04T08:47:47.676 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.931+0000 7f8f020ef700 1 -- 172.21.15.71:0/2423346038 shutdown_connections 2024-03-04T08:47:47.676 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.931+0000 7f8f020ef700 1 --2- 172.21.15.71:0/2423346038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8efc108540 0x7f8efc10a930 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:47.676 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.931+0000 7f8f020ef700 1 -- 172.21.15.71:0/2423346038 >> 172.21.15.71:0/2423346038 conn(0x7f8efc101fb0 msgr2=0x7f8efc1043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:47.676 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.931+0000 7f8f020ef700 1 -- 172.21.15.71:0/2423346038 shutdown_connections 2024-03-04T08:47:47.676 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.931+0000 7f8f020ef700 1 -- 172.21.15.71:0/2423346038 wait complete. 2024-03-04T08:47:47.676 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.931+0000 7f8f020ef700 1 Processor -- start 2024-03-04T08:47:47.676 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.931+0000 7f8f020ef700 1 -- start start 2024-03-04T08:47:47.676 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.931+0000 7f8f020ef700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8efc108540 0x7f8efc19b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:47.676 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.931+0000 7f8f020ef700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc10bd70 con 0x7f8efc108540 2024-03-04T08:47:47.676 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.932+0000 7f8efb7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8efc108540 0x7f8efc19b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:47.676 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.932+0000 7f8efb7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8efc108540 0x7f8efc19b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43160/0 (socket says 172.21.15.71:43160) 2024-03-04T08:47:47.677 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.932+0000 7f8efb7fe700 1 -- 172.21.15.71:0/1119673696 learned_addr learned my addr 172.21.15.71:0/1119673696 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:47.677 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.932+0000 7f8efb7fe700 1 -- 172.21.15.71:0/1119673696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8efc19beb0 con 0x7f8efc108540 2024-03-04T08:47:47.677 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.932+0000 7f8efb7fe700 1 --2- 172.21.15.71:0/1119673696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8efc108540 0x7f8efc19b970 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009fb0 tx=0x7f8eec011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:47.677 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.932+0000 7f8ef8ff9700 1 -- 172.21.15.71:0/1119673696 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8eec0038b0 con 0x7f8efc108540 2024-03-04T08:47:47.677 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.932+0000 7f8ef8ff9700 1 -- 172.21.15.71:0/1119673696 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8eec011e50 con 0x7f8efc108540 2024-03-04T08:47:47.677 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.932+0000 7f8f020ef700 1 -- 172.21.15.71:0/1119673696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8efc19c140 con 0x7f8efc108540 2024-03-04T08:47:47.677 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.932+0000 7f8ef8ff9700 1 -- 172.21.15.71:0/1119673696 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8eec019420 con 0x7f8efc108540 2024-03-04T08:47:47.677 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.932+0000 7f8f020ef700 1 -- 172.21.15.71:0/1119673696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8efc19ee30 con 0x7f8efc108540 2024-03-04T08:47:47.677 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.933+0000 7f8ef8ff9700 1 -- 172.21.15.71:0/1119673696 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 4) v1 ==== 45917+0+0 (secure 0 0 0) 0x7f8eec019720 con 0x7f8efc108540 2024-03-04T08:47:47.677 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.933+0000 7f8f020ef700 1 -- 172.21.15.71:0/1119673696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8efc069150 con 0x7f8efc108540 2024-03-04T08:47:47.677 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.933+0000 7f8ef8ff9700 1 --2- 172.21.15.71:0/1119673696 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f8ee4038b00 0x7f8ee403afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:47.677 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.933+0000 7f8ef8ff9700 1 -- 172.21.15.71:0/1119673696 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8eec04ccf0 con 0x7f8efc108540 2024-03-04T08:47:47.677 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.933+0000 7f8efaffd700 1 --2- 172.21.15.71:0/1119673696 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f8ee4038b00 0x7f8ee403afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:47.677 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.934+0000 7f8efaffd700 1 --2- 172.21.15.71:0/1119673696 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f8ee4038b00 0x7f8ee403afc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0006fd0 tx=0x7f8ef0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:47.678 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:46.938+0000 7f8ef8ff9700 1 -- 172.21.15.71:0/1119673696 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8eec00bc70 con 0x7f8efc108540 2024-03-04T08:47:47.678 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:47.131+0000 7f8f020ef700 1 -- 172.21.15.71:0/1119673696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f8efc19f000 con 0x7f8efc108540 2024-03-04T08:47:47.678 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:47.132+0000 7f8ef8ff9700 1 -- 172.21.15.71:0/1119673696 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f8eec04c710 con 0x7f8efc108540 2024-03-04T08:47:47.678 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:47.134+0000 7f8f020ef700 1 -- 172.21.15.71:0/1119673696 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f8ee4038b00 msgr2=0x7f8ee403afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:47.678 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:47.134+0000 7f8f020ef700 1 --2- 172.21.15.71:0/1119673696 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f8ee4038b00 0x7f8ee403afc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0006fd0 tx=0x7f8ef0006e40 comp rx=0 tx=0).stop 2024-03-04T08:47:47.678 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:47.134+0000 7f8f020ef700 1 -- 172.21.15.71:0/1119673696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8efc108540 msgr2=0x7f8efc19b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:47.678 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:47.134+0000 7f8f020ef700 1 --2- 172.21.15.71:0/1119673696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8efc108540 0x7f8efc19b970 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009fb0 tx=0x7f8eec011bf0 comp rx=0 tx=0).stop 2024-03-04T08:47:47.678 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:47.134+0000 7f8f020ef700 1 -- 172.21.15.71:0/1119673696 shutdown_connections 2024-03-04T08:47:47.678 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:47.134+0000 7f8f020ef700 1 --2- 172.21.15.71:0/1119673696 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f8ee4038b00 0x7f8ee403afc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:47.678 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:47.134+0000 7f8f020ef700 1 --2- 172.21.15.71:0/1119673696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8efc108540 0x7f8efc19b970 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:47.678 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:47.134+0000 7f8f020ef700 1 -- 172.21.15.71:0/1119673696 >> 172.21.15.71:0/1119673696 conn(0x7f8efc101fb0 msgr2=0x7f8efc102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:47.678 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:47.134+0000 7f8f020ef700 1 -- 172.21.15.71:0/1119673696 shutdown_connections 2024-03-04T08:47:47.678 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:47.135+0000 7f8f020ef700 1 -- 172.21.15.71:0/1119673696 wait complete. 2024-03-04T08:47:47.678 INFO:teuthology.orchestra.run.smithi071.stdout:Enabling cephadm module... 2024-03-04T08:47:47.782 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:47 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1119673696' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-04T08:47:48.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:48 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3958458436' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-04T08:47:48.922 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:48 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: ignoring --setuser ceph since I am not root 2024-03-04T08:47:48.922 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:48 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: ignoring --setgroup ceph since I am not root 2024-03-04T08:47:49.246 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.081+0000 7f40024f6700 1 Processor -- start 2024-03-04T08:47:49.246 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.081+0000 7f40024f6700 1 -- start start 2024-03-04T08:47:49.247 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.081+0000 7f40024f6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ffc106e40 0x7f3ffc107260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:49.247 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.081+0000 7f40024f6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc107830 con 0x7f3ffc106e40 2024-03-04T08:47:49.247 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.081+0000 7f3ffbfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ffc106e40 0x7f3ffc107260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:49.247 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.081+0000 7f3ffbfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ffc106e40 0x7f3ffc107260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43166/0 (socket says 172.21.15.71:43166) 2024-03-04T08:47:49.247 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.081+0000 7f3ffbfff700 1 -- 172.21.15.71:0/329070494 learned_addr learned my addr 172.21.15.71:0/329070494 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:49.247 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.082+0000 7f3ffbfff700 1 -- 172.21.15.71:0/329070494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ffc107b60 con 0x7f3ffc106e40 2024-03-04T08:47:49.247 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.082+0000 7f3ffbfff700 1 --2- 172.21.15.71:0/329070494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ffc106e40 0x7f3ffc107260 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0098a0 tx=0x7f3fec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87ce2673bf732d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:49.247 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.082+0000 7f3ffaffd700 1 -- 172.21.15.71:0/329070494 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fec004160 con 0x7f3ffc106e40 2024-03-04T08:47:49.247 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.082+0000 7f3ffaffd700 1 -- 172.21.15.71:0/329070494 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3fec004300 con 0x7f3ffc106e40 2024-03-04T08:47:49.247 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.082+0000 7f3ffaffd700 1 -- 172.21.15.71:0/329070494 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fec011b50 con 0x7f3ffc106e40 2024-03-04T08:47:49.247 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.082+0000 7f40024f6700 1 -- 172.21.15.71:0/329070494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ffc106e40 msgr2=0x7f3ffc107260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:49.247 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.082+0000 7f40024f6700 1 --2- 172.21.15.71:0/329070494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ffc106e40 0x7f3ffc107260 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0098a0 tx=0x7f3fec009bb0 comp rx=0 tx=0).stop 2024-03-04T08:47:49.247 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.082+0000 7f40024f6700 1 -- 172.21.15.71:0/329070494 shutdown_connections 2024-03-04T08:47:49.248 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.082+0000 7f40024f6700 1 --2- 172.21.15.71:0/329070494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ffc106e40 0x7f3ffc107260 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:49.248 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.083+0000 7f40024f6700 1 -- 172.21.15.71:0/329070494 >> 172.21.15.71:0/329070494 conn(0x7f3ffc1016a0 msgr2=0x7f3ffc103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:49.248 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.083+0000 7f40024f6700 1 -- 172.21.15.71:0/329070494 shutdown_connections 2024-03-04T08:47:49.248 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.083+0000 7f40024f6700 1 -- 172.21.15.71:0/329070494 wait complete. 2024-03-04T08:47:49.248 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.083+0000 7f40024f6700 1 Processor -- start 2024-03-04T08:47:49.248 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.083+0000 7f40024f6700 1 -- start start 2024-03-04T08:47:49.248 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.083+0000 7f40024f6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ffc106e40 0x7f3ffc19ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:49.248 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.083+0000 7f40024f6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ffc10af10 con 0x7f3ffc106e40 2024-03-04T08:47:49.248 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.083+0000 7f3ffbfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ffc106e40 0x7f3ffc19ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:49.248 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.084+0000 7f3ffbfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ffc106e40 0x7f3ffc19ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43172/0 (socket says 172.21.15.71:43172) 2024-03-04T08:47:49.248 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.084+0000 7f3ffbfff700 1 -- 172.21.15.71:0/3958458436 learned_addr learned my addr 172.21.15.71:0/3958458436 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:49.248 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.084+0000 7f3ffbfff700 1 -- 172.21.15.71:0/3958458436 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ffc19bf40 con 0x7f3ffc106e40 2024-03-04T08:47:49.248 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.084+0000 7f3ffbfff700 1 --2- 172.21.15.71:0/3958458436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ffc106e40 0x7f3ffc19ba00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3fec011b60 tx=0x7f3fec011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:49.249 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.084+0000 7f3ff97fa700 1 -- 172.21.15.71:0/3958458436 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fec003970 con 0x7f3ffc106e40 2024-03-04T08:47:49.249 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.084+0000 7f3ff97fa700 1 -- 172.21.15.71:0/3958458436 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3fec003b10 con 0x7f3ffc106e40 2024-03-04T08:47:49.249 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.084+0000 7f40024f6700 1 -- 172.21.15.71:0/3958458436 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ffc19c1d0 con 0x7f3ffc106e40 2024-03-04T08:47:49.249 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.084+0000 7f3ff97fa700 1 -- 172.21.15.71:0/3958458436 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fec024610 con 0x7f3ffc106e40 2024-03-04T08:47:49.249 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.084+0000 7f40024f6700 1 -- 172.21.15.71:0/3958458436 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ffc19eec0 con 0x7f3ffc106e40 2024-03-04T08:47:49.249 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.085+0000 7f3ff97fa700 1 -- 172.21.15.71:0/3958458436 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 4) v1 ==== 45917+0+0 (secure 0 0 0) 0x7f3fec024910 con 0x7f3ffc106e40 2024-03-04T08:47:49.249 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.085+0000 7f3ff97fa700 1 --2- 172.21.15.71:0/3958458436 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f3fe4038b50 0x7f3fe403b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:49.249 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.085+0000 7f3ff97fa700 1 -- 172.21.15.71:0/3958458436 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3fec011e00 con 0x7f3ffc106e40 2024-03-04T08:47:49.249 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.085+0000 7f40024f6700 1 -- 172.21.15.71:0/3958458436 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ffc0636f0 con 0x7f3ffc106e40 2024-03-04T08:47:49.249 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.085+0000 7f3ffb7fe700 1 --2- 172.21.15.71:0/3958458436 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f3fe4038b50 0x7f3fe403b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:49.249 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.085+0000 7f3ffb7fe700 1 --2- 172.21.15.71:0/3958458436 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f3fe4038b50 0x7f3fe403b010 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0006fd0 tx=0x7f3ff0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:49.249 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.090+0000 7f3ff97fa700 1 -- 172.21.15.71:0/3958458436 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3fec04c5c0 con 0x7f3ffc106e40 2024-03-04T08:47:49.249 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.335+0000 7f40024f6700 1 -- 172.21.15.71:0/3958458436 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f3ffc02dbe0 con 0x7f3ffc106e40 2024-03-04T08:47:49.249 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.669+0000 7f3ff97fa700 1 -- 172.21.15.71:0/3958458436 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f3fec00bc40 con 0x7f3ffc106e40 2024-03-04T08:47:49.250 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.670+0000 7f3ff97fa700 1 -- 172.21.15.71:0/3958458436 <== mon.0 v2:172.21.15.71:3300/0 8 ==== mgrmap(e 5) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f3fec024c00 con 0x7f3ffc106e40 2024-03-04T08:47:49.250 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.671+0000 7f40024f6700 1 -- 172.21.15.71:0/3958458436 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f3fe4038b50 msgr2=0x7f3fe403b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:49.250 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.671+0000 7f40024f6700 1 --2- 172.21.15.71:0/3958458436 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f3fe4038b50 0x7f3fe403b010 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0006fd0 tx=0x7f3ff0006e40 comp rx=0 tx=0).stop 2024-03-04T08:47:49.250 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.671+0000 7f40024f6700 1 -- 172.21.15.71:0/3958458436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ffc106e40 msgr2=0x7f3ffc19ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:49.250 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.671+0000 7f40024f6700 1 --2- 172.21.15.71:0/3958458436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ffc106e40 0x7f3ffc19ba00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3fec011b60 tx=0x7f3fec011c40 comp rx=0 tx=0).stop 2024-03-04T08:47:49.250 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.671+0000 7f40024f6700 1 -- 172.21.15.71:0/3958458436 shutdown_connections 2024-03-04T08:47:49.250 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.671+0000 7f40024f6700 1 --2- 172.21.15.71:0/3958458436 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f3fe4038b50 0x7f3fe403b010 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:49.250 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.671+0000 7f40024f6700 1 --2- 172.21.15.71:0/3958458436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ffc106e40 0x7f3ffc19ba00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:49.250 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.671+0000 7f40024f6700 1 -- 172.21.15.71:0/3958458436 >> 172.21.15.71:0/3958458436 conn(0x7f3ffc1016a0 msgr2=0x7f3ffc102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:49.250 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.672+0000 7f40024f6700 1 -- 172.21.15.71:0/3958458436 shutdown_connections 2024-03-04T08:47:49.250 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:48.672+0000 7f40024f6700 1 -- 172.21.15.71:0/3958458436 wait complete. 2024-03-04T08:47:49.321 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:49 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:49.219+0000 7feea1368200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T08:47:49.321 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:49 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:49.324+0000 7feea1368200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T08:47:49.878 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:49 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3958458436' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-04T08:47:49.878 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:49 smithi071 ceph-mon[32789]: mgrmap e5: x(active, since 6s) 2024-03-04T08:47:50.171 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:49 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:49.991+0000 7feea1368200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T08:47:50.172 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:50 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:50.085+0000 7feea1368200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T08:47:50.595 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:50 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:50.227+0000 7feea1368200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T08:47:50.595 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:50 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:50.322+0000 7feea1368200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T08:47:50.595 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:50 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:50.406+0000 7feea1368200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T08:47:50.603 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-03-04T08:47:50.603 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-04T08:47:50.603 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-04T08:47:50.603 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-04T08:47:50.603 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-04T08:47:50.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-03-04T08:47:50.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.676+0000 7fe01f582700 1 Processor -- start 2024-03-04T08:47:50.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.676+0000 7fe01f582700 1 -- start start 2024-03-04T08:47:50.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.676+0000 7fe01f582700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe018106910 0x7fe018106d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:50.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.676+0000 7fe01f582700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe018107300 con 0x7fe018106910 2024-03-04T08:47:50.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.677+0000 7fe01d31e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe018106910 0x7fe018106d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:50.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.677+0000 7fe01d31e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe018106910 0x7fe018106d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43206/0 (socket says 172.21.15.71:43206) 2024-03-04T08:47:50.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.677+0000 7fe01d31e700 1 -- 172.21.15.71:0/1530581841 learned_addr learned my addr 172.21.15.71:0/1530581841 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:50.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.677+0000 7fe01d31e700 1 -- 172.21.15.71:0/1530581841 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe018107b10 con 0x7fe018106910 2024-03-04T08:47:50.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.677+0000 7fe01d31e700 1 --2- 172.21.15.71:0/1530581841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe018106910 0x7fe018106d30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe0140098a0 tx=0x7fe014009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ed9979a438d3d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:50.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.677+0000 7fe00ffff700 1 -- 172.21.15.71:0/1530581841 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe014004160 con 0x7fe018106910 2024-03-04T08:47:50.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.677+0000 7fe00ffff700 1 -- 172.21.15.71:0/1530581841 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe014004300 con 0x7fe018106910 2024-03-04T08:47:50.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.677+0000 7fe00ffff700 1 -- 172.21.15.71:0/1530581841 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe014011b50 con 0x7fe018106910 2024-03-04T08:47:50.605 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.678+0000 7fe01f582700 1 -- 172.21.15.71:0/1530581841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe018106910 msgr2=0x7fe018106d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:50.605 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.678+0000 7fe01f582700 1 --2- 172.21.15.71:0/1530581841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe018106910 0x7fe018106d30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe0140098a0 tx=0x7fe014009bb0 comp rx=0 tx=0).stop 2024-03-04T08:47:50.605 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.678+0000 7fe01f582700 1 -- 172.21.15.71:0/1530581841 shutdown_connections 2024-03-04T08:47:50.605 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.678+0000 7fe01f582700 1 --2- 172.21.15.71:0/1530581841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe018106910 0x7fe018106d30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:50.605 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.678+0000 7fe01f582700 1 -- 172.21.15.71:0/1530581841 >> 172.21.15.71:0/1530581841 conn(0x7fe018101fb0 msgr2=0x7fe0181043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:50.605 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.678+0000 7fe01f582700 1 -- 172.21.15.71:0/1530581841 shutdown_connections 2024-03-04T08:47:50.605 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.678+0000 7fe01f582700 1 -- 172.21.15.71:0/1530581841 wait complete. 2024-03-04T08:47:50.605 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.679+0000 7fe01f582700 1 Processor -- start 2024-03-04T08:47:50.605 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.679+0000 7fe01f582700 1 -- start start 2024-03-04T08:47:50.605 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.679+0000 7fe01f582700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe018106910 0x7fe01819dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:50.605 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.679+0000 7fe01f582700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01810aec0 con 0x7fe018106910 2024-03-04T08:47:50.605 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.679+0000 7fe01d31e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe018106910 0x7fe01819dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:50.605 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.679+0000 7fe01d31e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe018106910 0x7fe01819dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43220/0 (socket says 172.21.15.71:43220) 2024-03-04T08:47:50.606 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.679+0000 7fe01d31e700 1 -- 172.21.15.71:0/1151475594 learned_addr learned my addr 172.21.15.71:0/1151475594 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:50.606 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.679+0000 7fe01d31e700 1 -- 172.21.15.71:0/1151475594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01819e140 con 0x7fe018106910 2024-03-04T08:47:50.606 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.679+0000 7fe01d31e700 1 --2- 172.21.15.71:0/1151475594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe018106910 0x7fe01819dc00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe014009fb0 tx=0x7fe014011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:50.606 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.679+0000 7fe00e7fc700 1 -- 172.21.15.71:0/1151475594 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0140038b0 con 0x7fe018106910 2024-03-04T08:47:50.606 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.679+0000 7fe00e7fc700 1 -- 172.21.15.71:0/1151475594 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe014011e50 con 0x7fe018106910 2024-03-04T08:47:50.606 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.679+0000 7fe01f582700 1 -- 172.21.15.71:0/1151475594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01819e3d0 con 0x7fe018106910 2024-03-04T08:47:50.606 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.679+0000 7fe00e7fc700 1 -- 172.21.15.71:0/1151475594 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe014019550 con 0x7fe018106910 2024-03-04T08:47:50.606 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.679+0000 7fe01f582700 1 -- 172.21.15.71:0/1151475594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0181a10c0 con 0x7fe018106910 2024-03-04T08:47:50.606 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.680+0000 7fe00e7fc700 1 -- 172.21.15.71:0/1151475594 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 45928+0+0 (secure 0 0 0) 0x7fe014019850 con 0x7fe018106910 2024-03-04T08:47:50.606 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.680+0000 7fe01f582700 1 -- 172.21.15.71:0/1151475594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe018069150 con 0x7fe018106910 2024-03-04T08:47:50.606 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.680+0000 7fe00e7fc700 1 --2- 172.21.15.71:0/1151475594 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7fe004038b50 0x7fe00403b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:50.606 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.681+0000 7fe00e7fc700 1 -- 172.21.15.71:0/1151475594 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe01404cc70 con 0x7fe018106910 2024-03-04T08:47:50.606 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.681+0000 7fe01cb1d700 1 -- 172.21.15.71:0/1151475594 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7fe004038b50 msgr2=0x7fe00403b010 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/1350134536 2024-03-04T08:47:50.607 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.681+0000 7fe01cb1d700 1 --2- 172.21.15.71:0/1151475594 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7fe004038b50 0x7fe00403b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:50.607 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.690+0000 7fe00e7fc700 1 -- 172.21.15.71:0/1151475594 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe01404c690 con 0x7fe018106910 2024-03-04T08:47:50.607 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.882+0000 7fe01cb1d700 1 -- 172.21.15.71:0/1151475594 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7fe004038b50 msgr2=0x7fe00403b010 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/1350134536 2024-03-04T08:47:50.607 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.882+0000 7fe01cb1d700 1 --2- 172.21.15.71:0/1151475594 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7fe004038b50 0x7fe00403b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:50.607 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.947+0000 7fe01f582700 1 -- 172.21.15.71:0/1151475594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fe01802dc20 con 0x7fe018106910 2024-03-04T08:47:50.607 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.948+0000 7fe00e7fc700 1 -- 172.21.15.71:0/1151475594 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fe01402d960 con 0x7fe018106910 2024-03-04T08:47:50.607 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.950+0000 7fe01f582700 1 -- 172.21.15.71:0/1151475594 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7fe004038b50 msgr2=0x7fe00403b010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T08:47:50.607 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.950+0000 7fe01f582700 1 --2- 172.21.15.71:0/1151475594 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7fe004038b50 0x7fe00403b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:50.607 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.950+0000 7fe01f582700 1 -- 172.21.15.71:0/1151475594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe018106910 msgr2=0x7fe01819dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:50.607 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.950+0000 7fe01f582700 1 --2- 172.21.15.71:0/1151475594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe018106910 0x7fe01819dc00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe014009fb0 tx=0x7fe014011bf0 comp rx=0 tx=0).stop 2024-03-04T08:47:50.607 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.950+0000 7fe01f582700 1 -- 172.21.15.71:0/1151475594 shutdown_connections 2024-03-04T08:47:50.607 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.950+0000 7fe01f582700 1 --2- 172.21.15.71:0/1151475594 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7fe004038b50 0x7fe00403b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:50.607 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.950+0000 7fe01f582700 1 --2- 172.21.15.71:0/1151475594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe018106910 0x7fe01819dc00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:50.607 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.950+0000 7fe01f582700 1 -- 172.21.15.71:0/1151475594 >> 172.21.15.71:0/1151475594 conn(0x7fe018101fb0 msgr2=0x7fe018102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:50.608 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.950+0000 7fe01f582700 1 -- 172.21.15.71:0/1151475594 shutdown_connections 2024-03-04T08:47:50.608 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:49.950+0000 7fe01f582700 1 -- 172.21.15.71:0/1151475594 wait complete. 2024-03-04T08:47:50.608 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for the mgr to restart... 2024-03-04T08:47:50.608 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mgr epoch 5... 2024-03-04T08:47:50.845 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:50 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1151475594' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-04T08:47:50.845 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:50 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:50.679+0000 7feea1368200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-04T08:47:50.845 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:50 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:50.789+0000 7feea1368200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T08:47:51.155 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:50 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:50.879+0000 7feea1368200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T08:47:51.155 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:50 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:50.974+0000 7feea1368200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T08:47:51.422 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:51 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:51.154+0000 7feea1368200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T08:47:52.421 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:52 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:52.112+0000 7feea1368200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-04T08:47:52.421 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:52 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:52.239+0000 7feea1368200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T08:47:52.849 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:52 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:52.567+0000 7feea1368200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T08:47:52.849 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:52 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:52.656+0000 7feea1368200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T08:47:52.849 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:52 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:52.853+0000 7feea1368200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T08:47:53.171 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:53 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:53.037+0000 7feea1368200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T08:47:53.576 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:53 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:53.323+0000 7feea1368200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T08:47:53.576 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:53 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:53.580+0000 7feea1368200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T08:47:53.921 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:53 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:53.816+0000 7feea1368200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T08:47:54.921 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:54 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:54.612+0000 7feea1368200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T08:47:55.421 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:55 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:55.175+0000 7feea1368200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T08:47:55.421 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:55 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:55.272+0000 7feea1368200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T08:47:55.921 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:47:55 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:47:55.488+0000 7feea1368200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T08:47:55.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:55 smithi071 ceph-mon[32789]: Active manager daemon x restarted 2024-03-04T08:47:55.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:55 smithi071 ceph-mon[32789]: Activating manager daemon x 2024-03-04T08:47:55.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:55 smithi071 ceph-mon[32789]: osdmap e2: 0 total, 0 up, 0 in 2024-03-04T08:47:55.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:55 smithi071 ceph-mon[32789]: mgrmap e6: x(active, starting, since 0.00350842s) 2024-03-04T08:47:55.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:55 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T08:47:55.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:55 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-04T08:47:55.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:55 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T08:47:55.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:55 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T08:47:55.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:55 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T08:47:55.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:55 smithi071 ceph-mon[32789]: Manager daemon x is now available 2024-03-04T08:47:55.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:55 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' 2024-03-04T08:47:55.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:55 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' 2024-03-04T08:47:55.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:55 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:47:55.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:55 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:47:56.808 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:56 smithi071 ceph-mon[32789]: Found migration_current of "None". Setting to last migration. 2024-03-04T08:47:56.808 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:56 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:47:56.808 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:56 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-04T08:47:56.808 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:56 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-04T08:47:56.808 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:56 smithi071 ceph-mon[32789]: mgrmap e7: x(active, since 1.00535s) 2024-03-04T08:47:57.277 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-03-04T08:47:57.277 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-04T08:47:57.277 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-04T08:47:57.277 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-03-04T08:47:57.277 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.035+0000 7f7459e56700 1 Processor -- start 2024-03-04T08:47:57.277 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.035+0000 7f7459e56700 1 -- start start 2024-03-04T08:47:57.278 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.035+0000 7f7459e56700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7454107aa0 0x7f7454107ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:57.278 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.035+0000 7f7459e56700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7454108490 con 0x7f7454107aa0 2024-03-04T08:47:57.278 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.036+0000 7f74537fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7454107aa0 0x7f7454107ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:57.278 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.036+0000 7f74537fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7454107aa0 0x7f7454107ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43230/0 (socket says 172.21.15.71:43230) 2024-03-04T08:47:57.278 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.036+0000 7f74537fe700 1 -- 172.21.15.71:0/1773161868 learned_addr learned my addr 172.21.15.71:0/1773161868 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:57.278 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.036+0000 7f74537fe700 1 -- 172.21.15.71:0/1773161868 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7454108ca0 con 0x7f7454107aa0 2024-03-04T08:47:57.278 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.036+0000 7f74537fe700 1 --2- 172.21.15.71:0/1773161868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7454107aa0 0x7f7454107ec0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f74480098a0 tx=0x7f7448009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4141e3c7149bf60f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:57.278 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.036+0000 7f74527fc700 1 -- 172.21.15.71:0/1773161868 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7448004160 con 0x7f7454107aa0 2024-03-04T08:47:57.278 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.036+0000 7f74527fc700 1 -- 172.21.15.71:0/1773161868 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7448004300 con 0x7f7454107aa0 2024-03-04T08:47:57.278 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.036+0000 7f74527fc700 1 -- 172.21.15.71:0/1773161868 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7448011b50 con 0x7f7454107aa0 2024-03-04T08:47:57.278 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.037+0000 7f7459e56700 1 -- 172.21.15.71:0/1773161868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7454107aa0 msgr2=0x7f7454107ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:57.278 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.037+0000 7f7459e56700 1 --2- 172.21.15.71:0/1773161868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7454107aa0 0x7f7454107ec0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f74480098a0 tx=0x7f7448009bb0 comp rx=0 tx=0).stop 2024-03-04T08:47:57.278 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.037+0000 7f7459e56700 1 -- 172.21.15.71:0/1773161868 shutdown_connections 2024-03-04T08:47:57.279 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.037+0000 7f7459e56700 1 --2- 172.21.15.71:0/1773161868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7454107aa0 0x7f7454107ec0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:57.279 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.037+0000 7f7459e56700 1 -- 172.21.15.71:0/1773161868 >> 172.21.15.71:0/1773161868 conn(0x7f74541030a0 msgr2=0x7f74541054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:57.279 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.037+0000 7f7459e56700 1 -- 172.21.15.71:0/1773161868 shutdown_connections 2024-03-04T08:47:57.279 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.037+0000 7f7459e56700 1 -- 172.21.15.71:0/1773161868 wait complete. 2024-03-04T08:47:57.279 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.038+0000 7f7459e56700 1 Processor -- start 2024-03-04T08:47:57.279 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.038+0000 7f7459e56700 1 -- start start 2024-03-04T08:47:57.279 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.038+0000 7f7459e56700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7454107aa0 0x7f745419ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:57.279 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.038+0000 7f7459e56700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745410bed0 con 0x7f7454107aa0 2024-03-04T08:47:57.279 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.038+0000 7f74537fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7454107aa0 0x7f745419ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:57.279 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.038+0000 7f74537fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7454107aa0 0x7f745419ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43240/0 (socket says 172.21.15.71:43240) 2024-03-04T08:47:57.279 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.038+0000 7f74537fe700 1 -- 172.21.15.71:0/2138061506 learned_addr learned my addr 172.21.15.71:0/2138061506 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:57.279 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.038+0000 7f74537fe700 1 -- 172.21.15.71:0/2138061506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745419bf80 con 0x7f7454107aa0 2024-03-04T08:47:57.279 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.039+0000 7f74537fe700 1 --2- 172.21.15.71:0/2138061506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7454107aa0 0x7f745419ba40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7448009fb0 tx=0x7f7448011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:57.279 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.039+0000 7f7450ff9700 1 -- 172.21.15.71:0/2138061506 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74480039e0 con 0x7f7454107aa0 2024-03-04T08:47:57.280 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.039+0000 7f7459e56700 1 -- 172.21.15.71:0/2138061506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f745419c210 con 0x7f7454107aa0 2024-03-04T08:47:57.280 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.039+0000 7f7450ff9700 1 -- 172.21.15.71:0/2138061506 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7448011e50 con 0x7f7454107aa0 2024-03-04T08:47:57.280 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.039+0000 7f7459e56700 1 -- 172.21.15.71:0/2138061506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f745419ef00 con 0x7f7454107aa0 2024-03-04T08:47:57.280 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.039+0000 7f7450ff9700 1 -- 172.21.15.71:0/2138061506 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7448023550 con 0x7f7454107aa0 2024-03-04T08:47:57.280 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.040+0000 7f7450ff9700 1 -- 172.21.15.71:0/2138061506 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f7448022030 con 0x7f7454107aa0 2024-03-04T08:47:57.280 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.040+0000 7f7450ff9700 1 --2- 172.21.15.71:0/2138061506 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f743c038ba0 0x7f743c03b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:57.280 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.040+0000 7f7450ff9700 1 -- 172.21.15.71:0/2138061506 --> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f743c03b770 con 0x7f743c038ba0 2024-03-04T08:47:57.280 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.040+0000 7f7450ff9700 1 -- 172.21.15.71:0/2138061506 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f744804c610 con 0x7f7454107aa0 2024-03-04T08:47:57.280 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.040+0000 7f7452ffd700 1 -- 172.21.15.71:0/2138061506 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f743c038ba0 msgr2=0x7f743c03b060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/1350134536 2024-03-04T08:47:57.280 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.040+0000 7f7452ffd700 1 --2- 172.21.15.71:0/2138061506 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f743c038ba0 0x7f743c03b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:57.280 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.241+0000 7f7452ffd700 1 -- 172.21.15.71:0/2138061506 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f743c038ba0 msgr2=0x7f743c03b060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/1350134536 2024-03-04T08:47:57.280 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.241+0000 7f7452ffd700 1 --2- 172.21.15.71:0/2138061506 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f743c038ba0 0x7f743c03b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:57.280 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.642+0000 7f7452ffd700 1 -- 172.21.15.71:0/2138061506 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f743c038ba0 msgr2=0x7f743c03b060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/1350134536 2024-03-04T08:47:57.280 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:51.642+0000 7f7452ffd700 1 --2- 172.21.15.71:0/2138061506 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f743c038ba0 0x7f743c03b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:57.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:52.443+0000 7f7452ffd700 1 -- 172.21.15.71:0/2138061506 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f743c038ba0 msgr2=0x7f743c03b060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/1350134536 2024-03-04T08:47:57.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:52.443+0000 7f7452ffd700 1 --2- 172.21.15.71:0/2138061506 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f743c038ba0 0x7f743c03b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:57.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:54.044+0000 7f7452ffd700 1 -- 172.21.15.71:0/2138061506 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f743c038ba0 msgr2=0x7f743c03b060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/1350134536 2024-03-04T08:47:57.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:54.044+0000 7f7452ffd700 1 --2- 172.21.15.71:0/2138061506 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f743c038ba0 0x7f743c03b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-04T08:47:57.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:55.504+0000 7f7450ff9700 1 -- 172.21.15.71:0/2138061506 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mgrmap(e 6) v1 ==== 45695+0+0 (secure 0 0 0) 0x7f7448023d20 con 0x7f7454107aa0 2024-03-04T08:47:57.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:55.504+0000 7f7450ff9700 1 -- 172.21.15.71:0/2138061506 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f743c038ba0 msgr2=0x7f743c03b060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T08:47:57.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:55.504+0000 7f7450ff9700 1 --2- 172.21.15.71:0/2138061506 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f743c038ba0 0x7f743c03b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:57.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:56.505+0000 7f7450ff9700 1 -- 172.21.15.71:0/2138061506 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 7) v1 ==== 45822+0+0 (secure 0 0 0) 0x7f7448010d50 con 0x7f7454107aa0 2024-03-04T08:47:57.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:56.505+0000 7f7450ff9700 1 --2- 172.21.15.71:0/2138061506 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f743c03c4d0 0x7f743c03e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:57.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:56.505+0000 7f7450ff9700 1 -- 172.21.15.71:0/2138061506 --> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f743c03b770 con 0x7f743c03c4d0 2024-03-04T08:47:57.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:56.505+0000 7f7452ffd700 1 --2- 172.21.15.71:0/2138061506 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f743c03c4d0 0x7f743c03e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:57.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:56.506+0000 7f7452ffd700 1 --2- 172.21.15.71:0/2138061506 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f743c03c4d0 0x7f743c03e8c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7444003a10 tx=0x7f74440092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:57.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:56.506+0000 7f7450ff9700 1 -- 172.21.15.71:0/2138061506 <== mgr.14120 v2:172.21.15.71:6800/2808156336 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f743c03b770 con 0x7f743c03c4d0 2024-03-04T08:47:57.281 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:56.513+0000 7f7459e56700 1 -- 172.21.15.71:0/2138061506 --> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f745410c050 con 0x7f743c03c4d0 2024-03-04T08:47:57.282 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:56.513+0000 7f7450ff9700 1 -- 172.21.15.71:0/2138061506 <== mgr.14120 v2:172.21.15.71:6800/2808156336 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f745410c050 con 0x7f743c03c4d0 2024-03-04T08:47:57.282 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:56.513+0000 7f7459e56700 1 -- 172.21.15.71:0/2138061506 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f743c03c4d0 msgr2=0x7f743c03e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:57.282 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:56.513+0000 7f7459e56700 1 --2- 172.21.15.71:0/2138061506 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f743c03c4d0 0x7f743c03e8c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7444003a10 tx=0x7f74440092b0 comp rx=0 tx=0).stop 2024-03-04T08:47:57.282 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:56.513+0000 7f7459e56700 1 -- 172.21.15.71:0/2138061506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7454107aa0 msgr2=0x7f745419ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:57.282 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:56.513+0000 7f7459e56700 1 --2- 172.21.15.71:0/2138061506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7454107aa0 0x7f745419ba40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7448009fb0 tx=0x7f7448011c10 comp rx=0 tx=0).stop 2024-03-04T08:47:57.282 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:56.513+0000 7f7459e56700 1 -- 172.21.15.71:0/2138061506 shutdown_connections 2024-03-04T08:47:57.282 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:56.513+0000 7f7459e56700 1 --2- 172.21.15.71:0/2138061506 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f743c03c4d0 0x7f743c03e8c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:57.282 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:56.513+0000 7f7459e56700 1 --2- 172.21.15.71:0/2138061506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7454107aa0 0x7f745419ba40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:57.282 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:56.513+0000 7f7459e56700 1 --2- 172.21.15.71:0/2138061506 >> [v2:172.21.15.71:6800/1350134536,v1:172.21.15.71:6801/1350134536] conn(0x7f743c038ba0 0x7f743c03b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:57.282 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:56.513+0000 7f7459e56700 1 -- 172.21.15.71:0/2138061506 >> 172.21.15.71:0/2138061506 conn(0x7f74541030a0 msgr2=0x7f7454103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:57.282 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:56.513+0000 7f7459e56700 1 -- 172.21.15.71:0/2138061506 shutdown_connections 2024-03-04T08:47:57.282 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:56.513+0000 7f7459e56700 1 -- 172.21.15.71:0/2138061506 wait complete. 2024-03-04T08:47:57.282 INFO:teuthology.orchestra.run.smithi071.stdout:mgr epoch 5 is available 2024-03-04T08:47:57.282 INFO:teuthology.orchestra.run.smithi071.stdout:Setting orchestrator backend to cephadm... 2024-03-04T08:47:57.898 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:57 smithi071 ceph-mon[32789]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-04T08:47:57.898 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:57 smithi071 ceph-mon[32789]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-04T08:47:57.898 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:57 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' 2024-03-04T08:47:57.899 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:57 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' 2024-03-04T08:47:57.899 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:57 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:47:58.424 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.684+0000 7fd7a3cb7700 1 Processor -- start 2024-03-04T08:47:58.424 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.684+0000 7fd7a3cb7700 1 -- start start 2024-03-04T08:47:58.424 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.684+0000 7fd7a3cb7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd79c108550 0x7fd79c10a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:58.424 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.684+0000 7fd7a3cb7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c07adc0 con 0x7fd79c108550 2024-03-04T08:47:58.424 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.684+0000 7fd7a1a53700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd79c108550 0x7fd79c10a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:58.424 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.684+0000 7fd7a1a53700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd79c108550 0x7fd79c10a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49878/0 (socket says 172.21.15.71:49878) 2024-03-04T08:47:58.424 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.684+0000 7fd7a1a53700 1 -- 172.21.15.71:0/33025949 learned_addr learned my addr 172.21.15.71:0/33025949 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:58.425 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.685+0000 7fd7a1a53700 1 -- 172.21.15.71:0/33025949 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79c10ae80 con 0x7fd79c108550 2024-03-04T08:47:58.425 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.685+0000 7fd7a1a53700 1 --2- 172.21.15.71:0/33025949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd79c108550 0x7fd79c10a940 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd7980098a0 tx=0x7fd798009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feea60aa9f3b4a0c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:58.425 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.685+0000 7fd7a0a51700 1 -- 172.21.15.71:0/33025949 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd798004160 con 0x7fd79c108550 2024-03-04T08:47:58.425 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.685+0000 7fd7a0a51700 1 -- 172.21.15.71:0/33025949 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd798004300 con 0x7fd79c108550 2024-03-04T08:47:58.425 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.685+0000 7fd7a0a51700 1 -- 172.21.15.71:0/33025949 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd798011b50 con 0x7fd79c108550 2024-03-04T08:47:58.425 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.686+0000 7fd7a3cb7700 1 -- 172.21.15.71:0/33025949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd79c108550 msgr2=0x7fd79c10a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:58.425 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.686+0000 7fd7a3cb7700 1 --2- 172.21.15.71:0/33025949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd79c108550 0x7fd79c10a940 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd7980098a0 tx=0x7fd798009bb0 comp rx=0 tx=0).stop 2024-03-04T08:47:58.425 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.686+0000 7fd7a3cb7700 1 -- 172.21.15.71:0/33025949 shutdown_connections 2024-03-04T08:47:58.425 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.686+0000 7fd7a3cb7700 1 --2- 172.21.15.71:0/33025949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd79c108550 0x7fd79c10a940 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:58.425 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.686+0000 7fd7a3cb7700 1 -- 172.21.15.71:0/33025949 >> 172.21.15.71:0/33025949 conn(0x7fd79c101f60 msgr2=0x7fd79c104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:58.425 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.686+0000 7fd7a3cb7700 1 -- 172.21.15.71:0/33025949 shutdown_connections 2024-03-04T08:47:58.425 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.686+0000 7fd7a3cb7700 1 -- 172.21.15.71:0/33025949 wait complete. 2024-03-04T08:47:58.425 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.686+0000 7fd7a3cb7700 1 Processor -- start 2024-03-04T08:47:58.425 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.686+0000 7fd7a3cb7700 1 -- start start 2024-03-04T08:47:58.426 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.686+0000 7fd7a3cb7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd79c108550 0x7fd79c107dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:58.426 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.686+0000 7fd7a3cb7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c10bd70 con 0x7fd79c108550 2024-03-04T08:47:58.426 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.687+0000 7fd7a1a53700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd79c108550 0x7fd79c107dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:58.426 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.687+0000 7fd7a1a53700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd79c108550 0x7fd79c107dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49890/0 (socket says 172.21.15.71:49890) 2024-03-04T08:47:58.426 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.687+0000 7fd7a1a53700 1 -- 172.21.15.71:0/1753935917 learned_addr learned my addr 172.21.15.71:0/1753935917 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:58.426 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.687+0000 7fd7a1a53700 1 -- 172.21.15.71:0/1753935917 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79c108310 con 0x7fd79c108550 2024-03-04T08:47:58.426 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.687+0000 7fd7a1a53700 1 --2- 172.21.15.71:0/1753935917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd79c108550 0x7fd79c107dd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd798011b60 tx=0x7fd798011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:58.426 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.687+0000 7fd792ffd700 1 -- 172.21.15.71:0/1753935917 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd798003970 con 0x7fd79c108550 2024-03-04T08:47:58.426 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.687+0000 7fd7a3cb7700 1 -- 172.21.15.71:0/1753935917 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd79c106500 con 0x7fd79c108550 2024-03-04T08:47:58.426 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.687+0000 7fd792ffd700 1 -- 172.21.15.71:0/1753935917 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd798003b10 con 0x7fd79c108550 2024-03-04T08:47:58.426 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.687+0000 7fd792ffd700 1 -- 172.21.15.71:0/1753935917 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd798024610 con 0x7fd79c108550 2024-03-04T08:47:58.426 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.687+0000 7fd7a3cb7700 1 -- 172.21.15.71:0/1753935917 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd79c106a40 con 0x7fd79c108550 2024-03-04T08:47:58.426 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.688+0000 7fd792ffd700 1 -- 172.21.15.71:0/1753935917 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7fd798024910 con 0x7fd79c108550 2024-03-04T08:47:58.427 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.688+0000 7fd7a3cb7700 1 -- 172.21.15.71:0/1753935917 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd79c0636f0 con 0x7fd79c108550 2024-03-04T08:47:58.427 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.688+0000 7fd792ffd700 1 --2- 172.21.15.71:0/1753935917 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fd788038ba0 0x7fd78803b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:58.427 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.688+0000 7fd792ffd700 1 -- 172.21.15.71:0/1753935917 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd798011e00 con 0x7fd79c108550 2024-03-04T08:47:58.427 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.688+0000 7fd7a1252700 1 --2- 172.21.15.71:0/1753935917 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fd788038ba0 0x7fd78803b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:58.427 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.689+0000 7fd7a1252700 1 --2- 172.21.15.71:0/1753935917 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fd788038ba0 0x7fd78803b060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd78c006fd0 tx=0x7fd78c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:58.427 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.693+0000 7fd792ffd700 1 -- 172.21.15.71:0/1753935917 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd79800b3e0 con 0x7fd79c108550 2024-03-04T08:47:58.427 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.902+0000 7fd7a3cb7700 1 -- 172.21.15.71:0/1753935917 --> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fd79c103220 con 0x7fd788038ba0 2024-03-04T08:47:58.427 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.908+0000 7fd792ffd700 1 -- 172.21.15.71:0/1753935917 <== mgr.14120 v2:172.21.15.71:6800/2808156336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd79c103220 con 0x7fd788038ba0 2024-03-04T08:47:58.427 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.910+0000 7fd7a3cb7700 1 -- 172.21.15.71:0/1753935917 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fd788038ba0 msgr2=0x7fd78803b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:58.427 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.910+0000 7fd7a3cb7700 1 --2- 172.21.15.71:0/1753935917 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fd788038ba0 0x7fd78803b060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd78c006fd0 tx=0x7fd78c006e40 comp rx=0 tx=0).stop 2024-03-04T08:47:58.427 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.910+0000 7fd7a3cb7700 1 -- 172.21.15.71:0/1753935917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd79c108550 msgr2=0x7fd79c107dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:58.427 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.910+0000 7fd7a3cb7700 1 --2- 172.21.15.71:0/1753935917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd79c108550 0x7fd79c107dd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd798011b60 tx=0x7fd798011c40 comp rx=0 tx=0).stop 2024-03-04T08:47:58.427 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.910+0000 7fd7a3cb7700 1 -- 172.21.15.71:0/1753935917 shutdown_connections 2024-03-04T08:47:58.427 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.910+0000 7fd7a3cb7700 1 --2- 172.21.15.71:0/1753935917 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fd788038ba0 0x7fd78803b060 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:58.428 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.910+0000 7fd7a3cb7700 1 --2- 172.21.15.71:0/1753935917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd79c108550 0x7fd79c107dd0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:58.428 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.910+0000 7fd7a3cb7700 1 -- 172.21.15.71:0/1753935917 >> 172.21.15.71:0/1753935917 conn(0x7fd79c101f60 msgr2=0x7fd79c102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:58.428 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.910+0000 7fd7a3cb7700 1 -- 172.21.15.71:0/1753935917 shutdown_connections 2024-03-04T08:47:58.428 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:57.910+0000 7fd7a3cb7700 1 -- 172.21.15.71:0/1753935917 wait complete. 2024-03-04T08:47:58.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:58 smithi071 ceph-mon[32789]: [04/Mar/2024:08:47:56] ENGINE Bus STARTING 2024-03-04T08:47:58.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:58 smithi071 ceph-mon[32789]: [04/Mar/2024:08:47:57] ENGINE Serving on http://172.21.15.71:8765 2024-03-04T08:47:58.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:58 smithi071 ceph-mon[32789]: [04/Mar/2024:08:47:57] ENGINE Serving on https://172.21.15.71:7150 2024-03-04T08:47:58.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:58 smithi071 ceph-mon[32789]: [04/Mar/2024:08:47:57] ENGINE Bus STARTED 2024-03-04T08:47:58.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:58 smithi071 ceph-mon[32789]: mgrmap e8: x(active, since 2s) 2024-03-04T08:47:58.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:58 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' 2024-03-04T08:47:58.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:58 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:47:59.714 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-04T08:47:59.715 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.844+0000 7fe734c3a700 1 Processor -- start 2024-03-04T08:47:59.715 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.844+0000 7fe734c3a700 1 -- start start 2024-03-04T08:47:59.715 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.844+0000 7fe734c3a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7301084e0 0x7fe73010a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:59.715 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.844+0000 7fe734c3a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73007ae40 con 0x7fe7301084e0 2024-03-04T08:47:59.715 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.844+0000 7fe72e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7301084e0 0x7fe73010a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:59.715 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.844+0000 7fe72e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7301084e0 0x7fe73010a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49900/0 (socket says 172.21.15.71:49900) 2024-03-04T08:47:59.715 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.844+0000 7fe72e59c700 1 -- 172.21.15.71:0/3908537835 learned_addr learned my addr 172.21.15.71:0/3908537835 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:59.715 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.844+0000 7fe72e59c700 1 -- 172.21.15.71:0/3908537835 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73010ae10 con 0x7fe7301084e0 2024-03-04T08:47:59.715 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.845+0000 7fe72e59c700 1 --2- 172.21.15.71:0/3908537835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7301084e0 0x7fe73010a8d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe7200098a0 tx=0x7fe720009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a339eb20d2124d83 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:59.715 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.845+0000 7fe72d59a700 1 -- 172.21.15.71:0/3908537835 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe720004160 con 0x7fe7301084e0 2024-03-04T08:47:59.715 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.845+0000 7fe72d59a700 1 -- 172.21.15.71:0/3908537835 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe720004300 con 0x7fe7301084e0 2024-03-04T08:47:59.716 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.845+0000 7fe72d59a700 1 -- 172.21.15.71:0/3908537835 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe720011b50 con 0x7fe7301084e0 2024-03-04T08:47:59.716 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.845+0000 7fe734c3a700 1 -- 172.21.15.71:0/3908537835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7301084e0 msgr2=0x7fe73010a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:59.716 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.845+0000 7fe734c3a700 1 --2- 172.21.15.71:0/3908537835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7301084e0 0x7fe73010a8d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe7200098a0 tx=0x7fe720009bb0 comp rx=0 tx=0).stop 2024-03-04T08:47:59.716 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.845+0000 7fe734c3a700 1 -- 172.21.15.71:0/3908537835 shutdown_connections 2024-03-04T08:47:59.716 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.845+0000 7fe734c3a700 1 --2- 172.21.15.71:0/3908537835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7301084e0 0x7fe73010a8d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:59.716 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.846+0000 7fe734c3a700 1 -- 172.21.15.71:0/3908537835 >> 172.21.15.71:0/3908537835 conn(0x7fe730101f90 msgr2=0x7fe7301043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:59.716 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.846+0000 7fe734c3a700 1 -- 172.21.15.71:0/3908537835 shutdown_connections 2024-03-04T08:47:59.716 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.846+0000 7fe734c3a700 1 -- 172.21.15.71:0/3908537835 wait complete. 2024-03-04T08:47:59.716 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.846+0000 7fe734c3a700 1 Processor -- start 2024-03-04T08:47:59.716 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.846+0000 7fe734c3a700 1 -- start start 2024-03-04T08:47:59.716 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.846+0000 7fe734c3a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7301084e0 0x7fe73019b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:59.716 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.846+0000 7fe734c3a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73010bd70 con 0x7fe7301084e0 2024-03-04T08:47:59.717 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.847+0000 7fe72e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7301084e0 0x7fe73019b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:59.717 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.847+0000 7fe72e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7301084e0 0x7fe73019b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49908/0 (socket says 172.21.15.71:49908) 2024-03-04T08:47:59.717 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.847+0000 7fe72e59c700 1 -- 172.21.15.71:0/3420985881 learned_addr learned my addr 172.21.15.71:0/3420985881 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:47:59.717 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.847+0000 7fe72e59c700 1 -- 172.21.15.71:0/3420985881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73019beb0 con 0x7fe7301084e0 2024-03-04T08:47:59.717 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.847+0000 7fe72e59c700 1 --2- 172.21.15.71:0/3420985881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7301084e0 0x7fe73019b970 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe720009fb0 tx=0x7fe720011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:59.717 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.847+0000 7fe71f7fe700 1 -- 172.21.15.71:0/3420985881 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe7200038b0 con 0x7fe7301084e0 2024-03-04T08:47:59.717 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.847+0000 7fe71f7fe700 1 -- 172.21.15.71:0/3420985881 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe720011e50 con 0x7fe7301084e0 2024-03-04T08:47:59.717 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.847+0000 7fe734c3a700 1 -- 172.21.15.71:0/3420985881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe73019c140 con 0x7fe7301084e0 2024-03-04T08:47:59.717 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.847+0000 7fe71f7fe700 1 -- 172.21.15.71:0/3420985881 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe720019420 con 0x7fe7301084e0 2024-03-04T08:47:59.717 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.847+0000 7fe734c3a700 1 -- 172.21.15.71:0/3420985881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe73019ee30 con 0x7fe7301084e0 2024-03-04T08:47:59.717 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.848+0000 7fe71f7fe700 1 -- 172.21.15.71:0/3420985881 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7fe7200196d0 con 0x7fe7301084e0 2024-03-04T08:47:59.717 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.848+0000 7fe71f7fe700 1 --2- 172.21.15.71:0/3420985881 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fe718038ba0 0x7fe71803b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:59.717 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.848+0000 7fe71f7fe700 1 -- 172.21.15.71:0/3420985881 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe72004dc90 con 0x7fe7301084e0 2024-03-04T08:47:59.718 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.848+0000 7fe72dd9b700 1 --2- 172.21.15.71:0/3420985881 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fe718038ba0 0x7fe71803b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:47:59.718 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.848+0000 7fe734c3a700 1 -- 172.21.15.71:0/3420985881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe730069150 con 0x7fe7301084e0 2024-03-04T08:47:59.718 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.848+0000 7fe72dd9b700 1 --2- 172.21.15.71:0/3420985881 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fe718038ba0 0x7fe71803b060 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe724006fd0 tx=0x7fe724006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:47:59.718 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:58.855+0000 7fe71f7fe700 1 -- 172.21.15.71:0/3420985881 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe720023c90 con 0x7fe7301084e0 2024-03-04T08:47:59.718 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:59.052+0000 7fe734c3a700 1 -- 172.21.15.71:0/3420985881 --> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7301031e0 con 0x7fe718038ba0 2024-03-04T08:47:59.718 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:59.053+0000 7fe71f7fe700 1 -- 172.21.15.71:0/3420985881 <== mgr.14120 v2:172.21.15.71:6800/2808156336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fe7301031e0 con 0x7fe718038ba0 2024-03-04T08:47:59.718 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:59.055+0000 7fe734c3a700 1 -- 172.21.15.71:0/3420985881 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fe718038ba0 msgr2=0x7fe71803b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:59.718 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:59.055+0000 7fe734c3a700 1 --2- 172.21.15.71:0/3420985881 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fe718038ba0 0x7fe71803b060 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe724006fd0 tx=0x7fe724006e40 comp rx=0 tx=0).stop 2024-03-04T08:47:59.718 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:59.055+0000 7fe734c3a700 1 -- 172.21.15.71:0/3420985881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7301084e0 msgr2=0x7fe73019b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:47:59.718 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:59.055+0000 7fe734c3a700 1 --2- 172.21.15.71:0/3420985881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7301084e0 0x7fe73019b970 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe720009fb0 tx=0x7fe720011bf0 comp rx=0 tx=0).stop 2024-03-04T08:47:59.718 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:59.055+0000 7fe734c3a700 1 -- 172.21.15.71:0/3420985881 shutdown_connections 2024-03-04T08:47:59.718 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:59.055+0000 7fe734c3a700 1 --2- 172.21.15.71:0/3420985881 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fe718038ba0 0x7fe71803b060 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:59.718 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:59.055+0000 7fe734c3a700 1 --2- 172.21.15.71:0/3420985881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7301084e0 0x7fe73019b970 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:59.719 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:59.055+0000 7fe734c3a700 1 -- 172.21.15.71:0/3420985881 >> 172.21.15.71:0/3420985881 conn(0x7fe730101f90 msgr2=0x7fe730102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:47:59.719 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:59.055+0000 7fe734c3a700 1 -- 172.21.15.71:0/3420985881 shutdown_connections 2024-03-04T08:47:59.719 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:47:59.055+0000 7fe734c3a700 1 -- 172.21.15.71:0/3420985881 wait complete. 2024-03-04T08:47:59.719 INFO:teuthology.orchestra.run.smithi071.stdout:Generating ssh key... 2024-03-04T08:47:59.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:47:59 smithi071 ceph-mon[32789]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:48:00.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:00 smithi071 ceph-mon[32789]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:48:00.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:00 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' 2024-03-04T08:48:00.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:00 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' 2024-03-04T08:48:00.922 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: Generating public/private rsa key pair. 2024-03-04T08:48:00.922 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: Your identification has been saved in /tmp/tmpi50grfxb/key. 2024-03-04T08:48:00.922 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: Your public key has been saved in /tmp/tmpi50grfxb/key.pub. 2024-03-04T08:48:00.922 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: The key fingerprint is: 2024-03-04T08:48:00.922 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: SHA256:nvC30q0MH2bwAnXGqrXRJ9prsuucDDUBDJ4q6flM1kA ceph-b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T08:48:00.922 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: The key's randomart image is: 2024-03-04T08:48:00.922 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: +---[RSA 3072]----+ 2024-03-04T08:48:00.922 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: | .o. | 2024-03-04T08:48:00.923 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: | . ... . | 2024-03-04T08:48:00.923 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: | Eo o + | 2024-03-04T08:48:00.923 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: | ... . * | 2024-03-04T08:48:00.923 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: |o .. o S o . | 2024-03-04T08:48:00.923 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: |... o O X o | 2024-03-04T08:48:00.923 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: | o o .o O.B. | 2024-03-04T08:48:00.923 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: | = +oXo+. | 2024-03-04T08:48:00.923 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: | o .B**. | 2024-03-04T08:48:00.923 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: +----[SHA256]-----+ 2024-03-04T08:48:01.186 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.154+0000 7f92e2dc1700 1 Processor -- start 2024-03-04T08:48:01.186 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.154+0000 7f92e2dc1700 1 -- start start 2024-03-04T08:48:01.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.155+0000 7f92e2dc1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92dc07ce40 0x7f92dc07b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:01.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.155+0000 7f92e2dc1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc07b8c0 con 0x7f92dc07ce40 2024-03-04T08:48:01.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.155+0000 7f92e0b5d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92dc07ce40 0x7f92dc07b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:01.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.155+0000 7f92e0b5d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92dc07ce40 0x7f92dc07b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49912/0 (socket says 172.21.15.71:49912) 2024-03-04T08:48:01.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.155+0000 7f92e0b5d700 1 -- 172.21.15.71:0/1662536944 learned_addr learned my addr 172.21.15.71:0/1662536944 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:01.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.155+0000 7f92e0b5d700 1 -- 172.21.15.71:0/1662536944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92dc07ba40 con 0x7f92dc07ce40 2024-03-04T08:48:01.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.155+0000 7f92e0b5d700 1 --2- 172.21.15.71:0/1662536944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92dc07ce40 0x7f92dc07b2f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f92d00098a0 tx=0x7f92d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f555efa6647d476b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:01.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.155+0000 7f92db7fe700 1 -- 172.21.15.71:0/1662536944 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92d0004160 con 0x7f92dc07ce40 2024-03-04T08:48:01.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.155+0000 7f92db7fe700 1 -- 172.21.15.71:0/1662536944 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92d0004300 con 0x7f92dc07ce40 2024-03-04T08:48:01.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.155+0000 7f92db7fe700 1 -- 172.21.15.71:0/1662536944 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92d0011b50 con 0x7f92dc07ce40 2024-03-04T08:48:01.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.156+0000 7f92e2dc1700 1 -- 172.21.15.71:0/1662536944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92dc07ce40 msgr2=0x7f92dc07b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:01.187 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.156+0000 7f92e2dc1700 1 --2- 172.21.15.71:0/1662536944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92dc07ce40 0x7f92dc07b2f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f92d00098a0 tx=0x7f92d0009bb0 comp rx=0 tx=0).stop 2024-03-04T08:48:01.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.156+0000 7f92e2dc1700 1 -- 172.21.15.71:0/1662536944 shutdown_connections 2024-03-04T08:48:01.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.156+0000 7f92e2dc1700 1 --2- 172.21.15.71:0/1662536944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92dc07ce40 0x7f92dc07b2f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:01.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.156+0000 7f92e2dc1700 1 -- 172.21.15.71:0/1662536944 >> 172.21.15.71:0/1662536944 conn(0x7f92dc103040 msgr2=0x7f92dc105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:01.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.156+0000 7f92e2dc1700 1 -- 172.21.15.71:0/1662536944 shutdown_connections 2024-03-04T08:48:01.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.156+0000 7f92e2dc1700 1 -- 172.21.15.71:0/1662536944 wait complete. 2024-03-04T08:48:01.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.156+0000 7f92e2dc1700 1 Processor -- start 2024-03-04T08:48:01.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.156+0000 7f92e2dc1700 1 -- start start 2024-03-04T08:48:01.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.157+0000 7f92e2dc1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92dc07ce40 0x7f92dc19db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:01.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.157+0000 7f92e2dc1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc10df70 con 0x7f92dc07ce40 2024-03-04T08:48:01.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.157+0000 7f92e0b5d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92dc07ce40 0x7f92dc19db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:01.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.157+0000 7f92e0b5d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92dc07ce40 0x7f92dc19db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49920/0 (socket says 172.21.15.71:49920) 2024-03-04T08:48:01.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.157+0000 7f92e0b5d700 1 -- 172.21.15.71:0/4076323583 learned_addr learned my addr 172.21.15.71:0/4076323583 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:01.188 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.157+0000 7f92e0b5d700 1 -- 172.21.15.71:0/4076323583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92dc19e040 con 0x7f92dc07ce40 2024-03-04T08:48:01.189 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.157+0000 7f92e0b5d700 1 --2- 172.21.15.71:0/4076323583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92dc07ce40 0x7f92dc19db00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f92d0011b60 tx=0x7f92d0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:01.189 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.157+0000 7f92d9ffb700 1 -- 172.21.15.71:0/4076323583 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92d0003b20 con 0x7f92dc07ce40 2024-03-04T08:48:01.189 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.157+0000 7f92d9ffb700 1 -- 172.21.15.71:0/4076323583 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92d0003cc0 con 0x7f92dc07ce40 2024-03-04T08:48:01.189 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.157+0000 7f92e2dc1700 1 -- 172.21.15.71:0/4076323583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92dc19e2d0 con 0x7f92dc07ce40 2024-03-04T08:48:01.189 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.157+0000 7f92d9ffb700 1 -- 172.21.15.71:0/4076323583 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92d0024840 con 0x7f92dc07ce40 2024-03-04T08:48:01.189 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.158+0000 7f92e2dc1700 1 -- 172.21.15.71:0/4076323583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92dc1a0fc0 con 0x7f92dc07ce40 2024-03-04T08:48:01.189 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.158+0000 7f92d9ffb700 1 -- 172.21.15.71:0/4076323583 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f92d0022020 con 0x7f92dc07ce40 2024-03-04T08:48:01.189 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.158+0000 7f92d9ffb700 1 --2- 172.21.15.71:0/4076323583 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f92cc038bf0 0x7f92cc03b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:01.189 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.158+0000 7f92e2dc1700 1 -- 172.21.15.71:0/4076323583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92dc069150 con 0x7f92dc07ce40 2024-03-04T08:48:01.189 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.159+0000 7f92d9ffb700 1 -- 172.21.15.71:0/4076323583 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f92d0011e00 con 0x7f92dc07ce40 2024-03-04T08:48:01.189 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.159+0000 7f92dbfff700 1 --2- 172.21.15.71:0/4076323583 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f92cc038bf0 0x7f92cc03b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:01.189 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.159+0000 7f92dbfff700 1 --2- 172.21.15.71:0/4076323583 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f92cc038bf0 0x7f92cc03b0b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f92c8006fd0 tx=0x7f92c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:01.189 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.164+0000 7f92d9ffb700 1 -- 172.21.15.71:0/4076323583 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f92d001a7b0 con 0x7f92dc07ce40 2024-03-04T08:48:01.189 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.355+0000 7f92e2dc1700 1 -- 172.21.15.71:0/4076323583 --> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f92dc1042a0 con 0x7f92cc038bf0 2024-03-04T08:48:01.190 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.626+0000 7f92d9ffb700 1 -- 172.21.15.71:0/4076323583 <== mgr.14120 v2:172.21.15.71:6800/2808156336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f92dc1042a0 con 0x7f92cc038bf0 2024-03-04T08:48:01.190 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.628+0000 7f92e2dc1700 1 -- 172.21.15.71:0/4076323583 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f92cc038bf0 msgr2=0x7f92cc03b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:01.190 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.628+0000 7f92e2dc1700 1 --2- 172.21.15.71:0/4076323583 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f92cc038bf0 0x7f92cc03b0b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f92c8006fd0 tx=0x7f92c8006e40 comp rx=0 tx=0).stop 2024-03-04T08:48:01.190 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.628+0000 7f92e2dc1700 1 -- 172.21.15.71:0/4076323583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92dc07ce40 msgr2=0x7f92dc19db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:01.190 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.628+0000 7f92e2dc1700 1 --2- 172.21.15.71:0/4076323583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92dc07ce40 0x7f92dc19db00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f92d0011b60 tx=0x7f92d0011c40 comp rx=0 tx=0).stop 2024-03-04T08:48:01.190 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.628+0000 7f92e2dc1700 1 -- 172.21.15.71:0/4076323583 shutdown_connections 2024-03-04T08:48:01.190 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.628+0000 7f92e2dc1700 1 --2- 172.21.15.71:0/4076323583 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f92cc038bf0 0x7f92cc03b0b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:01.190 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.628+0000 7f92e2dc1700 1 --2- 172.21.15.71:0/4076323583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92dc07ce40 0x7f92dc19db00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:01.190 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.628+0000 7f92e2dc1700 1 -- 172.21.15.71:0/4076323583 >> 172.21.15.71:0/4076323583 conn(0x7f92dc103040 msgr2=0x7f92dc103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:01.190 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.629+0000 7f92e2dc1700 1 -- 172.21.15.71:0/4076323583 shutdown_connections 2024-03-04T08:48:01.190 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:00.629+0000 7f92e2dc1700 1 -- 172.21.15.71:0/4076323583 wait complete. 2024-03-04T08:48:01.783 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:01 smithi071 ceph-mon[32789]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:48:01.783 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:01 smithi071 ceph-mon[32789]: Generating ssh key... 2024-03-04T08:48:02.402 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOlctcSGAm7SKs6telu5OvLIN3GIcaEB7pA8Hd5/LzxnOmVaKpKfd55rE7esuMKFJbqZhYWtmB7ibbkbqyIjWJcKDl/Vpu6JhkaC1B54bcRyZYhm554Um2MvXxsfEjFWRr9D7/txZwBzerdxaR11SBiKFZZbfM8wolT1ZiYNdJl7np7qoW61tnmze5R9kjj2x0HrhsjosuuISLMyBB+WthV1QgsAxSWLCc+LHek564P8kGhvod/YzLMApqLiPdyva9tx7QE/v8izOA1pQtgfhWRmBqh50KsVn5x4evoM2SaDng3aN86f5zVKez1KGoeZZaYQHbMaDHBWXunAklIujSYe/XQR8cxTl/Cod05OTWYcaIm+uKhNqDipAGePOaROQmubHf7RUo7SyzBns6ikVvqcrscHRBALKy4FqUGJUsAoMVWsqRmC9zBtL8NM13peHgf72x4kvDh64n1EmAHDKzmxuIJ+PLGt+puBdfi5cc7ZuF7TW8e0zDObp7n/T1sp8= ceph-b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T08:48:02.402 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.584+0000 7f1f27560700 1 Processor -- start 2024-03-04T08:48:02.402 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.584+0000 7f1f27560700 1 -- start start 2024-03-04T08:48:02.402 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.584+0000 7f1f27560700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f20107a20 0x7f1f20107e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:02.402 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.584+0000 7f1f27560700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f20108410 con 0x7f1f20107a20 2024-03-04T08:48:02.402 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.584+0000 7f1f252fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f20107a20 0x7f1f20107e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:02.402 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.584+0000 7f1f252fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f20107a20 0x7f1f20107e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49934/0 (socket says 172.21.15.71:49934) 2024-03-04T08:48:02.402 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.584+0000 7f1f252fc700 1 -- 172.21.15.71:0/2095386710 learned_addr learned my addr 172.21.15.71:0/2095386710 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:02.403 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.584+0000 7f1f252fc700 1 -- 172.21.15.71:0/2095386710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f20108bd0 con 0x7f1f20107a20 2024-03-04T08:48:02.403 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.585+0000 7f1f252fc700 1 --2- 172.21.15.71:0/2095386710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f20107a20 0x7f1f20107e40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0098a0 tx=0x7f1f1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5625ba3c361793 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:02.403 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.585+0000 7f1f17fff700 1 -- 172.21.15.71:0/2095386710 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f1c004160 con 0x7f1f20107a20 2024-03-04T08:48:02.403 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.585+0000 7f1f17fff700 1 -- 172.21.15.71:0/2095386710 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f1c004300 con 0x7f1f20107a20 2024-03-04T08:48:02.403 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.585+0000 7f1f17fff700 1 -- 172.21.15.71:0/2095386710 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f1c011b50 con 0x7f1f20107a20 2024-03-04T08:48:02.403 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.585+0000 7f1f27560700 1 -- 172.21.15.71:0/2095386710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f20107a20 msgr2=0x7f1f20107e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:02.403 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.585+0000 7f1f27560700 1 --2- 172.21.15.71:0/2095386710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f20107a20 0x7f1f20107e40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0098a0 tx=0x7f1f1c009bb0 comp rx=0 tx=0).stop 2024-03-04T08:48:02.403 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.585+0000 7f1f27560700 1 -- 172.21.15.71:0/2095386710 shutdown_connections 2024-03-04T08:48:02.403 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.585+0000 7f1f27560700 1 --2- 172.21.15.71:0/2095386710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f20107a20 0x7f1f20107e40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:02.403 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.585+0000 7f1f27560700 1 -- 172.21.15.71:0/2095386710 >> 172.21.15.71:0/2095386710 conn(0x7f1f20103040 msgr2=0x7f1f20105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:02.403 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.585+0000 7f1f27560700 1 -- 172.21.15.71:0/2095386710 shutdown_connections 2024-03-04T08:48:02.403 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.586+0000 7f1f27560700 1 -- 172.21.15.71:0/2095386710 wait complete. 2024-03-04T08:48:02.403 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.586+0000 7f1f27560700 1 Processor -- start 2024-03-04T08:48:02.403 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.586+0000 7f1f27560700 1 -- start start 2024-03-04T08:48:02.404 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.586+0000 7f1f27560700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f20107a20 0x7f1f2019b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:02.404 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.586+0000 7f1f27560700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2010bf80 con 0x7f1f20107a20 2024-03-04T08:48:02.404 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.586+0000 7f1f252fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f20107a20 0x7f1f2019b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:02.404 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.586+0000 7f1f252fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f20107a20 0x7f1f2019b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49942/0 (socket says 172.21.15.71:49942) 2024-03-04T08:48:02.404 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.586+0000 7f1f252fc700 1 -- 172.21.15.71:0/3958893401 learned_addr learned my addr 172.21.15.71:0/3958893401 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:02.404 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.587+0000 7f1f252fc700 1 -- 172.21.15.71:0/3958893401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2019be50 con 0x7f1f20107a20 2024-03-04T08:48:02.404 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.587+0000 7f1f252fc700 1 --2- 172.21.15.71:0/3958893401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f20107a20 0x7f1f2019b910 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c011670 tx=0x7f1f1c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:02.404 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.587+0000 7f1f167fc700 1 -- 172.21.15.71:0/3958893401 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f1c0039c0 con 0x7f1f20107a20 2024-03-04T08:48:02.404 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.587+0000 7f1f167fc700 1 -- 172.21.15.71:0/3958893401 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f1c011e50 con 0x7f1f20107a20 2024-03-04T08:48:02.404 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.587+0000 7f1f27560700 1 -- 172.21.15.71:0/3958893401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2019c0e0 con 0x7f1f20107a20 2024-03-04T08:48:02.404 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.587+0000 7f1f167fc700 1 -- 172.21.15.71:0/3958893401 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f1c01a5c0 con 0x7f1f20107a20 2024-03-04T08:48:02.404 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.587+0000 7f1f27560700 1 -- 172.21.15.71:0/3958893401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2019edd0 con 0x7f1f20107a20 2024-03-04T08:48:02.404 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.588+0000 7f1f167fc700 1 -- 172.21.15.71:0/3958893401 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f1f1c024430 con 0x7f1f20107a20 2024-03-04T08:48:02.405 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.588+0000 7f1f167fc700 1 --2- 172.21.15.71:0/3958893401 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f1f0c038b50 0x7f1f0c03b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:02.405 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.588+0000 7f1f167fc700 1 -- 172.21.15.71:0/3958893401 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1f1c04cde0 con 0x7f1f20107a20 2024-03-04T08:48:02.405 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.588+0000 7f1f24afb700 1 --2- 172.21.15.71:0/3958893401 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f1f0c038b50 0x7f1f0c03b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:02.405 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.588+0000 7f1f27560700 1 -- 172.21.15.71:0/3958893401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f20069150 con 0x7f1f20107a20 2024-03-04T08:48:02.405 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.588+0000 7f1f24afb700 1 --2- 172.21.15.71:0/3958893401 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f1f0c038b50 0x7f1f0c03b010 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1f10006fd0 tx=0x7f1f10006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:02.405 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.593+0000 7f1f167fc700 1 -- 172.21.15.71:0/3958893401 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1f1c003b60 con 0x7f1f20107a20 2024-03-04T08:48:02.405 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.787+0000 7f1f27560700 1 -- 172.21.15.71:0/3958893401 --> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f20104220 con 0x7f1f0c038b50 2024-03-04T08:48:02.405 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.787+0000 7f1f167fc700 1 -- 172.21.15.71:0/3958893401 <== mgr.14120 v2:172.21.15.71:6800/2808156336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f1f20104220 con 0x7f1f0c038b50 2024-03-04T08:48:02.405 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.789+0000 7f1f27560700 1 -- 172.21.15.71:0/3958893401 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f1f0c038b50 msgr2=0x7f1f0c03b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:02.405 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.789+0000 7f1f27560700 1 --2- 172.21.15.71:0/3958893401 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f1f0c038b50 0x7f1f0c03b010 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1f10006fd0 tx=0x7f1f10006e40 comp rx=0 tx=0).stop 2024-03-04T08:48:02.405 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.789+0000 7f1f27560700 1 -- 172.21.15.71:0/3958893401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f20107a20 msgr2=0x7f1f2019b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:02.405 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.789+0000 7f1f27560700 1 --2- 172.21.15.71:0/3958893401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f20107a20 0x7f1f2019b910 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c011670 tx=0x7f1f1c011b60 comp rx=0 tx=0).stop 2024-03-04T08:48:02.406 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.789+0000 7f1f27560700 1 -- 172.21.15.71:0/3958893401 shutdown_connections 2024-03-04T08:48:02.406 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.789+0000 7f1f27560700 1 --2- 172.21.15.71:0/3958893401 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f1f0c038b50 0x7f1f0c03b010 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:02.406 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.789+0000 7f1f27560700 1 --2- 172.21.15.71:0/3958893401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f20107a20 0x7f1f2019b910 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:02.406 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.789+0000 7f1f27560700 1 -- 172.21.15.71:0/3958893401 >> 172.21.15.71:0/3958893401 conn(0x7f1f20103040 msgr2=0x7f1f20103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:02.406 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.790+0000 7f1f27560700 1 -- 172.21.15.71:0/3958893401 shutdown_connections 2024-03-04T08:48:02.406 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:01.790+0000 7f1f27560700 1 -- 172.21.15.71:0/3958893401 wait complete. 2024-03-04T08:48:02.406 INFO:teuthology.orchestra.run.smithi071.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-04T08:48:02.406 INFO:teuthology.orchestra.run.smithi071.stdout:Adding key to root@localhost authorized_keys... 2024-03-04T08:48:02.406 INFO:teuthology.orchestra.run.smithi071.stdout:Adding host smithi071... 2024-03-04T08:48:02.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:02 smithi071 ceph-mon[32789]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:48:03.914 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:03 smithi071 ceph-mon[32789]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi071", "addr": "172.21.15.71", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:48:03.914 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:03 smithi071 ceph-mon[32789]: Deploying cephadm binary to smithi071 2024-03-04T08:48:07.901 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Added host 'smithi071' with addr '172.21.15.71' 2024-03-04T08:48:07.901 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.852+0000 7f5bb5103700 1 Processor -- start 2024-03-04T08:48:07.901 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.852+0000 7f5bb5103700 1 -- start start 2024-03-04T08:48:07.901 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.852+0000 7f5bb5103700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bb0105d30 0x7f5bb0106150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:07.901 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.852+0000 7f5bb5103700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb0106720 con 0x7f5bb0105d30 2024-03-04T08:48:07.901 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.852+0000 7f5baed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bb0105d30 0x7f5bb0106150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:07.901 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.852+0000 7f5baed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bb0105d30 0x7f5bb0106150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49944/0 (socket says 172.21.15.71:49944) 2024-03-04T08:48:07.901 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.853+0000 7f5baed9d700 1 -- 172.21.15.71:0/1443819752 learned_addr learned my addr 172.21.15.71:0/1443819752 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:07.901 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.853+0000 7f5baed9d700 1 -- 172.21.15.71:0/1443819752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb0108cc0 con 0x7f5bb0105d30 2024-03-04T08:48:07.902 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.853+0000 7f5baed9d700 1 --2- 172.21.15.71:0/1443819752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bb0105d30 0x7f5bb0106150 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5ba00098a0 tx=0x7f5ba0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be34da0e06df0257 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:07.902 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.853+0000 7f5bae59c700 1 -- 172.21.15.71:0/1443819752 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ba0004160 con 0x7f5bb0105d30 2024-03-04T08:48:07.902 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.853+0000 7f5bae59c700 1 -- 172.21.15.71:0/1443819752 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ba0004300 con 0x7f5bb0105d30 2024-03-04T08:48:07.902 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.853+0000 7f5bae59c700 1 -- 172.21.15.71:0/1443819752 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ba0011b50 con 0x7f5bb0105d30 2024-03-04T08:48:07.902 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.854+0000 7f5bb5103700 1 -- 172.21.15.71:0/1443819752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bb0105d30 msgr2=0x7f5bb0106150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:07.902 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.854+0000 7f5bb5103700 1 --2- 172.21.15.71:0/1443819752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bb0105d30 0x7f5bb0106150 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5ba00098a0 tx=0x7f5ba0009bb0 comp rx=0 tx=0).stop 2024-03-04T08:48:07.902 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.854+0000 7f5bb5103700 1 -- 172.21.15.71:0/1443819752 shutdown_connections 2024-03-04T08:48:07.902 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.854+0000 7f5bb5103700 1 --2- 172.21.15.71:0/1443819752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bb0105d30 0x7f5bb0106150 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:07.902 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.854+0000 7f5bb5103700 1 -- 172.21.15.71:0/1443819752 >> 172.21.15.71:0/1443819752 conn(0x7f5bb01016a0 msgr2=0x7f5bb0103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:07.902 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.854+0000 7f5bb5103700 1 -- 172.21.15.71:0/1443819752 shutdown_connections 2024-03-04T08:48:07.902 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.854+0000 7f5bb5103700 1 -- 172.21.15.71:0/1443819752 wait complete. 2024-03-04T08:48:07.902 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.854+0000 7f5bb5103700 1 Processor -- start 2024-03-04T08:48:07.902 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.855+0000 7f5bb5103700 1 -- start start 2024-03-04T08:48:07.902 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.855+0000 7f5bb5103700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bb0105d30 0x7f5bb0108540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:07.903 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.855+0000 7f5bb5103700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb010c070 con 0x7f5bb0105d30 2024-03-04T08:48:07.903 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.855+0000 7f5baed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bb0105d30 0x7f5bb0108540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:07.903 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.855+0000 7f5baed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bb0105d30 0x7f5bb0108540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49952/0 (socket says 172.21.15.71:49952) 2024-03-04T08:48:07.903 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.855+0000 7f5baed9d700 1 -- 172.21.15.71:0/313334297 learned_addr learned my addr 172.21.15.71:0/313334297 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:07.903 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.855+0000 7f5baed9d700 1 -- 172.21.15.71:0/313334297 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb0108a80 con 0x7f5bb0105d30 2024-03-04T08:48:07.903 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.855+0000 7f5baed9d700 1 --2- 172.21.15.71:0/313334297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bb0105d30 0x7f5bb0108540 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009fb0 tx=0x7f5ba0011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:07.903 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.855+0000 7f5bacd99700 1 -- 172.21.15.71:0/313334297 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ba00038b0 con 0x7f5bb0105d30 2024-03-04T08:48:07.903 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.855+0000 7f5bacd99700 1 -- 172.21.15.71:0/313334297 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ba0011e50 con 0x7f5bb0105d30 2024-03-04T08:48:07.903 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.855+0000 7f5bb5103700 1 -- 172.21.15.71:0/313334297 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb0106c70 con 0x7f5bb0105d30 2024-03-04T08:48:07.903 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.855+0000 7f5bacd99700 1 -- 172.21.15.71:0/313334297 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ba0019550 con 0x7f5bb0105d30 2024-03-04T08:48:07.903 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.855+0000 7f5bb5103700 1 -- 172.21.15.71:0/313334297 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb01071b0 con 0x7f5bb0105d30 2024-03-04T08:48:07.903 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.856+0000 7f5bacd99700 1 -- 172.21.15.71:0/313334297 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f5ba0019800 con 0x7f5bb0105d30 2024-03-04T08:48:07.903 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.856+0000 7f5bb5103700 1 -- 172.21.15.71:0/313334297 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bb00636f0 con 0x7f5bb0105d30 2024-03-04T08:48:07.903 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.856+0000 7f5bacd99700 1 --2- 172.21.15.71:0/313334297 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f5b9c038ba0 0x7f5b9c03b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:07.904 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.856+0000 7f5bacd99700 1 -- 172.21.15.71:0/313334297 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5ba004cc80 con 0x7f5bb0105d30 2024-03-04T08:48:07.904 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.856+0000 7f5ba7fff700 1 --2- 172.21.15.71:0/313334297 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f5b9c038ba0 0x7f5b9c03b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:07.904 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.857+0000 7f5ba7fff700 1 --2- 172.21.15.71:0/313334297 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f5b9c038ba0 0x7f5b9c03b060 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5b98006fd0 tx=0x7f5b98006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:07.904 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:02.861+0000 7f5bacd99700 1 -- 172.21.15.71:0/313334297 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5ba0053080 con 0x7f5bb0105d30 2024-03-04T08:48:07.904 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:03.057+0000 7f5bb5103700 1 -- 172.21.15.71:0/313334297 --> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi071", "addr": "172.21.15.71", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bb0102950 con 0x7f5b9c038ba0 2024-03-04T08:48:07.904 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:07.284+0000 7f5bacd99700 1 -- 172.21.15.71:0/313334297 <== mgr.14120 v2:172.21.15.71:6800/2808156336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f5bb0102950 con 0x7f5b9c038ba0 2024-03-04T08:48:07.904 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:07.287+0000 7f5bb5103700 1 -- 172.21.15.71:0/313334297 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f5b9c038ba0 msgr2=0x7f5b9c03b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:07.904 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:07.287+0000 7f5bb5103700 1 --2- 172.21.15.71:0/313334297 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f5b9c038ba0 0x7f5b9c03b060 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5b98006fd0 tx=0x7f5b98006e40 comp rx=0 tx=0).stop 2024-03-04T08:48:07.904 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:07.287+0000 7f5bb5103700 1 -- 172.21.15.71:0/313334297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bb0105d30 msgr2=0x7f5bb0108540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:07.904 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:07.287+0000 7f5bb5103700 1 --2- 172.21.15.71:0/313334297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bb0105d30 0x7f5bb0108540 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009fb0 tx=0x7f5ba0011bf0 comp rx=0 tx=0).stop 2024-03-04T08:48:07.904 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:07.287+0000 7f5bb5103700 1 -- 172.21.15.71:0/313334297 shutdown_connections 2024-03-04T08:48:07.904 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:07.287+0000 7f5bb5103700 1 --2- 172.21.15.71:0/313334297 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f5b9c038ba0 0x7f5b9c03b060 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:07.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:07.287+0000 7f5bb5103700 1 --2- 172.21.15.71:0/313334297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bb0105d30 0x7f5bb0108540 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:07.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:07.288+0000 7f5bb5103700 1 -- 172.21.15.71:0/313334297 >> 172.21.15.71:0/313334297 conn(0x7f5bb01016a0 msgr2=0x7f5bb0102240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:07.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:07.288+0000 7f5bb5103700 1 -- 172.21.15.71:0/313334297 shutdown_connections 2024-03-04T08:48:07.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:07.288+0000 7f5bb5103700 1 -- 172.21.15.71:0/313334297 wait complete. 2024-03-04T08:48:07.905 INFO:teuthology.orchestra.run.smithi071.stdout:Deploying unmanaged mon service... 2024-03-04T08:48:08.527 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:08 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' 2024-03-04T08:48:08.527 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:08 smithi071 ceph-mon[32789]: Added host smithi071 2024-03-04T08:48:08.527 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:08 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:48:09.134 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-04T08:48:09.134 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.310+0000 7f936f5b1700 1 Processor -- start 2024-03-04T08:48:09.134 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.310+0000 7f936f5b1700 1 -- start start 2024-03-04T08:48:09.134 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.310+0000 7f936f5b1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f936807ce20 0x7f936807b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:09.134 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.310+0000 7f936f5b1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936807b810 con 0x7f936807ce20 2024-03-04T08:48:09.134 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.310+0000 7f936d34d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f936807ce20 0x7f936807b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:09.134 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.310+0000 7f936d34d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f936807ce20 0x7f936807b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34394/0 (socket says 172.21.15.71:34394) 2024-03-04T08:48:09.134 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.310+0000 7f936d34d700 1 -- 172.21.15.71:0/511976549 learned_addr learned my addr 172.21.15.71:0/511976549 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:09.134 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.311+0000 7f936d34d700 1 -- 172.21.15.71:0/511976549 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936807b990 con 0x7f936807ce20 2024-03-04T08:48:09.135 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.311+0000 7f936d34d700 1 --2- 172.21.15.71:0/511976549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f936807ce20 0x7f936807b2d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f93640098a0 tx=0x7f9364009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4289aaef2e4500ef server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:09.135 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.311+0000 7f935ffff700 1 -- 172.21.15.71:0/511976549 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9364004160 con 0x7f936807ce20 2024-03-04T08:48:09.135 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.311+0000 7f935ffff700 1 -- 172.21.15.71:0/511976549 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9364004300 con 0x7f936807ce20 2024-03-04T08:48:09.135 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.311+0000 7f935ffff700 1 -- 172.21.15.71:0/511976549 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9364011b50 con 0x7f936807ce20 2024-03-04T08:48:09.135 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.311+0000 7f936f5b1700 1 -- 172.21.15.71:0/511976549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f936807ce20 msgr2=0x7f936807b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:09.135 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.311+0000 7f936f5b1700 1 --2- 172.21.15.71:0/511976549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f936807ce20 0x7f936807b2d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f93640098a0 tx=0x7f9364009bb0 comp rx=0 tx=0).stop 2024-03-04T08:48:09.135 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.311+0000 7f936f5b1700 1 -- 172.21.15.71:0/511976549 shutdown_connections 2024-03-04T08:48:09.135 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.311+0000 7f936f5b1700 1 --2- 172.21.15.71:0/511976549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f936807ce20 0x7f936807b2d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:09.135 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.311+0000 7f936f5b1700 1 -- 172.21.15.71:0/511976549 >> 172.21.15.71:0/511976549 conn(0x7f9368103040 msgr2=0x7f9368105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:09.135 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.312+0000 7f936f5b1700 1 -- 172.21.15.71:0/511976549 shutdown_connections 2024-03-04T08:48:09.135 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.312+0000 7f936f5b1700 1 -- 172.21.15.71:0/511976549 wait complete. 2024-03-04T08:48:09.135 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.312+0000 7f936f5b1700 1 Processor -- start 2024-03-04T08:48:09.135 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.312+0000 7f936f5b1700 1 -- start start 2024-03-04T08:48:09.136 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.312+0000 7f936f5b1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f936807ce20 0x7f936819db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:09.136 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.312+0000 7f936f5b1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936810df30 con 0x7f936807ce20 2024-03-04T08:48:09.136 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.312+0000 7f936d34d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f936807ce20 0x7f936819db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:09.136 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.313+0000 7f936d34d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f936807ce20 0x7f936819db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34410/0 (socket says 172.21.15.71:34410) 2024-03-04T08:48:09.136 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.313+0000 7f936d34d700 1 -- 172.21.15.71:0/2797899998 learned_addr learned my addr 172.21.15.71:0/2797899998 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:09.136 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.313+0000 7f936d34d700 1 -- 172.21.15.71:0/2797899998 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936819e050 con 0x7f936807ce20 2024-03-04T08:48:09.136 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.313+0000 7f936d34d700 1 --2- 172.21.15.71:0/2797899998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f936807ce20 0x7f936819db10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9364011b60 tx=0x7f9364011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:09.136 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.313+0000 7f935e7fc700 1 -- 172.21.15.71:0/2797899998 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9364003b20 con 0x7f936807ce20 2024-03-04T08:48:09.136 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.313+0000 7f935e7fc700 1 -- 172.21.15.71:0/2797899998 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9364003cc0 con 0x7f936807ce20 2024-03-04T08:48:09.136 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.313+0000 7f936f5b1700 1 -- 172.21.15.71:0/2797899998 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f936819e2e0 con 0x7f936807ce20 2024-03-04T08:48:09.136 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.313+0000 7f935e7fc700 1 -- 172.21.15.71:0/2797899998 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9364024840 con 0x7f936807ce20 2024-03-04T08:48:09.136 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.313+0000 7f936f5b1700 1 -- 172.21.15.71:0/2797899998 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93681a0fd0 con 0x7f936807ce20 2024-03-04T08:48:09.136 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.314+0000 7f935e7fc700 1 -- 172.21.15.71:0/2797899998 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f9364022020 con 0x7f936807ce20 2024-03-04T08:48:09.136 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.314+0000 7f936f5b1700 1 -- 172.21.15.71:0/2797899998 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93680636f0 con 0x7f936807ce20 2024-03-04T08:48:09.137 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.314+0000 7f935e7fc700 1 --2- 172.21.15.71:0/2797899998 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f9354038bf0 0x7f935403b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:09.137 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.314+0000 7f935e7fc700 1 -- 172.21.15.71:0/2797899998 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9364011e00 con 0x7f936807ce20 2024-03-04T08:48:09.137 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.314+0000 7f936cb4c700 1 --2- 172.21.15.71:0/2797899998 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f9354038bf0 0x7f935403b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:09.137 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.315+0000 7f936cb4c700 1 --2- 172.21.15.71:0/2797899998 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f9354038bf0 0x7f935403b0b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9358006fd0 tx=0x7f9358006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:09.137 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.319+0000 7f935e7fc700 1 -- 172.21.15.71:0/2797899998 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f936400b9e0 con 0x7f936807ce20 2024-03-04T08:48:09.137 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.530+0000 7f936f5b1700 1 -- 172.21.15.71:0/2797899998 --> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f9368104260 con 0x7f9354038bf0 2024-03-04T08:48:09.137 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.535+0000 7f935e7fc700 1 -- 172.21.15.71:0/2797899998 <== mgr.14120 v2:172.21.15.71:6800/2808156336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f9368104260 con 0x7f9354038bf0 2024-03-04T08:48:09.137 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.537+0000 7f936f5b1700 1 -- 172.21.15.71:0/2797899998 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f9354038bf0 msgr2=0x7f935403b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:09.137 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.537+0000 7f936f5b1700 1 --2- 172.21.15.71:0/2797899998 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f9354038bf0 0x7f935403b0b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9358006fd0 tx=0x7f9358006e40 comp rx=0 tx=0).stop 2024-03-04T08:48:09.137 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.537+0000 7f936f5b1700 1 -- 172.21.15.71:0/2797899998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f936807ce20 msgr2=0x7f936819db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:09.137 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.537+0000 7f936f5b1700 1 --2- 172.21.15.71:0/2797899998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f936807ce20 0x7f936819db10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9364011b60 tx=0x7f9364011c40 comp rx=0 tx=0).stop 2024-03-04T08:48:09.137 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.537+0000 7f936f5b1700 1 -- 172.21.15.71:0/2797899998 shutdown_connections 2024-03-04T08:48:09.137 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.537+0000 7f936f5b1700 1 --2- 172.21.15.71:0/2797899998 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f9354038bf0 0x7f935403b0b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:09.137 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.537+0000 7f936f5b1700 1 --2- 172.21.15.71:0/2797899998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f936807ce20 0x7f936819db10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:09.138 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.537+0000 7f936f5b1700 1 -- 172.21.15.71:0/2797899998 >> 172.21.15.71:0/2797899998 conn(0x7f9368103040 msgr2=0x7f9368103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:09.138 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.537+0000 7f936f5b1700 1 -- 172.21.15.71:0/2797899998 shutdown_connections 2024-03-04T08:48:09.138 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:08.537+0000 7f936f5b1700 1 -- 172.21.15.71:0/2797899998 wait complete. 2024-03-04T08:48:09.138 INFO:teuthology.orchestra.run.smithi071.stdout:Deploying unmanaged mgr service... 2024-03-04T08:48:09.785 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:09 smithi071 ceph-mon[32789]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:48:09.785 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:09 smithi071 ceph-mon[32789]: Saving service mon spec with placement count:5 2024-03-04T08:48:09.785 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:09 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' 2024-03-04T08:48:11.373 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:11 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' 2024-03-04T08:48:11.373 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:11 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' 2024-03-04T08:48:11.373 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:11 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' 2024-03-04T08:48:11.380 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-04T08:48:11.380 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.543+0000 7fe386878700 1 Processor -- start 2024-03-04T08:48:11.380 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.543+0000 7fe386878700 1 -- start start 2024-03-04T08:48:11.380 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.543+0000 7fe386878700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe380108480 0x7fe38010a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:11.380 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.543+0000 7fe386878700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38007aea0 con 0x7fe380108480 2024-03-04T08:48:11.380 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.543+0000 7fe37ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe380108480 0x7fe38010a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:11.380 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.544+0000 7fe37ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe380108480 0x7fe38010a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34422/0 (socket says 172.21.15.71:34422) 2024-03-04T08:48:11.380 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.544+0000 7fe37ffff700 1 -- 172.21.15.71:0/418166510 learned_addr learned my addr 172.21.15.71:0/418166510 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:11.381 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.544+0000 7fe37ffff700 1 -- 172.21.15.71:0/418166510 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38010adb0 con 0x7fe380108480 2024-03-04T08:48:11.381 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.544+0000 7fe37ffff700 1 --2- 172.21.15.71:0/418166510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe380108480 0x7fe38010a870 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe3700098a0 tx=0x7fe370009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77c3fba1e8a69c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:11.381 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.544+0000 7fe37effd700 1 -- 172.21.15.71:0/418166510 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe370004160 con 0x7fe380108480 2024-03-04T08:48:11.381 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.544+0000 7fe37effd700 1 -- 172.21.15.71:0/418166510 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe370004300 con 0x7fe380108480 2024-03-04T08:48:11.381 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.544+0000 7fe37effd700 1 -- 172.21.15.71:0/418166510 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe370011b20 con 0x7fe380108480 2024-03-04T08:48:11.381 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.545+0000 7fe386878700 1 -- 172.21.15.71:0/418166510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe380108480 msgr2=0x7fe38010a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:11.381 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.545+0000 7fe386878700 1 --2- 172.21.15.71:0/418166510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe380108480 0x7fe38010a870 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe3700098a0 tx=0x7fe370009bb0 comp rx=0 tx=0).stop 2024-03-04T08:48:11.381 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.545+0000 7fe386878700 1 -- 172.21.15.71:0/418166510 shutdown_connections 2024-03-04T08:48:11.381 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.545+0000 7fe386878700 1 --2- 172.21.15.71:0/418166510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe380108480 0x7fe38010a870 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:11.381 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.545+0000 7fe386878700 1 -- 172.21.15.71:0/418166510 >> 172.21.15.71:0/418166510 conn(0x7fe380101f70 msgr2=0x7fe380104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:11.382 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.545+0000 7fe386878700 1 -- 172.21.15.71:0/418166510 shutdown_connections 2024-03-04T08:48:11.382 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.545+0000 7fe386878700 1 -- 172.21.15.71:0/418166510 wait complete. 2024-03-04T08:48:11.382 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.546+0000 7fe386878700 1 Processor -- start 2024-03-04T08:48:11.382 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.546+0000 7fe386878700 1 -- start start 2024-03-04T08:48:11.383 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.546+0000 7fe386878700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe380108480 0x7fe38019b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:11.383 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.546+0000 7fe386878700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38010be20 con 0x7fe380108480 2024-03-04T08:48:11.383 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.546+0000 7fe37ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe380108480 0x7fe38019b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:11.383 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.546+0000 7fe37ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe380108480 0x7fe38019b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34424/0 (socket says 172.21.15.71:34424) 2024-03-04T08:48:11.383 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.546+0000 7fe37ffff700 1 -- 172.21.15.71:0/2165762901 learned_addr learned my addr 172.21.15.71:0/2165762901 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:11.383 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.547+0000 7fe37ffff700 1 -- 172.21.15.71:0/2165762901 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38019be20 con 0x7fe380108480 2024-03-04T08:48:11.383 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.547+0000 7fe37ffff700 1 --2- 172.21.15.71:0/2165762901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe380108480 0x7fe38019b8e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe3700117e0 tx=0x7fe370011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:11.383 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.547+0000 7fe37d7fa700 1 -- 172.21.15.71:0/2165762901 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe370003bf0 con 0x7fe380108480 2024-03-04T08:48:11.383 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.547+0000 7fe37d7fa700 1 -- 172.21.15.71:0/2165762901 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe37001a910 con 0x7fe380108480 2024-03-04T08:48:11.383 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.547+0000 7fe386878700 1 -- 172.21.15.71:0/2165762901 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe38019c0b0 con 0x7fe380108480 2024-03-04T08:48:11.383 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.547+0000 7fe37d7fa700 1 -- 172.21.15.71:0/2165762901 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe370024970 con 0x7fe380108480 2024-03-04T08:48:11.383 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.547+0000 7fe386878700 1 -- 172.21.15.71:0/2165762901 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe38019c510 con 0x7fe380108480 2024-03-04T08:48:11.383 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.548+0000 7fe386878700 1 -- 172.21.15.71:0/2165762901 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3800636f0 con 0x7fe380108480 2024-03-04T08:48:11.383 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.548+0000 7fe37d7fa700 1 -- 172.21.15.71:0/2165762901 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7fe370022020 con 0x7fe380108480 2024-03-04T08:48:11.384 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.548+0000 7fe37d7fa700 1 --2- 172.21.15.71:0/2165762901 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fe368038b50 0x7fe36803b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:11.384 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.548+0000 7fe37d7fa700 1 -- 172.21.15.71:0/2165762901 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe37004cd80 con 0x7fe380108480 2024-03-04T08:48:11.384 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.549+0000 7fe37f7fe700 1 --2- 172.21.15.71:0/2165762901 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fe368038b50 0x7fe36803b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:11.384 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.549+0000 7fe37f7fe700 1 --2- 172.21.15.71:0/2165762901 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fe368038b50 0x7fe36803b010 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe374006fd0 tx=0x7fe374006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:11.384 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.553+0000 7fe37d7fa700 1 -- 172.21.15.71:0/2165762901 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe370011e20 con 0x7fe380108480 2024-03-04T08:48:11.384 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.768+0000 7fe386878700 1 -- 172.21.15.71:0/2165762901 --> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fe380103120 con 0x7fe368038b50 2024-03-04T08:48:11.384 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.774+0000 7fe37d7fa700 1 -- 172.21.15.71:0/2165762901 <== mgr.14120 v2:172.21.15.71:6800/2808156336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fe380103120 con 0x7fe368038b50 2024-03-04T08:48:11.384 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.776+0000 7fe386878700 1 -- 172.21.15.71:0/2165762901 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fe368038b50 msgr2=0x7fe36803b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:11.384 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.776+0000 7fe386878700 1 --2- 172.21.15.71:0/2165762901 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fe368038b50 0x7fe36803b010 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe374006fd0 tx=0x7fe374006e40 comp rx=0 tx=0).stop 2024-03-04T08:48:11.384 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.776+0000 7fe386878700 1 -- 172.21.15.71:0/2165762901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe380108480 msgr2=0x7fe38019b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:11.384 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.776+0000 7fe386878700 1 --2- 172.21.15.71:0/2165762901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe380108480 0x7fe38019b8e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe3700117e0 tx=0x7fe370011c10 comp rx=0 tx=0).stop 2024-03-04T08:48:11.384 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.776+0000 7fe386878700 1 -- 172.21.15.71:0/2165762901 shutdown_connections 2024-03-04T08:48:11.384 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.776+0000 7fe386878700 1 --2- 172.21.15.71:0/2165762901 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fe368038b50 0x7fe36803b010 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:11.385 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.776+0000 7fe386878700 1 --2- 172.21.15.71:0/2165762901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe380108480 0x7fe38019b8e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:11.385 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.776+0000 7fe386878700 1 -- 172.21.15.71:0/2165762901 >> 172.21.15.71:0/2165762901 conn(0x7fe380101f70 msgr2=0x7fe380102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:11.385 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.776+0000 7fe386878700 1 -- 172.21.15.71:0/2165762901 shutdown_connections 2024-03-04T08:48:11.385 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:10.776+0000 7fe386878700 1 -- 172.21.15.71:0/2165762901 wait complete. 2024-03-04T08:48:12.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:12 smithi071 ceph-mon[32789]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:48:12.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:12 smithi071 ceph-mon[32789]: Saving service mgr spec with placement count:2 2024-03-04T08:48:12.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:12 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/99761312' entity='client.admin' 2024-03-04T08:48:13.131 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.821+0000 7f9e69828700 1 Processor -- start 2024-03-04T08:48:13.131 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.821+0000 7f9e69828700 1 -- start start 2024-03-04T08:48:13.132 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.821+0000 7f9e69828700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e64106910 0x7f9e64106d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:13.132 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.821+0000 7f9e69828700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e64107300 con 0x7f9e64106910 2024-03-04T08:48:13.132 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.821+0000 7f9e62ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e64106910 0x7f9e64106d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:13.132 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.821+0000 7f9e62ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e64106910 0x7f9e64106d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34430/0 (socket says 172.21.15.71:34430) 2024-03-04T08:48:13.132 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.821+0000 7f9e62ffd700 1 -- 172.21.15.71:0/1346764797 learned_addr learned my addr 172.21.15.71:0/1346764797 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:13.132 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.821+0000 7f9e62ffd700 1 -- 172.21.15.71:0/1346764797 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e64107b10 con 0x7f9e64106910 2024-03-04T08:48:13.132 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.822+0000 7f9e62ffd700 1 --2- 172.21.15.71:0/1346764797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e64106910 0x7f9e64106d30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9e540098a0 tx=0x7f9e54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605d7939b00f2236 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:13.132 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.822+0000 7f9e61ffb700 1 -- 172.21.15.71:0/1346764797 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e54004160 con 0x7f9e64106910 2024-03-04T08:48:13.132 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.822+0000 7f9e61ffb700 1 -- 172.21.15.71:0/1346764797 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e54004300 con 0x7f9e64106910 2024-03-04T08:48:13.132 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.822+0000 7f9e61ffb700 1 -- 172.21.15.71:0/1346764797 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e54011b20 con 0x7f9e64106910 2024-03-04T08:48:13.133 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.822+0000 7f9e69828700 1 -- 172.21.15.71:0/1346764797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e64106910 msgr2=0x7f9e64106d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:13.133 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.822+0000 7f9e69828700 1 --2- 172.21.15.71:0/1346764797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e64106910 0x7f9e64106d30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9e540098a0 tx=0x7f9e54009bb0 comp rx=0 tx=0).stop 2024-03-04T08:48:13.133 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.822+0000 7f9e69828700 1 -- 172.21.15.71:0/1346764797 shutdown_connections 2024-03-04T08:48:13.133 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.822+0000 7f9e69828700 1 --2- 172.21.15.71:0/1346764797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e64106910 0x7f9e64106d30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:13.133 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.822+0000 7f9e69828700 1 -- 172.21.15.71:0/1346764797 >> 172.21.15.71:0/1346764797 conn(0x7f9e64101f90 msgr2=0x7f9e641043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:13.133 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.822+0000 7f9e69828700 1 -- 172.21.15.71:0/1346764797 shutdown_connections 2024-03-04T08:48:13.133 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.822+0000 7f9e69828700 1 -- 172.21.15.71:0/1346764797 wait complete. 2024-03-04T08:48:13.133 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.823+0000 7f9e69828700 1 Processor -- start 2024-03-04T08:48:13.133 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.823+0000 7f9e69828700 1 -- start start 2024-03-04T08:48:13.133 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.823+0000 7f9e69828700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e64106910 0x7f9e6407aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:13.134 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.823+0000 7f9e69828700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6410aec0 con 0x7f9e64106910 2024-03-04T08:48:13.134 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.823+0000 7f9e62ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e64106910 0x7f9e6407aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:13.134 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.823+0000 7f9e62ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e64106910 0x7f9e6407aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34440/0 (socket says 172.21.15.71:34440) 2024-03-04T08:48:13.134 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.823+0000 7f9e62ffd700 1 -- 172.21.15.71:0/99761312 learned_addr learned my addr 172.21.15.71:0/99761312 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:13.134 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.824+0000 7f9e62ffd700 1 -- 172.21.15.71:0/99761312 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6419c480 con 0x7f9e64106910 2024-03-04T08:48:13.134 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.824+0000 7f9e62ffd700 1 --2- 172.21.15.71:0/99761312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e64106910 0x7f9e6407aa40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9e540116a0 tx=0x7f9e54011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:13.134 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.824+0000 7f9e68826700 1 -- 172.21.15.71:0/99761312 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e54003a40 con 0x7f9e64106910 2024-03-04T08:48:13.134 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.824+0000 7f9e68826700 1 -- 172.21.15.71:0/99761312 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e54003be0 con 0x7f9e64106910 2024-03-04T08:48:13.134 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.824+0000 7f9e69828700 1 -- 172.21.15.71:0/99761312 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e641a0a90 con 0x7f9e64106910 2024-03-04T08:48:13.134 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.824+0000 7f9e68826700 1 -- 172.21.15.71:0/99761312 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e5401a720 con 0x7f9e64106910 2024-03-04T08:48:13.134 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.824+0000 7f9e69828700 1 -- 172.21.15.71:0/99761312 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e641a0e30 con 0x7f9e64106910 2024-03-04T08:48:13.134 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.825+0000 7f9e68826700 1 -- 172.21.15.71:0/99761312 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f9e5401aa40 con 0x7f9e64106910 2024-03-04T08:48:13.134 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.825+0000 7f9e69828700 1 -- 172.21.15.71:0/99761312 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e64069150 con 0x7f9e64106910 2024-03-04T08:48:13.135 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.825+0000 7f9e68826700 1 --2- 172.21.15.71:0/99761312 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f9e4c038b50 0x7f9e4c03b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:13.135 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.825+0000 7f9e68826700 1 -- 172.21.15.71:0/99761312 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9e5404d1d0 con 0x7f9e64106910 2024-03-04T08:48:13.135 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.825+0000 7f9e627fc700 1 --2- 172.21.15.71:0/99761312 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f9e4c038b50 0x7f9e4c03b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:13.135 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.825+0000 7f9e627fc700 1 --2- 172.21.15.71:0/99761312 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f9e4c038b50 0x7f9e4c03b010 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9e58006fd0 tx=0x7f9e58006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:13.135 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:11.830+0000 7f9e68826700 1 -- 172.21.15.71:0/99761312 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9e54011e20 con 0x7f9e64106910 2024-03-04T08:48:13.135 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:12.042+0000 7f9e69828700 1 -- 172.21.15.71:0/99761312 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f9e641a1570 con 0x7f9e64106910 2024-03-04T08:48:13.135 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:12.045+0000 7f9e68826700 1 -- 172.21.15.71:0/99761312 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f9e5401ade0 con 0x7f9e64106910 2024-03-04T08:48:13.135 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:12.047+0000 7f9e69828700 1 -- 172.21.15.71:0/99761312 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f9e4c038b50 msgr2=0x7f9e4c03b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:13.135 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:12.047+0000 7f9e69828700 1 --2- 172.21.15.71:0/99761312 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f9e4c038b50 0x7f9e4c03b010 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9e58006fd0 tx=0x7f9e58006e40 comp rx=0 tx=0).stop 2024-03-04T08:48:13.135 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:12.047+0000 7f9e69828700 1 -- 172.21.15.71:0/99761312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e64106910 msgr2=0x7f9e6407aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:13.135 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:12.047+0000 7f9e69828700 1 --2- 172.21.15.71:0/99761312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e64106910 0x7f9e6407aa40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9e540116a0 tx=0x7f9e54011ba0 comp rx=0 tx=0).stop 2024-03-04T08:48:13.136 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:12.047+0000 7f9e69828700 1 -- 172.21.15.71:0/99761312 shutdown_connections 2024-03-04T08:48:13.136 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:12.047+0000 7f9e69828700 1 --2- 172.21.15.71:0/99761312 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f9e4c038b50 0x7f9e4c03b010 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:13.136 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:12.047+0000 7f9e69828700 1 --2- 172.21.15.71:0/99761312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e64106910 0x7f9e6407aa40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:13.136 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:12.047+0000 7f9e69828700 1 -- 172.21.15.71:0/99761312 >> 172.21.15.71:0/99761312 conn(0x7f9e64101f90 msgr2=0x7f9e64102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:13.136 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:12.047+0000 7f9e69828700 1 -- 172.21.15.71:0/99761312 shutdown_connections 2024-03-04T08:48:13.136 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:12.047+0000 7f9e69828700 1 -- 172.21.15.71:0/99761312 wait complete. 2024-03-04T08:48:13.838 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:13 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' 2024-03-04T08:48:14.407 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.619+0000 7f931cf1b700 1 Processor -- start 2024-03-04T08:48:14.407 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.619+0000 7f931cf1b700 1 -- start start 2024-03-04T08:48:14.407 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.619+0000 7f931cf1b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9318106240 0x7f9318106660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:14.408 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.619+0000 7f931cf1b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931807ae00 con 0x7f9318106240 2024-03-04T08:48:14.408 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.619+0000 7f931659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9318106240 0x7f9318106660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:14.408 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.619+0000 7f931659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9318106240 0x7f9318106660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34456/0 (socket says 172.21.15.71:34456) 2024-03-04T08:48:14.408 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.619+0000 7f931659c700 1 -- 172.21.15.71:0/429321968 learned_addr learned my addr 172.21.15.71:0/429321968 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:14.408 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.619+0000 7f931659c700 1 -- 172.21.15.71:0/429321968 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9318106ba0 con 0x7f9318106240 2024-03-04T08:48:14.408 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.619+0000 7f931659c700 1 --2- 172.21.15.71:0/429321968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9318106240 0x7f9318106660 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f93080098a0 tx=0x7f9308009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f8b1a272cc36e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:14.408 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.620+0000 7f931559a700 1 -- 172.21.15.71:0/429321968 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9308004160 con 0x7f9318106240 2024-03-04T08:48:14.408 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.620+0000 7f931559a700 1 -- 172.21.15.71:0/429321968 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9308004300 con 0x7f9318106240 2024-03-04T08:48:14.408 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.620+0000 7f931559a700 1 -- 172.21.15.71:0/429321968 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9308011b20 con 0x7f9318106240 2024-03-04T08:48:14.408 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.620+0000 7f931cf1b700 1 -- 172.21.15.71:0/429321968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9318106240 msgr2=0x7f9318106660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:14.408 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.620+0000 7f931cf1b700 1 --2- 172.21.15.71:0/429321968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9318106240 0x7f9318106660 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f93080098a0 tx=0x7f9308009bb0 comp rx=0 tx=0).stop 2024-03-04T08:48:14.408 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.620+0000 7f931cf1b700 1 -- 172.21.15.71:0/429321968 shutdown_connections 2024-03-04T08:48:14.408 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.620+0000 7f931cf1b700 1 --2- 172.21.15.71:0/429321968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9318106240 0x7f9318106660 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:14.409 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.620+0000 7f931cf1b700 1 -- 172.21.15.71:0/429321968 >> 172.21.15.71:0/429321968 conn(0x7f9318101e80 msgr2=0x7f93181042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:14.409 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.620+0000 7f931cf1b700 1 -- 172.21.15.71:0/429321968 shutdown_connections 2024-03-04T08:48:14.409 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.620+0000 7f931cf1b700 1 -- 172.21.15.71:0/429321968 wait complete. 2024-03-04T08:48:14.409 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.621+0000 7f931cf1b700 1 Processor -- start 2024-03-04T08:48:14.409 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.621+0000 7f931cf1b700 1 -- start start 2024-03-04T08:48:14.409 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.622+0000 7f931cf1b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9318106240 0x7f931819da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:14.409 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.622+0000 7f931cf1b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9318107c70 con 0x7f9318106240 2024-03-04T08:48:14.409 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.622+0000 7f931659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9318106240 0x7f931819da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:14.409 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.622+0000 7f931659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9318106240 0x7f931819da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34458/0 (socket says 172.21.15.71:34458) 2024-03-04T08:48:14.409 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.622+0000 7f931659c700 1 -- 172.21.15.71:0/3054610416 learned_addr learned my addr 172.21.15.71:0/3054610416 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:14.409 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.622+0000 7f931659c700 1 -- 172.21.15.71:0/3054610416 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931819dfd0 con 0x7f9318106240 2024-03-04T08:48:14.409 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.622+0000 7f931659c700 1 --2- 172.21.15.71:0/3054610416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9318106240 0x7f931819da90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f93080116a0 tx=0x7f9308011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:14.409 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.622+0000 7f93077fe700 1 -- 172.21.15.71:0/3054610416 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9308003a40 con 0x7f9318106240 2024-03-04T08:48:14.410 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.622+0000 7f93077fe700 1 -- 172.21.15.71:0/3054610416 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9308003be0 con 0x7f9318106240 2024-03-04T08:48:14.410 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.622+0000 7f93077fe700 1 -- 172.21.15.71:0/3054610416 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f930801a750 con 0x7f9318106240 2024-03-04T08:48:14.410 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.622+0000 7f931cf1b700 1 -- 172.21.15.71:0/3054610416 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931819e260 con 0x7f9318106240 2024-03-04T08:48:14.410 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.623+0000 7f931cf1b700 1 -- 172.21.15.71:0/3054610416 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931819e6c0 con 0x7f9318106240 2024-03-04T08:48:14.410 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.623+0000 7f93077fe700 1 -- 172.21.15.71:0/3054610416 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f930801aa50 con 0x7f9318106240 2024-03-04T08:48:14.410 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.624+0000 7f931cf1b700 1 -- 172.21.15.71:0/3054610416 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93180636f0 con 0x7f9318106240 2024-03-04T08:48:14.410 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.624+0000 7f93077fe700 1 --2- 172.21.15.71:0/3054610416 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f9300038b50 0x7f930003b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:14.410 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.624+0000 7f93077fe700 1 -- 172.21.15.71:0/3054610416 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f930804d210 con 0x7f9318106240 2024-03-04T08:48:14.410 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.624+0000 7f9315d9b700 1 --2- 172.21.15.71:0/3054610416 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f9300038b50 0x7f930003b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:14.410 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.624+0000 7f9315d9b700 1 --2- 172.21.15.71:0/3054610416 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f9300038b50 0x7f930003b010 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f930c006fd0 tx=0x7f930c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:14.410 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.633+0000 7f93077fe700 1 -- 172.21.15.71:0/3054610416 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9308011e20 con 0x7f9318106240 2024-03-04T08:48:14.410 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.842+0000 7f931cf1b700 1 -- 172.21.15.71:0/3054610416 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f9318062cd0 con 0x7f9318106240 2024-03-04T08:48:14.410 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.847+0000 7f93077fe700 1 -- 172.21.15.71:0/3054610416 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f930800bc40 con 0x7f9318106240 2024-03-04T08:48:14.411 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.849+0000 7f931cf1b700 1 -- 172.21.15.71:0/3054610416 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f9300038b50 msgr2=0x7f930003b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:14.411 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.849+0000 7f931cf1b700 1 --2- 172.21.15.71:0/3054610416 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f9300038b50 0x7f930003b010 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f930c006fd0 tx=0x7f930c006e40 comp rx=0 tx=0).stop 2024-03-04T08:48:14.411 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.849+0000 7f931cf1b700 1 -- 172.21.15.71:0/3054610416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9318106240 msgr2=0x7f931819da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:14.411 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.849+0000 7f931cf1b700 1 --2- 172.21.15.71:0/3054610416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9318106240 0x7f931819da90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f93080116a0 tx=0x7f9308011ba0 comp rx=0 tx=0).stop 2024-03-04T08:48:14.411 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.849+0000 7f931cf1b700 1 -- 172.21.15.71:0/3054610416 shutdown_connections 2024-03-04T08:48:14.411 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.849+0000 7f931cf1b700 1 --2- 172.21.15.71:0/3054610416 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f9300038b50 0x7f930003b010 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:14.411 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.849+0000 7f931cf1b700 1 --2- 172.21.15.71:0/3054610416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9318106240 0x7f931819da90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:14.411 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.849+0000 7f931cf1b700 1 -- 172.21.15.71:0/3054610416 >> 172.21.15.71:0/3054610416 conn(0x7f9318101e80 msgr2=0x7f9318194700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:14.411 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.849+0000 7f931cf1b700 1 -- 172.21.15.71:0/3054610416 shutdown_connections 2024-03-04T08:48:14.411 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:13.850+0000 7f931cf1b700 1 -- 172.21.15.71:0/3054610416 wait complete. 2024-03-04T08:48:14.411 INFO:teuthology.orchestra.run.smithi071.stdout:Enabling the dashboard module... 2024-03-04T08:48:15.034 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:14 smithi071 ceph-mon[32789]: from='mgr.14120 172.21.15.71:0/2961810335' entity='mgr.x' 2024-03-04T08:48:15.034 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:14 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3054610416' entity='client.admin' 2024-03-04T08:48:16.008 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:15 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/58925781' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-04T08:48:16.008 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:15 smithi071 ceph-mon[32789]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:48:16.008 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:15 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: ignoring --setuser ceph since I am not root 2024-03-04T08:48:16.008 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:15 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: ignoring --setgroup ceph since I am not root 2024-03-04T08:48:16.602 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:16 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:16.342+0000 7f05433e1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T08:48:16.602 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:16 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:16.437+0000 7f05433e1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T08:48:16.609 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.821+0000 7f47c8a4c700 1 Processor -- start 2024-03-04T08:48:16.610 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.821+0000 7f47c8a4c700 1 -- start start 2024-03-04T08:48:16.610 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.822+0000 7f47c8a4c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47c4106900 0x7f47c4106d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:16.610 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.822+0000 7f47c8a4c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c41072f0 con 0x7f47c4106900 2024-03-04T08:48:16.610 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.822+0000 7f47c259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47c4106900 0x7f47c4106d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:16.610 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.822+0000 7f47c259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47c4106900 0x7f47c4106d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34462/0 (socket says 172.21.15.71:34462) 2024-03-04T08:48:16.610 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.822+0000 7f47c259c700 1 -- 172.21.15.71:0/3961282928 learned_addr learned my addr 172.21.15.71:0/3961282928 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:16.610 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.822+0000 7f47c259c700 1 -- 172.21.15.71:0/3961282928 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c4107b00 con 0x7f47c4106900 2024-03-04T08:48:16.610 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.822+0000 7f47c259c700 1 --2- 172.21.15.71:0/3961282928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47c4106900 0x7f47c4106d20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f47b40098a0 tx=0x7f47b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578c0f677a5fcad1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:16.610 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.823+0000 7f47c159a700 1 -- 172.21.15.71:0/3961282928 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47b4004160 con 0x7f47c4106900 2024-03-04T08:48:16.610 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.823+0000 7f47c159a700 1 -- 172.21.15.71:0/3961282928 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47b4004300 con 0x7f47c4106900 2024-03-04T08:48:16.610 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.823+0000 7f47c159a700 1 -- 172.21.15.71:0/3961282928 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47b4011b20 con 0x7f47c4106900 2024-03-04T08:48:16.611 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.823+0000 7f47c8a4c700 1 -- 172.21.15.71:0/3961282928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47c4106900 msgr2=0x7f47c4106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:16.611 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.823+0000 7f47c8a4c700 1 --2- 172.21.15.71:0/3961282928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47c4106900 0x7f47c4106d20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f47b40098a0 tx=0x7f47b4009bb0 comp rx=0 tx=0).stop 2024-03-04T08:48:16.611 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.823+0000 7f47c8a4c700 1 -- 172.21.15.71:0/3961282928 shutdown_connections 2024-03-04T08:48:16.611 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.823+0000 7f47c8a4c700 1 --2- 172.21.15.71:0/3961282928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47c4106900 0x7f47c4106d20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:16.611 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.823+0000 7f47c8a4c700 1 -- 172.21.15.71:0/3961282928 >> 172.21.15.71:0/3961282928 conn(0x7f47c4101f60 msgr2=0x7f47c4104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:16.611 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.823+0000 7f47c8a4c700 1 -- 172.21.15.71:0/3961282928 shutdown_connections 2024-03-04T08:48:16.611 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.823+0000 7f47c8a4c700 1 -- 172.21.15.71:0/3961282928 wait complete. 2024-03-04T08:48:16.611 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.824+0000 7f47c8a4c700 1 Processor -- start 2024-03-04T08:48:16.611 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.824+0000 7f47c8a4c700 1 -- start start 2024-03-04T08:48:16.611 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.824+0000 7f47c8a4c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47c4106900 0x7f47c4191a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:16.611 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.824+0000 7f47c8a4c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c410aeb0 con 0x7f47c4106900 2024-03-04T08:48:16.611 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.824+0000 7f47c259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47c4106900 0x7f47c4191a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:16.611 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.824+0000 7f47c259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47c4106900 0x7f47c4191a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34472/0 (socket says 172.21.15.71:34472) 2024-03-04T08:48:16.611 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.824+0000 7f47c259c700 1 -- 172.21.15.71:0/58925781 learned_addr learned my addr 172.21.15.71:0/58925781 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:16.612 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.825+0000 7f47c259c700 1 -- 172.21.15.71:0/58925781 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c4191fb0 con 0x7f47c4106900 2024-03-04T08:48:16.612 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.825+0000 7f47c259c700 1 --2- 172.21.15.71:0/58925781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47c4106900 0x7f47c4191a70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f47b4000c00 tx=0x7f47b4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:16.612 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.825+0000 7f47b37fe700 1 -- 172.21.15.71:0/58925781 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47b4003b60 con 0x7f47c4106900 2024-03-04T08:48:16.612 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.825+0000 7f47b37fe700 1 -- 172.21.15.71:0/58925781 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47b401a910 con 0x7f47c4106900 2024-03-04T08:48:16.612 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.825+0000 7f47c8a4c700 1 -- 172.21.15.71:0/58925781 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c41901a0 con 0x7f47c4106900 2024-03-04T08:48:16.612 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.825+0000 7f47b37fe700 1 -- 172.21.15.71:0/58925781 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47b4024930 con 0x7f47c4106900 2024-03-04T08:48:16.612 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.825+0000 7f47c8a4c700 1 -- 172.21.15.71:0/58925781 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c4190660 con 0x7f47c4106900 2024-03-04T08:48:16.612 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.826+0000 7f47b37fe700 1 -- 172.21.15.71:0/58925781 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f47b4022020 con 0x7f47c4106900 2024-03-04T08:48:16.612 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.826+0000 7f47b37fe700 1 --2- 172.21.15.71:0/58925781 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f47ac038ba0 0x7f47ac03b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:16.612 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.826+0000 7f47b37fe700 1 -- 172.21.15.71:0/58925781 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f47b404cdb0 con 0x7f47c4106900 2024-03-04T08:48:16.612 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.826+0000 7f47c1d9b700 1 --2- 172.21.15.71:0/58925781 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f47ac038ba0 0x7f47ac03b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:16.612 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.826+0000 7f47c8a4c700 1 -- 172.21.15.71:0/58925781 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c40636f0 con 0x7f47c4106900 2024-03-04T08:48:16.612 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.826+0000 7f47c1d9b700 1 --2- 172.21.15.71:0/58925781 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f47ac038ba0 0x7f47ac03b060 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f47b8006fd0 tx=0x7f47b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:16.613 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:14.831+0000 7f47b37fe700 1 -- 172.21.15.71:0/58925781 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f47b400b3e0 con 0x7f47c4106900 2024-03-04T08:48:16.613 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:15.090+0000 7f47c8a4c700 1 -- 172.21.15.71:0/58925781 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f47c4195610 con 0x7f47c4106900 2024-03-04T08:48:16.613 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:15.774+0000 7f47b37fe700 1 -- 172.21.15.71:0/58925781 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f47b40172b0 con 0x7f47c4106900 2024-03-04T08:48:16.613 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:15.776+0000 7f47b37fe700 1 -- 172.21.15.71:0/58925781 <== mon.0 v2:172.21.15.71:3300/0 8 ==== mgrmap(e 9) v1 ==== 45941+0+0 (secure 0 0 0) 0x7f47b402dc40 con 0x7f47c4106900 2024-03-04T08:48:16.613 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:15.777+0000 7f47c8a4c700 1 -- 172.21.15.71:0/58925781 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f47ac038ba0 msgr2=0x7f47ac03b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:16.613 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:15.777+0000 7f47c8a4c700 1 --2- 172.21.15.71:0/58925781 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f47ac038ba0 0x7f47ac03b060 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f47b8006fd0 tx=0x7f47b8006e40 comp rx=0 tx=0).stop 2024-03-04T08:48:16.613 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:15.777+0000 7f47c8a4c700 1 -- 172.21.15.71:0/58925781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47c4106900 msgr2=0x7f47c4191a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:16.613 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:15.778+0000 7f47c8a4c700 1 --2- 172.21.15.71:0/58925781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47c4106900 0x7f47c4191a70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f47b4000c00 tx=0x7f47b4011c10 comp rx=0 tx=0).stop 2024-03-04T08:48:16.613 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:15.778+0000 7f47c8a4c700 1 -- 172.21.15.71:0/58925781 shutdown_connections 2024-03-04T08:48:16.613 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:15.778+0000 7f47c8a4c700 1 --2- 172.21.15.71:0/58925781 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f47ac038ba0 0x7f47ac03b060 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:16.613 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:15.778+0000 7f47c8a4c700 1 --2- 172.21.15.71:0/58925781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47c4106900 0x7f47c4191a70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:16.613 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:15.778+0000 7f47c8a4c700 1 -- 172.21.15.71:0/58925781 >> 172.21.15.71:0/58925781 conn(0x7f47c4101f60 msgr2=0x7f47c4102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:16.613 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:15.778+0000 7f47c8a4c700 1 -- 172.21.15.71:0/58925781 shutdown_connections 2024-03-04T08:48:16.613 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:15.778+0000 7f47c8a4c700 1 -- 172.21.15.71:0/58925781 wait complete. 2024-03-04T08:48:16.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:16 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/58925781' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-04T08:48:16.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:16 smithi071 ceph-mon[32789]: mgrmap e9: x(active, since 20s) 2024-03-04T08:48:17.259 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:17 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:17.094+0000 7f05433e1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T08:48:17.259 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:17 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:17.189+0000 7f05433e1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T08:48:17.510 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:17 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:17.332+0000 7f05433e1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T08:48:17.510 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:17 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:17.431+0000 7f05433e1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T08:48:17.510 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:17 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:17.514+0000 7f05433e1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T08:48:17.883 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-03-04T08:48:17.883 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-04T08:48:17.884 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-04T08:48:17.884 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-04T08:48:17.884 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-04T08:48:17.884 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-03-04T08:48:17.884 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.058+0000 7fadc2a14700 1 Processor -- start 2024-03-04T08:48:17.884 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.058+0000 7fadc2a14700 1 -- start start 2024-03-04T08:48:17.884 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.058+0000 7fadc2a14700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc108440 0x7fadbc10a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:17.884 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.058+0000 7fadc2a14700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc07aea0 con 0x7fadbc108440 2024-03-04T08:48:17.884 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.058+0000 7fadbbfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc108440 0x7fadbc10a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:17.884 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.058+0000 7fadbbfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc108440 0x7fadbc10a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33386/0 (socket says 172.21.15.71:33386) 2024-03-04T08:48:17.884 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.058+0000 7fadbbfff700 1 -- 172.21.15.71:0/1512893383 learned_addr learned my addr 172.21.15.71:0/1512893383 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:17.884 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.058+0000 7fadbbfff700 1 -- 172.21.15.71:0/1512893383 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc10ad70 con 0x7fadbc108440 2024-03-04T08:48:17.885 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.059+0000 7fadbbfff700 1 --2- 172.21.15.71:0/1512893383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc108440 0x7fadbc10a830 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fadac0098a0 tx=0x7fadac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d53a191a5db7e79 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:17.885 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.059+0000 7fadbaffd700 1 -- 172.21.15.71:0/1512893383 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fadac004160 con 0x7fadbc108440 2024-03-04T08:48:17.885 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.059+0000 7fadbaffd700 1 -- 172.21.15.71:0/1512893383 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadac004300 con 0x7fadbc108440 2024-03-04T08:48:17.885 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.059+0000 7fadbaffd700 1 -- 172.21.15.71:0/1512893383 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fadac011b20 con 0x7fadbc108440 2024-03-04T08:48:17.885 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.059+0000 7fadc2a14700 1 -- 172.21.15.71:0/1512893383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc108440 msgr2=0x7fadbc10a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:17.885 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.059+0000 7fadc2a14700 1 --2- 172.21.15.71:0/1512893383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc108440 0x7fadbc10a830 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fadac0098a0 tx=0x7fadac009bb0 comp rx=0 tx=0).stop 2024-03-04T08:48:17.885 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.059+0000 7fadc2a14700 1 -- 172.21.15.71:0/1512893383 shutdown_connections 2024-03-04T08:48:17.885 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.059+0000 7fadc2a14700 1 --2- 172.21.15.71:0/1512893383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc108440 0x7fadbc10a830 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:17.885 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.059+0000 7fadc2a14700 1 -- 172.21.15.71:0/1512893383 >> 172.21.15.71:0/1512893383 conn(0x7fadbc101f50 msgr2=0x7fadbc104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:17.885 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.060+0000 7fadc2a14700 1 -- 172.21.15.71:0/1512893383 shutdown_connections 2024-03-04T08:48:17.885 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.060+0000 7fadc2a14700 1 -- 172.21.15.71:0/1512893383 wait complete. 2024-03-04T08:48:17.885 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.060+0000 7fadc2a14700 1 Processor -- start 2024-03-04T08:48:17.886 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.060+0000 7fadc2a14700 1 -- start start 2024-03-04T08:48:17.886 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.060+0000 7fadc2a14700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc108440 0x7fadbc19b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:17.886 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.060+0000 7fadc2a14700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc10bde0 con 0x7fadbc108440 2024-03-04T08:48:17.886 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.061+0000 7fadbbfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc108440 0x7fadbc19b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:17.886 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.061+0000 7fadbbfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc108440 0x7fadbc19b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33402/0 (socket says 172.21.15.71:33402) 2024-03-04T08:48:17.886 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.061+0000 7fadbbfff700 1 -- 172.21.15.71:0/3381893482 learned_addr learned my addr 172.21.15.71:0/3381893482 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:17.886 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.061+0000 7fadbbfff700 1 -- 172.21.15.71:0/3381893482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc19be30 con 0x7fadbc108440 2024-03-04T08:48:17.886 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.061+0000 7fadbbfff700 1 --2- 172.21.15.71:0/3381893482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc108440 0x7fadbc19b8f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fadac011b60 tx=0x7fadac011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:17.886 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.061+0000 7fadb97fa700 1 -- 172.21.15.71:0/3381893482 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fadac003b60 con 0x7fadbc108440 2024-03-04T08:48:17.886 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.061+0000 7fadb97fa700 1 -- 172.21.15.71:0/3381893482 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadac01a910 con 0x7fadbc108440 2024-03-04T08:48:17.886 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.061+0000 7fadc2a14700 1 -- 172.21.15.71:0/3381893482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadbc19c0c0 con 0x7fadbc108440 2024-03-04T08:48:17.886 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.061+0000 7fadb97fa700 1 -- 172.21.15.71:0/3381893482 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fadac024930 con 0x7fadbc108440 2024-03-04T08:48:17.886 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.061+0000 7fadc2a14700 1 -- 172.21.15.71:0/3381893482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadbc19c520 con 0x7fadbc108440 2024-03-04T08:48:17.886 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.062+0000 7fadb97fa700 1 -- 172.21.15.71:0/3381893482 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 9) v1 ==== 45941+0+0 (secure 0 0 0) 0x7fadac022020 con 0x7fadbc108440 2024-03-04T08:48:17.887 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.062+0000 7fadb97fa700 1 --2- 172.21.15.71:0/3381893482 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fada4038bf0 0x7fada403b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:17.887 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.062+0000 7fadc2a14700 1 -- 172.21.15.71:0/3381893482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadbc069150 con 0x7fadbc108440 2024-03-04T08:48:17.887 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.062+0000 7fadb97fa700 1 -- 172.21.15.71:0/3381893482 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fadac04ccc0 con 0x7fadbc108440 2024-03-04T08:48:17.887 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.062+0000 7fadbb7fe700 1 -- 172.21.15.71:0/3381893482 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fada4038bf0 msgr2=0x7fada403b0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/2808156336 2024-03-04T08:48:17.887 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.062+0000 7fadbb7fe700 1 --2- 172.21.15.71:0/3381893482 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fada4038bf0 0x7fada403b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:17.887 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.067+0000 7fadb97fa700 1 -- 172.21.15.71:0/3381893482 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fadac02d910 con 0x7fadbc108440 2024-03-04T08:48:17.887 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.263+0000 7fadbb7fe700 1 -- 172.21.15.71:0/3381893482 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fada4038bf0 msgr2=0x7fada403b0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/2808156336 2024-03-04T08:48:17.887 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.263+0000 7fadbb7fe700 1 --2- 172.21.15.71:0/3381893482 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fada4038bf0 0x7fada403b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:17.887 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.319+0000 7fadc2a14700 1 -- 172.21.15.71:0/3381893482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fadbc1068f0 con 0x7fadbc108440 2024-03-04T08:48:17.887 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.320+0000 7fadb97fa700 1 -- 172.21.15.71:0/3381893482 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fadac032070 con 0x7fadbc108440 2024-03-04T08:48:17.887 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.322+0000 7fadc2a14700 1 -- 172.21.15.71:0/3381893482 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fada4038bf0 msgr2=0x7fada403b0b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T08:48:17.887 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.322+0000 7fadc2a14700 1 --2- 172.21.15.71:0/3381893482 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fada4038bf0 0x7fada403b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:17.888 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.322+0000 7fadc2a14700 1 -- 172.21.15.71:0/3381893482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc108440 msgr2=0x7fadbc19b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:17.888 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.322+0000 7fadc2a14700 1 --2- 172.21.15.71:0/3381893482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc108440 0x7fadbc19b8f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fadac011b60 tx=0x7fadac011c40 comp rx=0 tx=0).stop 2024-03-04T08:48:17.888 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.322+0000 7fadc2a14700 1 -- 172.21.15.71:0/3381893482 shutdown_connections 2024-03-04T08:48:17.888 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.322+0000 7fadc2a14700 1 --2- 172.21.15.71:0/3381893482 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7fada4038bf0 0x7fada403b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:17.888 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.322+0000 7fadc2a14700 1 --2- 172.21.15.71:0/3381893482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc108440 0x7fadbc19b8f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:17.888 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.322+0000 7fadc2a14700 1 -- 172.21.15.71:0/3381893482 >> 172.21.15.71:0/3381893482 conn(0x7fadbc101f50 msgr2=0x7fadbc102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:17.888 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.322+0000 7fadc2a14700 1 -- 172.21.15.71:0/3381893482 shutdown_connections 2024-03-04T08:48:17.888 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:17.322+0000 7fadc2a14700 1 -- 172.21.15.71:0/3381893482 wait complete. 2024-03-04T08:48:17.888 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for the mgr to restart... 2024-03-04T08:48:17.888 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mgr epoch 9... 2024-03-04T08:48:18.025 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:17 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3381893482' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-04T08:48:18.025 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:17 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:17.782+0000 7f05433e1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-04T08:48:18.025 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:17 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:17.895+0000 7f05433e1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T08:48:18.025 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:17 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:17.989+0000 7f05433e1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T08:48:18.286 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:18 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:18.084+0000 7f05433e1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T08:48:18.286 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:18 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:18.265+0000 7f05433e1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T08:48:19.667 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:19 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:19.217+0000 7f05433e1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-04T08:48:19.667 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:19 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:19.345+0000 7f05433e1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T08:48:19.667 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:19 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:19.670+0000 7f05433e1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T08:48:19.922 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:19 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:19.759+0000 7f05433e1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T08:48:20.417 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:19 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:19.953+0000 7f05433e1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T08:48:20.417 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:20 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:20.140+0000 7f05433e1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T08:48:20.671 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:20 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:20.420+0000 7f05433e1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T08:48:21.171 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:20 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:20.681+0000 7f05433e1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T08:48:21.171 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:20 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:20.916+0000 7f05433e1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T08:48:22.172 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:21 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:21.708+0000 7f05433e1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T08:48:22.576 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:22 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:22.268+0000 7f05433e1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T08:48:22.577 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:22 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:22.365+0000 7f05433e1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T08:48:22.577 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:48:22 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:48:22.580+0000 7f05433e1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T08:48:22.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:22 smithi071 ceph-mon[32789]: Active manager daemon x restarted 2024-03-04T08:48:22.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:22 smithi071 ceph-mon[32789]: Activating manager daemon x 2024-03-04T08:48:22.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:22 smithi071 ceph-mon[32789]: osdmap e3: 0 total, 0 up, 0 in 2024-03-04T08:48:22.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:22 smithi071 ceph-mon[32789]: mgrmap e10: x(active, starting, since 0.00312056s) 2024-03-04T08:48:22.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:22 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T08:48:22.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:22 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-04T08:48:22.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:22 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T08:48:22.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:22 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T08:48:22.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:22 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T08:48:22.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:22 smithi071 ceph-mon[32789]: Manager daemon x is now available 2024-03-04T08:48:22.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:22 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:48:22.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:22 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:48:23.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:23 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-04T08:48:23.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:23 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-04T08:48:23.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:23 smithi071 ceph-mon[32789]: mgrmap e11: x(active, since 1.00547s) 2024-03-04T08:48:24.535 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-03-04T08:48:24.535 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-04T08:48:24.535 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-04T08:48:24.535 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-03-04T08:48:24.535 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.307+0000 7f423dcc4700 1 Processor -- start 2024-03-04T08:48:24.535 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.307+0000 7f423dcc4700 1 -- start start 2024-03-04T08:48:24.535 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.308+0000 7f423dcc4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4238106250 0x7f4238106670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:24.535 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.308+0000 7f423dcc4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423807ae80 con 0x7f4238106250 2024-03-04T08:48:24.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.308+0000 7f42377fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4238106250 0x7f4238106670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:24.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.308+0000 7f42377fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4238106250 0x7f4238106670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33410/0 (socket says 172.21.15.71:33410) 2024-03-04T08:48:24.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.308+0000 7f42377fe700 1 -- 172.21.15.71:0/1196418337 learned_addr learned my addr 172.21.15.71:0/1196418337 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:24.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.308+0000 7f42377fe700 1 -- 172.21.15.71:0/1196418337 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4238106bb0 con 0x7f4238106250 2024-03-04T08:48:24.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.308+0000 7f42377fe700 1 --2- 172.21.15.71:0/1196418337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4238106250 0x7f4238106670 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f42280098a0 tx=0x7f4228009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4f31ef4b751d1e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:24.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.308+0000 7f42367fc700 1 -- 172.21.15.71:0/1196418337 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4228004160 con 0x7f4238106250 2024-03-04T08:48:24.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.308+0000 7f42367fc700 1 -- 172.21.15.71:0/1196418337 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4228004300 con 0x7f4238106250 2024-03-04T08:48:24.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.308+0000 7f42367fc700 1 -- 172.21.15.71:0/1196418337 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4228011b20 con 0x7f4238106250 2024-03-04T08:48:24.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.309+0000 7f423dcc4700 1 -- 172.21.15.71:0/1196418337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4238106250 msgr2=0x7f4238106670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:24.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.309+0000 7f423dcc4700 1 --2- 172.21.15.71:0/1196418337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4238106250 0x7f4238106670 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f42280098a0 tx=0x7f4228009bb0 comp rx=0 tx=0).stop 2024-03-04T08:48:24.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.309+0000 7f423dcc4700 1 -- 172.21.15.71:0/1196418337 shutdown_connections 2024-03-04T08:48:24.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.309+0000 7f423dcc4700 1 --2- 172.21.15.71:0/1196418337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4238106250 0x7f4238106670 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:24.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.309+0000 7f423dcc4700 1 -- 172.21.15.71:0/1196418337 >> 172.21.15.71:0/1196418337 conn(0x7f4238101f50 msgr2=0x7f4238104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:24.537 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.309+0000 7f423dcc4700 1 -- 172.21.15.71:0/1196418337 shutdown_connections 2024-03-04T08:48:24.537 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.309+0000 7f423dcc4700 1 -- 172.21.15.71:0/1196418337 wait complete. 2024-03-04T08:48:24.537 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.310+0000 7f423dcc4700 1 Processor -- start 2024-03-04T08:48:24.537 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.310+0000 7f423dcc4700 1 -- start start 2024-03-04T08:48:24.537 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.310+0000 7f423dcc4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4238106250 0x7f423819daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:24.537 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.310+0000 7f423dcc4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4238107c80 con 0x7f4238106250 2024-03-04T08:48:24.537 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.310+0000 7f42377fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4238106250 0x7f423819daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:24.537 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.310+0000 7f42377fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4238106250 0x7f423819daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33426/0 (socket says 172.21.15.71:33426) 2024-03-04T08:48:24.537 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.310+0000 7f42377fe700 1 -- 172.21.15.71:0/3071881596 learned_addr learned my addr 172.21.15.71:0/3071881596 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:24.537 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.310+0000 7f42377fe700 1 -- 172.21.15.71:0/3071881596 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423819e030 con 0x7f4238106250 2024-03-04T08:48:24.537 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.311+0000 7f42377fe700 1 --2- 172.21.15.71:0/3071881596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4238106250 0x7f423819daf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4228011b60 tx=0x7f4228011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:24.537 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.311+0000 7f4234ff9700 1 -- 172.21.15.71:0/3071881596 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4228003b60 con 0x7f4238106250 2024-03-04T08:48:24.537 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.311+0000 7f423dcc4700 1 -- 172.21.15.71:0/3071881596 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f423819e2c0 con 0x7f4238106250 2024-03-04T08:48:24.537 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.311+0000 7f423dcc4700 1 -- 172.21.15.71:0/3071881596 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423819e720 con 0x7f4238106250 2024-03-04T08:48:24.538 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.311+0000 7f4234ff9700 1 -- 172.21.15.71:0/3071881596 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f422801a910 con 0x7f4238106250 2024-03-04T08:48:24.538 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.311+0000 7f4234ff9700 1 -- 172.21.15.71:0/3071881596 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4228024930 con 0x7f4238106250 2024-03-04T08:48:24.538 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.312+0000 7f4234ff9700 1 -- 172.21.15.71:0/3071881596 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 9) v1 ==== 45941+0+0 (secure 0 0 0) 0x7f4228022020 con 0x7f4238106250 2024-03-04T08:48:24.538 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.312+0000 7f4234ff9700 1 --2- 172.21.15.71:0/3071881596 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f4220038bf0 0x7f422003b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:24.538 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.312+0000 7f4234ff9700 1 -- 172.21.15.71:0/3071881596 --> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f422003b7c0 con 0x7f4220038bf0 2024-03-04T08:48:24.538 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.312+0000 7f4234ff9700 1 -- 172.21.15.71:0/3071881596 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f422804ccd0 con 0x7f4238106250 2024-03-04T08:48:24.538 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.312+0000 7f4236ffd700 1 -- 172.21.15.71:0/3071881596 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f4220038bf0 msgr2=0x7f422003b0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/2808156336 2024-03-04T08:48:24.538 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.312+0000 7f4236ffd700 1 --2- 172.21.15.71:0/3071881596 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f4220038bf0 0x7f422003b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:24.538 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.513+0000 7f4236ffd700 1 -- 172.21.15.71:0/3071881596 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f4220038bf0 msgr2=0x7f422003b0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/2808156336 2024-03-04T08:48:24.538 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.513+0000 7f4236ffd700 1 --2- 172.21.15.71:0/3071881596 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f4220038bf0 0x7f422003b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:24.538 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.914+0000 7f4236ffd700 1 -- 172.21.15.71:0/3071881596 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f4220038bf0 msgr2=0x7f422003b0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/2808156336 2024-03-04T08:48:24.538 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:18.914+0000 7f4236ffd700 1 --2- 172.21.15.71:0/3071881596 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f4220038bf0 0x7f422003b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:24.538 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:19.715+0000 7f4236ffd700 1 -- 172.21.15.71:0/3071881596 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f4220038bf0 msgr2=0x7f422003b0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/2808156336 2024-03-04T08:48:24.538 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:19.715+0000 7f4236ffd700 1 --2- 172.21.15.71:0/3071881596 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f4220038bf0 0x7f422003b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:24.539 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:21.316+0000 7f4236ffd700 1 -- 172.21.15.71:0/3071881596 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f4220038bf0 msgr2=0x7f422003b0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/2808156336 2024-03-04T08:48:24.539 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:21.316+0000 7f4236ffd700 1 --2- 172.21.15.71:0/3071881596 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f4220038bf0 0x7f422003b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-04T08:48:24.539 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:22.595+0000 7f4234ff9700 1 -- 172.21.15.71:0/3071881596 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mgrmap(e 10) v1 ==== 45708+0+0 (secure 0 0 0) 0x7f422802daa0 con 0x7f4238106250 2024-03-04T08:48:24.539 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:22.595+0000 7f4234ff9700 1 -- 172.21.15.71:0/3071881596 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f4220038bf0 msgr2=0x7f422003b0b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T08:48:24.539 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:22.595+0000 7f4234ff9700 1 --2- 172.21.15.71:0/3071881596 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f4220038bf0 0x7f422003b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:24.539 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:23.596+0000 7f4234ff9700 1 -- 172.21.15.71:0/3071881596 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 11) v1 ==== 45835+0+0 (secure 0 0 0) 0x7f422804e590 con 0x7f4238106250 2024-03-04T08:48:24.539 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:23.597+0000 7f4234ff9700 1 --2- 172.21.15.71:0/3071881596 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f422003c6b0 0x7f422003eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:24.539 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:23.597+0000 7f4234ff9700 1 -- 172.21.15.71:0/3071881596 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f422003b7c0 con 0x7f422003c6b0 2024-03-04T08:48:24.539 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:23.597+0000 7f4236ffd700 1 --2- 172.21.15.71:0/3071881596 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f422003c6b0 0x7f422003eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:24.539 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:23.597+0000 7f4236ffd700 1 --2- 172.21.15.71:0/3071881596 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f422003c6b0 0x7f422003eaa0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f422c003a10 tx=0x7f422c0092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:24.539 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:23.598+0000 7f4234ff9700 1 -- 172.21.15.71:0/3071881596 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f422003b7c0 con 0x7f422003c6b0 2024-03-04T08:48:24.539 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:23.604+0000 7f421e7fc700 1 -- 172.21.15.71:0/3071881596 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f4238069150 con 0x7f422003c6b0 2024-03-04T08:48:24.539 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:23.604+0000 7f4234ff9700 1 -- 172.21.15.71:0/3071881596 <== mgr.14152 v2:172.21.15.71:6800/2901868399 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f4238069150 con 0x7f422003c6b0 2024-03-04T08:48:24.539 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:23.605+0000 7f421e7fc700 1 -- 172.21.15.71:0/3071881596 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f422003c6b0 msgr2=0x7f422003eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:24.540 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:23.605+0000 7f421e7fc700 1 --2- 172.21.15.71:0/3071881596 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f422003c6b0 0x7f422003eaa0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f422c003a10 tx=0x7f422c0092b0 comp rx=0 tx=0).stop 2024-03-04T08:48:24.540 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:23.605+0000 7f421e7fc700 1 -- 172.21.15.71:0/3071881596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4238106250 msgr2=0x7f423819daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:24.540 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:23.605+0000 7f421e7fc700 1 --2- 172.21.15.71:0/3071881596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4238106250 0x7f423819daf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4228011b60 tx=0x7f4228011c40 comp rx=0 tx=0).stop 2024-03-04T08:48:24.540 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:23.605+0000 7f421e7fc700 1 -- 172.21.15.71:0/3071881596 shutdown_connections 2024-03-04T08:48:24.540 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:23.605+0000 7f421e7fc700 1 --2- 172.21.15.71:0/3071881596 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f422003c6b0 0x7f422003eaa0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:24.540 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:23.605+0000 7f421e7fc700 1 --2- 172.21.15.71:0/3071881596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4238106250 0x7f423819daf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:24.540 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:23.605+0000 7f421e7fc700 1 --2- 172.21.15.71:0/3071881596 >> [v2:172.21.15.71:6800/2808156336,v1:172.21.15.71:6801/2808156336] conn(0x7f4220038bf0 0x7f422003b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:24.540 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:23.605+0000 7f421e7fc700 1 -- 172.21.15.71:0/3071881596 >> 172.21.15.71:0/3071881596 conn(0x7f4238101f50 msgr2=0x7f42381947b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:24.540 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:23.605+0000 7f421e7fc700 1 -- 172.21.15.71:0/3071881596 shutdown_connections 2024-03-04T08:48:24.540 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:23.605+0000 7f421e7fc700 1 -- 172.21.15.71:0/3071881596 wait complete. 2024-03-04T08:48:24.540 INFO:teuthology.orchestra.run.smithi071.stdout:mgr epoch 9 is available 2024-03-04T08:48:24.540 INFO:teuthology.orchestra.run.smithi071.stdout:Generating a dashboard self-signed certificate... 2024-03-04T08:48:24.804 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:24 smithi071 ceph-mon[32789]: [04/Mar/2024:08:48:23] ENGINE Bus STARTING 2024-03-04T08:48:24.804 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:24 smithi071 ceph-mon[32789]: [04/Mar/2024:08:48:23] ENGINE Serving on https://172.21.15.71:7150 2024-03-04T08:48:24.804 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:24 smithi071 ceph-mon[32789]: [04/Mar/2024:08:48:23] ENGINE Serving on http://172.21.15.71:8765 2024-03-04T08:48:24.804 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:24 smithi071 ceph-mon[32789]: [04/Mar/2024:08:48:23] ENGINE Bus STARTED 2024-03-04T08:48:24.805 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:24 smithi071 ceph-mon[32789]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-04T08:48:24.805 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:24 smithi071 ceph-mon[32789]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-04T08:48:24.805 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:24 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:25.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-04T08:48:25.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.008+0000 7f3fee44b700 1 Processor -- start 2024-03-04T08:48:25.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.008+0000 7f3fee44b700 1 -- start start 2024-03-04T08:48:25.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.009+0000 7f3fee44b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fe8106850 0x7f3fe8106c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:25.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.009+0000 7f3fee44b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe8107240 con 0x7f3fe8106850 2024-03-04T08:48:25.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.009+0000 7f3fe7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fe8106850 0x7f3fe8106c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:25.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.009+0000 7f3fe7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fe8106850 0x7f3fe8106c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33484/0 (socket says 172.21.15.71:33484) 2024-03-04T08:48:25.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.009+0000 7f3fe7fff700 1 -- 172.21.15.71:0/3258171011 learned_addr learned my addr 172.21.15.71:0/3258171011 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:25.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.009+0000 7f3fe7fff700 1 -- 172.21.15.71:0/3258171011 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe8107a50 con 0x7f3fe8106850 2024-03-04T08:48:25.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.009+0000 7f3fe7fff700 1 --2- 172.21.15.71:0/3258171011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fe8106850 0x7f3fe8106c70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80098a0 tx=0x7f3fd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4428ee93a5ebbda6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:25.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.009+0000 7f3fe6ffd700 1 -- 172.21.15.71:0/3258171011 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fd8004160 con 0x7f3fe8106850 2024-03-04T08:48:25.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.009+0000 7f3fe6ffd700 1 -- 172.21.15.71:0/3258171011 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fd8004300 con 0x7f3fe8106850 2024-03-04T08:48:25.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.009+0000 7f3fe6ffd700 1 -- 172.21.15.71:0/3258171011 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fd800fb90 con 0x7f3fe8106850 2024-03-04T08:48:25.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.010+0000 7f3fee44b700 1 -- 172.21.15.71:0/3258171011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fe8106850 msgr2=0x7f3fe8106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:25.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.010+0000 7f3fee44b700 1 --2- 172.21.15.71:0/3258171011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fe8106850 0x7f3fe8106c70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80098a0 tx=0x7f3fd8009bb0 comp rx=0 tx=0).stop 2024-03-04T08:48:25.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.010+0000 7f3fee44b700 1 -- 172.21.15.71:0/3258171011 shutdown_connections 2024-03-04T08:48:25.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.010+0000 7f3fee44b700 1 --2- 172.21.15.71:0/3258171011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fe8106850 0x7f3fe8106c70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:25.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.010+0000 7f3fee44b700 1 -- 172.21.15.71:0/3258171011 >> 172.21.15.71:0/3258171011 conn(0x7f3fe8101ed0 msgr2=0x7f3fe81042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:25.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.010+0000 7f3fee44b700 1 -- 172.21.15.71:0/3258171011 shutdown_connections 2024-03-04T08:48:25.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.010+0000 7f3fee44b700 1 -- 172.21.15.71:0/3258171011 wait complete. 2024-03-04T08:48:25.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.011+0000 7f3fee44b700 1 Processor -- start 2024-03-04T08:48:25.866 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.011+0000 7f3fee44b700 1 -- start start 2024-03-04T08:48:25.866 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.011+0000 7f3fee44b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fe8106850 0x7f3fe8199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:25.866 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.011+0000 7f3fee44b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe8079fc0 con 0x7f3fe8106850 2024-03-04T08:48:25.866 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.011+0000 7f3fe7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fe8106850 0x7f3fe8199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:25.866 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.011+0000 7f3fe7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fe8106850 0x7f3fe8199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33488/0 (socket says 172.21.15.71:33488) 2024-03-04T08:48:25.866 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.011+0000 7f3fe7fff700 1 -- 172.21.15.71:0/571995556 learned_addr learned my addr 172.21.15.71:0/571995556 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:25.866 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.011+0000 7f3fe7fff700 1 -- 172.21.15.71:0/571995556 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe8199c50 con 0x7f3fe8106850 2024-03-04T08:48:25.866 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.011+0000 7f3fe7fff700 1 --2- 172.21.15.71:0/571995556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fe8106850 0x7f3fe8199710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009fb0 tx=0x7f3fd8003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:25.866 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.012+0000 7f3fe57fa700 1 -- 172.21.15.71:0/571995556 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fd8010470 con 0x7f3fe8106850 2024-03-04T08:48:25.866 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.012+0000 7f3fe57fa700 1 -- 172.21.15.71:0/571995556 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fd8010af0 con 0x7f3fe8106850 2024-03-04T08:48:25.866 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.012+0000 7f3fe57fa700 1 -- 172.21.15.71:0/571995556 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fd80228e0 con 0x7f3fe8106850 2024-03-04T08:48:25.866 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.012+0000 7f3fee44b700 1 -- 172.21.15.71:0/571995556 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe8199ee0 con 0x7f3fe8106850 2024-03-04T08:48:25.866 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.012+0000 7f3fee44b700 1 -- 172.21.15.71:0/571995556 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe819a280 con 0x7f3fe8106850 2024-03-04T08:48:25.866 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.012+0000 7f3fe57fa700 1 -- 172.21.15.71:0/571995556 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 12) v1 ==== 45941+0+0 (secure 0 0 0) 0x7f3fd802b3f0 con 0x7f3fe8106850 2024-03-04T08:48:25.867 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.013+0000 7f3fe57fa700 1 --2- 172.21.15.71:0/571995556 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3fd003cfc0 0x7f3fd003f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:25.867 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.013+0000 7f3fee44b700 1 -- 172.21.15.71:0/571995556 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe8069150 con 0x7f3fe8106850 2024-03-04T08:48:25.867 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.013+0000 7f3fe57fa700 1 -- 172.21.15.71:0/571995556 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3fd804d0d0 con 0x7f3fe8106850 2024-03-04T08:48:25.867 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.013+0000 7f3fe77fe700 1 --2- 172.21.15.71:0/571995556 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3fd003cfc0 0x7f3fd003f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:25.867 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.013+0000 7f3fe77fe700 1 --2- 172.21.15.71:0/571995556 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3fd003cfc0 0x7f3fd003f480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc006fd0 tx=0x7f3fdc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:25.867 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.018+0000 7f3fe57fa700 1 -- 172.21.15.71:0/571995556 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3fd8015070 con 0x7f3fe8106850 2024-03-04T08:48:25.867 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.240+0000 7f3fee44b700 1 -- 172.21.15.71:0/571995556 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fe8062500 con 0x7f3fd003cfc0 2024-03-04T08:48:25.867 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.292+0000 7f3fe57fa700 1 -- 172.21.15.71:0/571995556 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f3fe8062500 con 0x7f3fd003cfc0 2024-03-04T08:48:25.867 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.294+0000 7f3fee44b700 1 -- 172.21.15.71:0/571995556 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3fd003cfc0 msgr2=0x7f3fd003f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:25.867 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.294+0000 7f3fee44b700 1 --2- 172.21.15.71:0/571995556 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3fd003cfc0 0x7f3fd003f480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc006fd0 tx=0x7f3fdc006e40 comp rx=0 tx=0).stop 2024-03-04T08:48:25.867 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.294+0000 7f3fee44b700 1 -- 172.21.15.71:0/571995556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fe8106850 msgr2=0x7f3fe8199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:25.867 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.294+0000 7f3fee44b700 1 --2- 172.21.15.71:0/571995556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fe8106850 0x7f3fe8199710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009fb0 tx=0x7f3fd8003960 comp rx=0 tx=0).stop 2024-03-04T08:48:25.867 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.294+0000 7f3fee44b700 1 -- 172.21.15.71:0/571995556 shutdown_connections 2024-03-04T08:48:25.867 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.294+0000 7f3fee44b700 1 --2- 172.21.15.71:0/571995556 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3fd003cfc0 0x7f3fd003f480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:25.868 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.294+0000 7f3fee44b700 1 --2- 172.21.15.71:0/571995556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fe8106850 0x7f3fe8199710 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:25.868 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.294+0000 7f3fee44b700 1 -- 172.21.15.71:0/571995556 >> 172.21.15.71:0/571995556 conn(0x7f3fe8101ed0 msgr2=0x7f3fe818fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:25.868 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.294+0000 7f3fee44b700 1 -- 172.21.15.71:0/571995556 shutdown_connections 2024-03-04T08:48:25.868 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:25.294+0000 7f3fee44b700 1 -- 172.21.15.71:0/571995556 wait complete. 2024-03-04T08:48:25.868 INFO:teuthology.orchestra.run.smithi071.stdout:Creating initial admin user... 2024-03-04T08:48:26.134 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:25 smithi071 ceph-mon[32789]: mgrmap e12: x(active, since 2s) 2024-03-04T08:48:26.134 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:25 smithi071 ceph-mon[32789]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:48:26.134 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:25 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:26.134 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:25 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:26.134 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:25 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:27.586 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$soJ5kpHnewBzNXLlxsYaH.q.WMooIrQ9ohPfUZICP03j8ebaBT5/6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1709542106, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-04T08:48:27.586 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.238+0000 7fa8cf525700 1 Processor -- start 2024-03-04T08:48:27.586 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.239+0000 7fa8cf525700 1 -- start start 2024-03-04T08:48:27.586 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.239+0000 7fa8cf525700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c8107a10 0x7fa8c8107e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:27.586 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.239+0000 7fa8cf525700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c8108400 con 0x7fa8c8107a10 2024-03-04T08:48:27.586 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.239+0000 7fa8cd2c1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c8107a10 0x7fa8c8107e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:27.587 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.239+0000 7fa8cd2c1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c8107a10 0x7fa8c8107e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56782/0 (socket says 172.21.15.71:56782) 2024-03-04T08:48:27.587 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.239+0000 7fa8cd2c1700 1 -- 172.21.15.71:0/4264603822 learned_addr learned my addr 172.21.15.71:0/4264603822 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:27.587 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.239+0000 7fa8cd2c1700 1 -- 172.21.15.71:0/4264603822 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c8108c10 con 0x7fa8c8107a10 2024-03-04T08:48:27.587 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.239+0000 7fa8cd2c1700 1 --2- 172.21.15.71:0/4264603822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c8107a10 0x7fa8c8107e30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80098a0 tx=0x7fa8b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d157abac60344c96 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:27.587 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.240+0000 7fa8bffff700 1 -- 172.21.15.71:0/4264603822 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa8b8004160 con 0x7fa8c8107a10 2024-03-04T08:48:27.587 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.240+0000 7fa8bffff700 1 -- 172.21.15.71:0/4264603822 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8b8004300 con 0x7fa8c8107a10 2024-03-04T08:48:27.587 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.240+0000 7fa8bffff700 1 -- 172.21.15.71:0/4264603822 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa8b8011b20 con 0x7fa8c8107a10 2024-03-04T08:48:27.587 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.240+0000 7fa8cf525700 1 -- 172.21.15.71:0/4264603822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c8107a10 msgr2=0x7fa8c8107e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:27.587 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.240+0000 7fa8cf525700 1 --2- 172.21.15.71:0/4264603822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c8107a10 0x7fa8c8107e30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80098a0 tx=0x7fa8b8009bb0 comp rx=0 tx=0).stop 2024-03-04T08:48:27.587 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.240+0000 7fa8cf525700 1 -- 172.21.15.71:0/4264603822 shutdown_connections 2024-03-04T08:48:27.587 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.240+0000 7fa8cf525700 1 --2- 172.21.15.71:0/4264603822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c8107a10 0x7fa8c8107e30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:27.588 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.240+0000 7fa8cf525700 1 -- 172.21.15.71:0/4264603822 >> 172.21.15.71:0/4264603822 conn(0x7fa8c8103070 msgr2=0x7fa8c8105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:27.588 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.240+0000 7fa8cf525700 1 -- 172.21.15.71:0/4264603822 shutdown_connections 2024-03-04T08:48:27.588 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.240+0000 7fa8cf525700 1 -- 172.21.15.71:0/4264603822 wait complete. 2024-03-04T08:48:27.588 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.241+0000 7fa8cf525700 1 Processor -- start 2024-03-04T08:48:27.589 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.241+0000 7fa8cf525700 1 -- start start 2024-03-04T08:48:27.589 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.241+0000 7fa8cf525700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c8107a10 0x7fa8c819b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:27.589 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.241+0000 7fa8cf525700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c810bfc0 con 0x7fa8c8107a10 2024-03-04T08:48:27.589 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.241+0000 7fa8cd2c1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c8107a10 0x7fa8c819b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:27.589 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.241+0000 7fa8cd2c1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c8107a10 0x7fa8c819b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56796/0 (socket says 172.21.15.71:56796) 2024-03-04T08:48:27.589 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.241+0000 7fa8cd2c1700 1 -- 172.21.15.71:0/94228668 learned_addr learned my addr 172.21.15.71:0/94228668 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:27.589 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.241+0000 7fa8cd2c1700 1 -- 172.21.15.71:0/94228668 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c819beb0 con 0x7fa8c8107a10 2024-03-04T08:48:27.589 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.242+0000 7fa8cd2c1700 1 --2- 172.21.15.71:0/94228668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c8107a10 0x7fa8c819b970 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8000c00 tx=0x7fa8b8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:27.589 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.242+0000 7fa8be7fc700 1 -- 172.21.15.71:0/94228668 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa8b8003b60 con 0x7fa8c8107a10 2024-03-04T08:48:27.589 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.242+0000 7fa8be7fc700 1 -- 172.21.15.71:0/94228668 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8b801a910 con 0x7fa8c8107a10 2024-03-04T08:48:27.590 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.242+0000 7fa8cf525700 1 -- 172.21.15.71:0/94228668 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c819c140 con 0x7fa8c8107a10 2024-03-04T08:48:27.590 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.242+0000 7fa8be7fc700 1 -- 172.21.15.71:0/94228668 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa8b8024930 con 0x7fa8c8107a10 2024-03-04T08:48:27.590 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.242+0000 7fa8cf525700 1 -- 172.21.15.71:0/94228668 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c819c5a0 con 0x7fa8c8107a10 2024-03-04T08:48:27.590 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.243+0000 7fa8be7fc700 1 -- 172.21.15.71:0/94228668 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 12) v1 ==== 45941+0+0 (secure 0 0 0) 0x7fa8b8022020 con 0x7fa8c8107a10 2024-03-04T08:48:27.590 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.243+0000 7fa8be7fc700 1 --2- 172.21.15.71:0/94228668 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa8b4038c50 0x7fa8b403b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:27.590 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.243+0000 7fa8cf525700 1 -- 172.21.15.71:0/94228668 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c8069150 con 0x7fa8c8107a10 2024-03-04T08:48:27.590 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.243+0000 7fa8be7fc700 1 -- 172.21.15.71:0/94228668 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa8b804ccf0 con 0x7fa8c8107a10 2024-03-04T08:48:27.590 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.243+0000 7fa8ccac0700 1 --2- 172.21.15.71:0/94228668 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa8b4038c50 0x7fa8b403b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:27.590 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.243+0000 7fa8ccac0700 1 --2- 172.21.15.71:0/94228668 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa8b4038c50 0x7fa8b403b110 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4006fd0 tx=0x7fa8c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:27.590 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.248+0000 7fa8be7fc700 1 -- 172.21.15.71:0/94228668 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa8b8016260 con 0x7fa8c8107a10 2024-03-04T08:48:27.591 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.474+0000 7fa8cf525700 1 -- 172.21.15.71:0/94228668 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fa8c819eef0 con 0x7fa8b4038c50 2024-03-04T08:48:27.591 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.697+0000 7fa8be7fc700 1 -- 172.21.15.71:0/94228668 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fa8c819eef0 con 0x7fa8b4038c50 2024-03-04T08:48:27.591 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.699+0000 7fa8cf525700 1 -- 172.21.15.71:0/94228668 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa8b4038c50 msgr2=0x7fa8b403b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:27.591 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.699+0000 7fa8cf525700 1 --2- 172.21.15.71:0/94228668 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa8b4038c50 0x7fa8b403b110 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4006fd0 tx=0x7fa8c4006e40 comp rx=0 tx=0).stop 2024-03-04T08:48:27.591 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.699+0000 7fa8cf525700 1 -- 172.21.15.71:0/94228668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c8107a10 msgr2=0x7fa8c819b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:27.591 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.699+0000 7fa8cf525700 1 --2- 172.21.15.71:0/94228668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c8107a10 0x7fa8c819b970 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8000c00 tx=0x7fa8b8011c10 comp rx=0 tx=0).stop 2024-03-04T08:48:27.591 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.699+0000 7fa8cf525700 1 -- 172.21.15.71:0/94228668 shutdown_connections 2024-03-04T08:48:27.591 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.699+0000 7fa8cf525700 1 --2- 172.21.15.71:0/94228668 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa8b4038c50 0x7fa8b403b110 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:27.591 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.699+0000 7fa8cf525700 1 --2- 172.21.15.71:0/94228668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8c8107a10 0x7fa8c819b970 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:27.592 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.699+0000 7fa8cf525700 1 -- 172.21.15.71:0/94228668 >> 172.21.15.71:0/94228668 conn(0x7fa8c8103070 msgr2=0x7fa8c8103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:27.592 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.699+0000 7fa8cf525700 1 -- 172.21.15.71:0/94228668 shutdown_connections 2024-03-04T08:48:27.592 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:26.699+0000 7fa8cf525700 1 -- 172.21.15.71:0/94228668 wait complete. 2024-03-04T08:48:27.592 INFO:teuthology.orchestra.run.smithi071.stdout:Fetching dashboard port number... 2024-03-04T08:48:27.853 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:27 smithi071 ceph-mon[32789]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:48:27.853 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:27 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:28.808 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:28 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3066943026' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-04T08:48:29.092 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 8443 2024-03-04T08:48:29.092 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.049+0000 7f18b3939700 1 Processor -- start 2024-03-04T08:48:29.092 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.049+0000 7f18b3939700 1 -- start start 2024-03-04T08:48:29.093 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.049+0000 7f18b3939700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18ac107a60 0x7f18ac107e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:29.093 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.049+0000 7f18b3939700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac108450 con 0x7f18ac107a60 2024-03-04T08:48:29.093 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.049+0000 7f18b16d5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18ac107a60 0x7f18ac107e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:29.093 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.049+0000 7f18b16d5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18ac107a60 0x7f18ac107e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56798/0 (socket says 172.21.15.71:56798) 2024-03-04T08:48:29.093 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.049+0000 7f18b16d5700 1 -- 172.21.15.71:0/2533858182 learned_addr learned my addr 172.21.15.71:0/2533858182 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:29.093 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.050+0000 7f18b16d5700 1 -- 172.21.15.71:0/2533858182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ac108c10 con 0x7f18ac107a60 2024-03-04T08:48:29.093 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.050+0000 7f18b16d5700 1 --2- 172.21.15.71:0/2533858182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18ac107a60 0x7f18ac107e80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f18a80098a0 tx=0x7f18a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fbda02195c65db server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:29.093 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.050+0000 7f18a3fff700 1 -- 172.21.15.71:0/2533858182 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18a8004160 con 0x7f18ac107a60 2024-03-04T08:48:29.093 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.050+0000 7f18a3fff700 1 -- 172.21.15.71:0/2533858182 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18a8004300 con 0x7f18ac107a60 2024-03-04T08:48:29.093 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.050+0000 7f18a3fff700 1 -- 172.21.15.71:0/2533858182 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18a8011b20 con 0x7f18ac107a60 2024-03-04T08:48:29.093 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.050+0000 7f18b3939700 1 -- 172.21.15.71:0/2533858182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18ac107a60 msgr2=0x7f18ac107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:29.093 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.050+0000 7f18b3939700 1 --2- 172.21.15.71:0/2533858182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18ac107a60 0x7f18ac107e80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f18a80098a0 tx=0x7f18a8009bb0 comp rx=0 tx=0).stop 2024-03-04T08:48:29.093 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.050+0000 7f18b3939700 1 -- 172.21.15.71:0/2533858182 shutdown_connections 2024-03-04T08:48:29.093 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.050+0000 7f18b3939700 1 --2- 172.21.15.71:0/2533858182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18ac107a60 0x7f18ac107e80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:29.094 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.050+0000 7f18b3939700 1 -- 172.21.15.71:0/2533858182 >> 172.21.15.71:0/2533858182 conn(0x7f18ac103060 msgr2=0x7f18ac105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:29.094 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.051+0000 7f18b3939700 1 -- 172.21.15.71:0/2533858182 shutdown_connections 2024-03-04T08:48:29.094 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.051+0000 7f18b3939700 1 -- 172.21.15.71:0/2533858182 wait complete. 2024-03-04T08:48:29.094 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.051+0000 7f18b3939700 1 Processor -- start 2024-03-04T08:48:29.094 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.051+0000 7f18b3939700 1 -- start start 2024-03-04T08:48:29.094 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.051+0000 7f18b3939700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18ac107a60 0x7f18ac07cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:29.094 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.051+0000 7f18b3939700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac10bfc0 con 0x7f18ac107a60 2024-03-04T08:48:29.094 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.051+0000 7f18b16d5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18ac107a60 0x7f18ac07cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:29.094 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.051+0000 7f18b16d5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18ac107a60 0x7f18ac07cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56810/0 (socket says 172.21.15.71:56810) 2024-03-04T08:48:29.094 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.051+0000 7f18b16d5700 1 -- 172.21.15.71:0/3066943026 learned_addr learned my addr 172.21.15.71:0/3066943026 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:29.094 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.052+0000 7f18b16d5700 1 -- 172.21.15.71:0/3066943026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ac07d0a0 con 0x7f18ac107a60 2024-03-04T08:48:29.094 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.052+0000 7f18b16d5700 1 --2- 172.21.15.71:0/3066943026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18ac107a60 0x7f18ac07cb60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f18a80117e0 tx=0x7f18a8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:29.094 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.052+0000 7f18a27fc700 1 -- 172.21.15.71:0/3066943026 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18a8003bf0 con 0x7f18ac107a60 2024-03-04T08:48:29.095 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.052+0000 7f18b3939700 1 -- 172.21.15.71:0/3066943026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ac07b290 con 0x7f18ac107a60 2024-03-04T08:48:29.095 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.052+0000 7f18a27fc700 1 -- 172.21.15.71:0/3066943026 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18a801a910 con 0x7f18ac107a60 2024-03-04T08:48:29.095 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.052+0000 7f18a27fc700 1 -- 172.21.15.71:0/3066943026 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18a80249a0 con 0x7f18ac107a60 2024-03-04T08:48:29.095 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.052+0000 7f18b3939700 1 -- 172.21.15.71:0/3066943026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ac07b750 con 0x7f18ac107a60 2024-03-04T08:48:29.095 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.053+0000 7f18a27fc700 1 -- 172.21.15.71:0/3066943026 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 12) v1 ==== 45941+0+0 (secure 0 0 0) 0x7f18a8022020 con 0x7f18ac107a60 2024-03-04T08:48:29.095 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.053+0000 7f18a27fc700 1 --2- 172.21.15.71:0/3066943026 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f1898038ba0 0x7f189803b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:29.095 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.053+0000 7f18b3939700 1 -- 172.21.15.71:0/3066943026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18ac0636f0 con 0x7f18ac107a60 2024-03-04T08:48:29.095 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.053+0000 7f18a27fc700 1 -- 172.21.15.71:0/3066943026 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f18a804d010 con 0x7f18ac107a60 2024-03-04T08:48:29.095 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.053+0000 7f18b0ed4700 1 --2- 172.21.15.71:0/3066943026 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f1898038ba0 0x7f189803b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:29.095 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.053+0000 7f18b0ed4700 1 --2- 172.21.15.71:0/3066943026 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f1898038ba0 0x7f189803b060 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f189c006fd0 tx=0x7f189c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:29.095 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.058+0000 7f18a27fc700 1 -- 172.21.15.71:0/3066943026 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f18a8011e20 con 0x7f18ac107a60 2024-03-04T08:48:29.095 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.257+0000 7f18b3939700 1 -- 172.21.15.71:0/3066943026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f18ac07b9d0 con 0x7f18ac107a60 2024-03-04T08:48:29.095 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.257+0000 7f18a27fc700 1 -- 172.21.15.71:0/3066943026 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f18a800b9d0 con 0x7f18ac107a60 2024-03-04T08:48:29.095 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.259+0000 7f18b3939700 1 -- 172.21.15.71:0/3066943026 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f1898038ba0 msgr2=0x7f189803b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:29.096 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.259+0000 7f18b3939700 1 --2- 172.21.15.71:0/3066943026 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f1898038ba0 0x7f189803b060 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f189c006fd0 tx=0x7f189c006e40 comp rx=0 tx=0).stop 2024-03-04T08:48:29.096 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.259+0000 7f18b3939700 1 -- 172.21.15.71:0/3066943026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18ac107a60 msgr2=0x7f18ac07cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:29.096 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.259+0000 7f18b3939700 1 --2- 172.21.15.71:0/3066943026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18ac107a60 0x7f18ac07cb60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f18a80117e0 tx=0x7f18a8011c10 comp rx=0 tx=0).stop 2024-03-04T08:48:29.096 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.260+0000 7f18b3939700 1 -- 172.21.15.71:0/3066943026 shutdown_connections 2024-03-04T08:48:29.096 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.260+0000 7f18b3939700 1 --2- 172.21.15.71:0/3066943026 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f1898038ba0 0x7f189803b060 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:29.096 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.260+0000 7f18b3939700 1 --2- 172.21.15.71:0/3066943026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18ac107a60 0x7f18ac07cb60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:29.096 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.260+0000 7f18b3939700 1 -- 172.21.15.71:0/3066943026 >> 172.21.15.71:0/3066943026 conn(0x7f18ac103060 msgr2=0x7f18ac103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:29.096 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.260+0000 7f18b3939700 1 -- 172.21.15.71:0/3066943026 shutdown_connections 2024-03-04T08:48:29.096 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:28.260+0000 7f18b3939700 1 -- 172.21.15.71:0/3066943026 wait complete. 2024-03-04T08:48:29.099 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-04T08:48:29.099 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout disabled 2024-03-04T08:48:29.107 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-04T08:48:29.107 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-03-04T08:48:29.107 INFO:teuthology.orchestra.run.smithi071.stdout:firewalld.service is not enabled 2024-03-04T08:48:29.107 INFO:teuthology.orchestra.run.smithi071.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-04T08:48:29.108 INFO:teuthology.orchestra.run.smithi071.stdout:Ceph Dashboard is now available at: 2024-03-04T08:48:29.109 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:48:29.109 INFO:teuthology.orchestra.run.smithi071.stdout: URL: https://smithi071.front.sepia.ceph.com:8443/ 2024-03-04T08:48:29.109 INFO:teuthology.orchestra.run.smithi071.stdout: User: admin 2024-03-04T08:48:29.109 INFO:teuthology.orchestra.run.smithi071.stdout: Password: mt17iazqh6 2024-03-04T08:48:29.109 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:48:29.109 INFO:teuthology.orchestra.run.smithi071.stdout:Saving cluster configuration to /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config directory 2024-03-04T08:48:29.110 INFO:teuthology.orchestra.run.smithi071.stdout:Enabling autotune for osd_memory_target 2024-03-04T08:48:30.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:29 smithi071 ceph-mon[32789]: mgrmap e13: x(active, since 6s) 2024-03-04T08:48:30.341 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.551+0000 7f43c3045700 1 Processor -- start 2024-03-04T08:48:30.341 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.551+0000 7f43c3045700 1 -- start start 2024-03-04T08:48:30.341 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.551+0000 7f43c3045700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43bc107a20 0x7f43bc107e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:30.341 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.551+0000 7f43c3045700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc108410 con 0x7f43bc107a20 2024-03-04T08:48:30.341 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.551+0000 7f43c0de1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43bc107a20 0x7f43bc107e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:30.341 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.551+0000 7f43c0de1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43bc107a20 0x7f43bc107e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56824/0 (socket says 172.21.15.71:56824) 2024-03-04T08:48:30.341 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.551+0000 7f43c0de1700 1 -- 172.21.15.71:0/426637678 learned_addr learned my addr 172.21.15.71:0/426637678 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:30.341 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.552+0000 7f43c0de1700 1 -- 172.21.15.71:0/426637678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43bc108bd0 con 0x7f43bc107a20 2024-03-04T08:48:30.341 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.552+0000 7f43c0de1700 1 --2- 172.21.15.71:0/426637678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43bc107a20 0x7f43bc107e40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f43b00098a0 tx=0x7f43b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9da91c6b1ca166b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:30.341 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.552+0000 7f43bb7fe700 1 -- 172.21.15.71:0/426637678 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43b0004160 con 0x7f43bc107a20 2024-03-04T08:48:30.341 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.552+0000 7f43bb7fe700 1 -- 172.21.15.71:0/426637678 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43b0004300 con 0x7f43bc107a20 2024-03-04T08:48:30.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.552+0000 7f43bb7fe700 1 -- 172.21.15.71:0/426637678 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43b0011b20 con 0x7f43bc107a20 2024-03-04T08:48:30.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.552+0000 7f43c3045700 1 -- 172.21.15.71:0/426637678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43bc107a20 msgr2=0x7f43bc107e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:30.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.552+0000 7f43c3045700 1 --2- 172.21.15.71:0/426637678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43bc107a20 0x7f43bc107e40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f43b00098a0 tx=0x7f43b0009bb0 comp rx=0 tx=0).stop 2024-03-04T08:48:30.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.553+0000 7f43c3045700 1 -- 172.21.15.71:0/426637678 shutdown_connections 2024-03-04T08:48:30.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.553+0000 7f43c3045700 1 --2- 172.21.15.71:0/426637678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43bc107a20 0x7f43bc107e40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:30.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.553+0000 7f43c3045700 1 -- 172.21.15.71:0/426637678 >> 172.21.15.71:0/426637678 conn(0x7f43bc103040 msgr2=0x7f43bc105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:30.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.553+0000 7f43c3045700 1 -- 172.21.15.71:0/426637678 shutdown_connections 2024-03-04T08:48:30.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.553+0000 7f43c3045700 1 -- 172.21.15.71:0/426637678 wait complete. 2024-03-04T08:48:30.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.553+0000 7f43c3045700 1 Processor -- start 2024-03-04T08:48:30.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.553+0000 7f43c3045700 1 -- start start 2024-03-04T08:48:30.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.553+0000 7f43c3045700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43bc107a20 0x7f43bc07cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:30.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.553+0000 7f43c3045700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc10bf80 con 0x7f43bc107a20 2024-03-04T08:48:30.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.553+0000 7f43c0de1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43bc107a20 0x7f43bc07cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:30.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.553+0000 7f43c0de1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43bc107a20 0x7f43bc07cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56838/0 (socket says 172.21.15.71:56838) 2024-03-04T08:48:30.343 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.553+0000 7f43c0de1700 1 -- 172.21.15.71:0/2985271831 learned_addr learned my addr 172.21.15.71:0/2985271831 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:30.343 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.554+0000 7f43c0de1700 1 -- 172.21.15.71:0/2985271831 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43bc07d0a0 con 0x7f43bc107a20 2024-03-04T08:48:30.343 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.554+0000 7f43c0de1700 1 --2- 172.21.15.71:0/2985271831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43bc107a20 0x7f43bc07cb60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f43b00117e0 tx=0x7f43b0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:30.343 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.554+0000 7f43b9ffb700 1 -- 172.21.15.71:0/2985271831 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43b0003bf0 con 0x7f43bc107a20 2024-03-04T08:48:30.343 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.554+0000 7f43b9ffb700 1 -- 172.21.15.71:0/2985271831 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43b001a910 con 0x7f43bc107a20 2024-03-04T08:48:30.343 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.554+0000 7f43c3045700 1 -- 172.21.15.71:0/2985271831 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43bc07b290 con 0x7f43bc107a20 2024-03-04T08:48:30.343 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.554+0000 7f43b9ffb700 1 -- 172.21.15.71:0/2985271831 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43b00249a0 con 0x7f43bc107a20 2024-03-04T08:48:30.343 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.554+0000 7f43c3045700 1 -- 172.21.15.71:0/2985271831 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43bc07b750 con 0x7f43bc107a20 2024-03-04T08:48:30.343 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.555+0000 7f43b9ffb700 1 -- 172.21.15.71:0/2985271831 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f43b0022020 con 0x7f43bc107a20 2024-03-04T08:48:30.343 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.555+0000 7f43b9ffb700 1 --2- 172.21.15.71:0/2985271831 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f43ac038c40 0x7f43ac03b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:30.343 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.555+0000 7f43b9ffb700 1 -- 172.21.15.71:0/2985271831 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f43b004cf90 con 0x7f43bc107a20 2024-03-04T08:48:30.343 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.555+0000 7f43c3045700 1 -- 172.21.15.71:0/2985271831 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43bc0636f0 con 0x7f43bc107a20 2024-03-04T08:48:30.344 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.555+0000 7f43bbfff700 1 --2- 172.21.15.71:0/2985271831 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f43ac038c40 0x7f43ac03b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:30.344 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.556+0000 7f43bbfff700 1 --2- 172.21.15.71:0/2985271831 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f43ac038c40 0x7f43ac03b100 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f43a8006fd0 tx=0x7f43a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:30.344 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.560+0000 7f43b9ffb700 1 -- 172.21.15.71:0/2985271831 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f43b0011e20 con 0x7f43bc107a20 2024-03-04T08:48:30.344 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.754+0000 7f43c3045700 1 -- 172.21.15.71:0/2985271831 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f43bc07b9d0 con 0x7f43bc107a20 2024-03-04T08:48:30.344 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.754+0000 7f43b9ffb700 1 -- 172.21.15.71:0/2985271831 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f43b0032020 con 0x7f43bc107a20 2024-03-04T08:48:30.344 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.756+0000 7f43c3045700 1 -- 172.21.15.71:0/2985271831 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f43ac038c40 msgr2=0x7f43ac03b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:30.344 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.756+0000 7f43c3045700 1 --2- 172.21.15.71:0/2985271831 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f43ac038c40 0x7f43ac03b100 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f43a8006fd0 tx=0x7f43a8006e40 comp rx=0 tx=0).stop 2024-03-04T08:48:30.345 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.756+0000 7f43c3045700 1 -- 172.21.15.71:0/2985271831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43bc107a20 msgr2=0x7f43bc07cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:30.345 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.756+0000 7f43c3045700 1 --2- 172.21.15.71:0/2985271831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43bc107a20 0x7f43bc07cb60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f43b00117e0 tx=0x7f43b0011c10 comp rx=0 tx=0).stop 2024-03-04T08:48:30.345 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.756+0000 7f43c3045700 1 -- 172.21.15.71:0/2985271831 shutdown_connections 2024-03-04T08:48:30.345 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.756+0000 7f43c3045700 1 --2- 172.21.15.71:0/2985271831 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f43ac038c40 0x7f43ac03b100 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:30.345 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.756+0000 7f43c3045700 1 --2- 172.21.15.71:0/2985271831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43bc107a20 0x7f43bc07cb60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:30.345 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.756+0000 7f43c3045700 1 -- 172.21.15.71:0/2985271831 >> 172.21.15.71:0/2985271831 conn(0x7f43bc103040 msgr2=0x7f43bc103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:30.345 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.756+0000 7f43c3045700 1 -- 172.21.15.71:0/2985271831 shutdown_connections 2024-03-04T08:48:30.345 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:29.756+0000 7f43c3045700 1 -- 172.21.15.71:0/2985271831 wait complete. 2024-03-04T08:48:31.640 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.738+0000 7fbd8c990700 1 Processor -- start 2024-03-04T08:48:31.640 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.738+0000 7fbd8c990700 1 -- start start 2024-03-04T08:48:31.640 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.738+0000 7fbd8c990700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd88107f10 0x7fbd88108330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:31.640 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.738+0000 7fbd8c990700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd88108900 con 0x7fbd88107f10 2024-03-04T08:48:31.640 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.738+0000 7fbd8659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd88107f10 0x7fbd88108330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:31.640 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.738+0000 7fbd8659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd88107f10 0x7fbd88108330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56848/0 (socket says 172.21.15.71:56848) 2024-03-04T08:48:31.641 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.738+0000 7fbd8659c700 1 -- 172.21.15.71:0/4109583240 learned_addr learned my addr 172.21.15.71:0/4109583240 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:31.641 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.738+0000 7fbd8659c700 1 -- 172.21.15.71:0/4109583240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd88108a80 con 0x7fbd88107f10 2024-03-04T08:48:31.641 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.739+0000 7fbd8659c700 1 --2- 172.21.15.71:0/4109583240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd88107f10 0x7fbd88108330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbd780098a0 tx=0x7fbd78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57fe5cc018fe705b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:31.641 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.739+0000 7fbd8559a700 1 -- 172.21.15.71:0/4109583240 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd78004160 con 0x7fbd88107f10 2024-03-04T08:48:31.641 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.739+0000 7fbd8559a700 1 -- 172.21.15.71:0/4109583240 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd78004300 con 0x7fbd88107f10 2024-03-04T08:48:31.641 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.739+0000 7fbd8559a700 1 -- 172.21.15.71:0/4109583240 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd78011b20 con 0x7fbd88107f10 2024-03-04T08:48:31.641 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.739+0000 7fbd8c990700 1 -- 172.21.15.71:0/4109583240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd88107f10 msgr2=0x7fbd88108330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:31.641 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.739+0000 7fbd8c990700 1 --2- 172.21.15.71:0/4109583240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd88107f10 0x7fbd88108330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbd780098a0 tx=0x7fbd78009bb0 comp rx=0 tx=0).stop 2024-03-04T08:48:31.641 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.739+0000 7fbd8c990700 1 -- 172.21.15.71:0/4109583240 shutdown_connections 2024-03-04T08:48:31.641 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.739+0000 7fbd8c990700 1 --2- 172.21.15.71:0/4109583240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd88107f10 0x7fbd88108330 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:31.642 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.739+0000 7fbd8c990700 1 -- 172.21.15.71:0/4109583240 >> 172.21.15.71:0/4109583240 conn(0x7fbd881016a0 msgr2=0x7fbd88103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:31.642 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.740+0000 7fbd8c990700 1 -- 172.21.15.71:0/4109583240 shutdown_connections 2024-03-04T08:48:31.642 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.740+0000 7fbd8c990700 1 -- 172.21.15.71:0/4109583240 wait complete. 2024-03-04T08:48:31.642 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.740+0000 7fbd8c990700 1 Processor -- start 2024-03-04T08:48:31.642 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.740+0000 7fbd8c990700 1 -- start start 2024-03-04T08:48:31.642 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.740+0000 7fbd8c990700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd88107f10 0x7fbd8819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:31.642 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.740+0000 7fbd8c990700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd88106930 con 0x7fbd88107f10 2024-03-04T08:48:31.642 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.740+0000 7fbd8659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd88107f10 0x7fbd8819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:31.642 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.740+0000 7fbd8659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd88107f10 0x7fbd8819db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56862/0 (socket says 172.21.15.71:56862) 2024-03-04T08:48:31.642 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.740+0000 7fbd8659c700 1 -- 172.21.15.71:0/3894308547 learned_addr learned my addr 172.21.15.71:0/3894308547 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:31.642 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.741+0000 7fbd8659c700 1 -- 172.21.15.71:0/3894308547 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8819e0b0 con 0x7fbd88107f10 2024-03-04T08:48:31.642 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.741+0000 7fbd8659c700 1 --2- 172.21.15.71:0/3894308547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd88107f10 0x7fbd8819db70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009fb0 tx=0x7fbd78011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:31.643 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.741+0000 7fbd777fe700 1 -- 172.21.15.71:0/3894308547 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd78003930 con 0x7fbd88107f10 2024-03-04T08:48:31.643 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.741+0000 7fbd777fe700 1 -- 172.21.15.71:0/3894308547 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd78003ad0 con 0x7fbd88107f10 2024-03-04T08:48:31.643 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.741+0000 7fbd777fe700 1 -- 172.21.15.71:0/3894308547 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd78019690 con 0x7fbd88107f10 2024-03-04T08:48:31.643 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.741+0000 7fbd8c990700 1 -- 172.21.15.71:0/3894308547 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd8819e3a0 con 0x7fbd88107f10 2024-03-04T08:48:31.643 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.741+0000 7fbd8c990700 1 -- 172.21.15.71:0/3894308547 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd881a0fe0 con 0x7fbd88107f10 2024-03-04T08:48:31.643 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.742+0000 7fbd777fe700 1 -- 172.21.15.71:0/3894308547 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7fbd78019940 con 0x7fbd88107f10 2024-03-04T08:48:31.643 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.742+0000 7fbd8c990700 1 -- 172.21.15.71:0/3894308547 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd880636f0 con 0x7fbd88107f10 2024-03-04T08:48:31.643 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.742+0000 7fbd777fe700 1 --2- 172.21.15.71:0/3894308547 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fbd70038ca0 0x7fbd7003b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:31.643 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.742+0000 7fbd777fe700 1 -- 172.21.15.71:0/3894308547 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbd7804d300 con 0x7fbd88107f10 2024-03-04T08:48:31.643 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.742+0000 7fbd85d9b700 1 --2- 172.21.15.71:0/3894308547 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fbd70038ca0 0x7fbd7003b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:31.643 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.742+0000 7fbd85d9b700 1 --2- 172.21.15.71:0/3894308547 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fbd70038ca0 0x7fbd7003b160 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c006fd0 tx=0x7fbd7c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:31.644 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:30.747+0000 7fbd777fe700 1 -- 172.21.15.71:0/3894308547 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbd78011e20 con 0x7fbd88107f10 2024-03-04T08:48:31.644 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:31.050+0000 7fbd8c990700 1 -- 172.21.15.71:0/3894308547 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fbd881970d0 con 0x7fbd88107f10 2024-03-04T08:48:31.644 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:31.052+0000 7fbd777fe700 1 -- 172.21.15.71:0/3894308547 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7fbd881970d0 con 0x7fbd88107f10 2024-03-04T08:48:31.644 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:31.054+0000 7fbd8c990700 1 -- 172.21.15.71:0/3894308547 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fbd70038ca0 msgr2=0x7fbd7003b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:31.644 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:31.054+0000 7fbd8c990700 1 --2- 172.21.15.71:0/3894308547 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fbd70038ca0 0x7fbd7003b160 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c006fd0 tx=0x7fbd7c006e40 comp rx=0 tx=0).stop 2024-03-04T08:48:31.644 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:31.054+0000 7fbd8c990700 1 -- 172.21.15.71:0/3894308547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd88107f10 msgr2=0x7fbd8819db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:31.644 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:31.054+0000 7fbd8c990700 1 --2- 172.21.15.71:0/3894308547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd88107f10 0x7fbd8819db70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009fb0 tx=0x7fbd78011c10 comp rx=0 tx=0).stop 2024-03-04T08:48:31.644 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:31.054+0000 7fbd8c990700 1 -- 172.21.15.71:0/3894308547 shutdown_connections 2024-03-04T08:48:31.644 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:31.054+0000 7fbd8c990700 1 --2- 172.21.15.71:0/3894308547 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fbd70038ca0 0x7fbd7003b160 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:31.644 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:31.054+0000 7fbd8c990700 1 --2- 172.21.15.71:0/3894308547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd88107f10 0x7fbd8819db70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:31.644 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:31.054+0000 7fbd8c990700 1 -- 172.21.15.71:0/3894308547 >> 172.21.15.71:0/3894308547 conn(0x7fbd881016a0 msgr2=0x7fbd88194190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:31.644 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:31.055+0000 7fbd8c990700 1 -- 172.21.15.71:0/3894308547 shutdown_connections 2024-03-04T08:48:31.644 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-03-04T08:48:31.055+0000 7fbd8c990700 1 -- 172.21.15.71:0/3894308547 wait complete. 2024-03-04T08:48:31.645 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-04T08:48:31.645 INFO:teuthology.orchestra.run.smithi071.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-04T08:48:31.645 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:48:31.645 INFO:teuthology.orchestra.run.smithi071.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-04T08:48:31.645 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:48:31.645 INFO:teuthology.orchestra.run.smithi071.stdout:Or, if you are only running a single cluster on this host: 2024-03-04T08:48:31.645 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:48:31.645 INFO:teuthology.orchestra.run.smithi071.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-04T08:48:31.645 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:48:31.645 INFO:teuthology.orchestra.run.smithi071.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-04T08:48:31.645 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:48:31.645 INFO:teuthology.orchestra.run.smithi071.stdout: ceph telemetry on 2024-03-04T08:48:31.645 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:48:31.646 INFO:teuthology.orchestra.run.smithi071.stdout:For more information see: 2024-03-04T08:48:31.646 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:48:31.646 INFO:teuthology.orchestra.run.smithi071.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-04T08:48:31.646 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:48:31.646 INFO:teuthology.orchestra.run.smithi071.stdout:Bootstrap complete. 2024-03-04T08:48:31.679 INFO:tasks.cephadm:Fetching config... 2024-03-04T08:48:31.680 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-04T08:48:31.680 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-04T08:48:31.699 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-04T08:48:31.699 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-04T08:48:31.700 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-04T08:48:31.752 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-04T08:48:31.752 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-04T08:48:31.752 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-04T08:48:31.819 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-04T08:48:31.819 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-04T08:48:31.819 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-04T08:48:31.875 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-04T08:48:31.875 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOlctcSGAm7SKs6telu5OvLIN3GIcaEB7pA8Hd5/LzxnOmVaKpKfd55rE7esuMKFJbqZhYWtmB7ibbkbqyIjWJcKDl/Vpu6JhkaC1B54bcRyZYhm554Um2MvXxsfEjFWRr9D7/txZwBzerdxaR11SBiKFZZbfM8wolT1ZiYNdJl7np7qoW61tnmze5R9kjj2x0HrhsjosuuISLMyBB+WthV1QgsAxSWLCc+LHek564P8kGhvod/YzLMApqLiPdyva9tx7QE/v8izOA1pQtgfhWRmBqh50KsVn5x4evoM2SaDng3aN86f5zVKez1KGoeZZaYQHbMaDHBWXunAklIujSYe/XQR8cxTl/Cod05OTWYcaIm+uKhNqDipAGePOaROQmubHf7RUo7SyzBns6ikVvqcrscHRBALKy4FqUGJUsAoMVWsqRmC9zBtL8NM13peHgf72x4kvDh64n1EmAHDKzmxuIJ+PLGt+puBdfi5cc7ZuF7TW8e0zDObp7n/T1sp8= ceph-b3c85c40-da03-11ee-95c6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-04T08:48:31.962 INFO:teuthology.orchestra.run.smithi071.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOlctcSGAm7SKs6telu5OvLIN3GIcaEB7pA8Hd5/LzxnOmVaKpKfd55rE7esuMKFJbqZhYWtmB7ibbkbqyIjWJcKDl/Vpu6JhkaC1B54bcRyZYhm554Um2MvXxsfEjFWRr9D7/txZwBzerdxaR11SBiKFZZbfM8wolT1ZiYNdJl7np7qoW61tnmze5R9kjj2x0HrhsjosuuISLMyBB+WthV1QgsAxSWLCc+LHek564P8kGhvod/YzLMApqLiPdyva9tx7QE/v8izOA1pQtgfhWRmBqh50KsVn5x4evoM2SaDng3aN86f5zVKez1KGoeZZaYQHbMaDHBWXunAklIujSYe/XQR8cxTl/Cod05OTWYcaIm+uKhNqDipAGePOaROQmubHf7RUo7SyzBns6ikVvqcrscHRBALKy4FqUGJUsAoMVWsqRmC9zBtL8NM13peHgf72x4kvDh64n1EmAHDKzmxuIJ+PLGt+puBdfi5cc7ZuF7TW8e0zDObp7n/T1sp8= ceph-b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T08:48:31.976 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOlctcSGAm7SKs6telu5OvLIN3GIcaEB7pA8Hd5/LzxnOmVaKpKfd55rE7esuMKFJbqZhYWtmB7ibbkbqyIjWJcKDl/Vpu6JhkaC1B54bcRyZYhm554Um2MvXxsfEjFWRr9D7/txZwBzerdxaR11SBiKFZZbfM8wolT1ZiYNdJl7np7qoW61tnmze5R9kjj2x0HrhsjosuuISLMyBB+WthV1QgsAxSWLCc+LHek564P8kGhvod/YzLMApqLiPdyva9tx7QE/v8izOA1pQtgfhWRmBqh50KsVn5x4evoM2SaDng3aN86f5zVKez1KGoeZZaYQHbMaDHBWXunAklIujSYe/XQR8cxTl/Cod05OTWYcaIm+uKhNqDipAGePOaROQmubHf7RUo7SyzBns6ikVvqcrscHRBALKy4FqUGJUsAoMVWsqRmC9zBtL8NM13peHgf72x4kvDh64n1EmAHDKzmxuIJ+PLGt+puBdfi5cc7ZuF7TW8e0zDObp7n/T1sp8= ceph-b3c85c40-da03-11ee-95c6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-04T08:48:32.030 INFO:teuthology.orchestra.run.smithi150.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOlctcSGAm7SKs6telu5OvLIN3GIcaEB7pA8Hd5/LzxnOmVaKpKfd55rE7esuMKFJbqZhYWtmB7ibbkbqyIjWJcKDl/Vpu6JhkaC1B54bcRyZYhm554Um2MvXxsfEjFWRr9D7/txZwBzerdxaR11SBiKFZZbfM8wolT1ZiYNdJl7np7qoW61tnmze5R9kjj2x0HrhsjosuuISLMyBB+WthV1QgsAxSWLCc+LHek564P8kGhvod/YzLMApqLiPdyva9tx7QE/v8izOA1pQtgfhWRmBqh50KsVn5x4evoM2SaDng3aN86f5zVKez1KGoeZZaYQHbMaDHBWXunAklIujSYe/XQR8cxTl/Cod05OTWYcaIm+uKhNqDipAGePOaROQmubHf7RUo7SyzBns6ikVvqcrscHRBALKy4FqUGJUsAoMVWsqRmC9zBtL8NM13peHgf72x4kvDh64n1EmAHDKzmxuIJ+PLGt+puBdfi5cc7ZuF7TW8e0zDObp7n/T1sp8= ceph-b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T08:48:32.046 DEBUG:teuthology.orchestra.run.smithi158:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOlctcSGAm7SKs6telu5OvLIN3GIcaEB7pA8Hd5/LzxnOmVaKpKfd55rE7esuMKFJbqZhYWtmB7ibbkbqyIjWJcKDl/Vpu6JhkaC1B54bcRyZYhm554Um2MvXxsfEjFWRr9D7/txZwBzerdxaR11SBiKFZZbfM8wolT1ZiYNdJl7np7qoW61tnmze5R9kjj2x0HrhsjosuuISLMyBB+WthV1QgsAxSWLCc+LHek564P8kGhvod/YzLMApqLiPdyva9tx7QE/v8izOA1pQtgfhWRmBqh50KsVn5x4evoM2SaDng3aN86f5zVKez1KGoeZZaYQHbMaDHBWXunAklIujSYe/XQR8cxTl/Cod05OTWYcaIm+uKhNqDipAGePOaROQmubHf7RUo7SyzBns6ikVvqcrscHRBALKy4FqUGJUsAoMVWsqRmC9zBtL8NM13peHgf72x4kvDh64n1EmAHDKzmxuIJ+PLGt+puBdfi5cc7ZuF7TW8e0zDObp7n/T1sp8= ceph-b3c85c40-da03-11ee-95c6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-04T08:48:32.093 INFO:teuthology.orchestra.run.smithi158.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOlctcSGAm7SKs6telu5OvLIN3GIcaEB7pA8Hd5/LzxnOmVaKpKfd55rE7esuMKFJbqZhYWtmB7ibbkbqyIjWJcKDl/Vpu6JhkaC1B54bcRyZYhm554Um2MvXxsfEjFWRr9D7/txZwBzerdxaR11SBiKFZZbfM8wolT1ZiYNdJl7np7qoW61tnmze5R9kjj2x0HrhsjosuuISLMyBB+WthV1QgsAxSWLCc+LHek564P8kGhvod/YzLMApqLiPdyva9tx7QE/v8izOA1pQtgfhWRmBqh50KsVn5x4evoM2SaDng3aN86f5zVKez1KGoeZZaYQHbMaDHBWXunAklIujSYe/XQR8cxTl/Cod05OTWYcaIm+uKhNqDipAGePOaROQmubHf7RUo7SyzBns6ikVvqcrscHRBALKy4FqUGJUsAoMVWsqRmC9zBtL8NM13peHgf72x4kvDh64n1EmAHDKzmxuIJ+PLGt+puBdfi5cc7ZuF7TW8e0zDObp7n/T1sp8= ceph-b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T08:48:32.108 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-04T08:48:32.349 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:48:32.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:32 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3894308547' entity='client.admin' 2024-03-04T08:48:34.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.810+0000 7f1dd2d78700 1 -- 172.21.15.71:0/184046795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dcc101b00 msgr2=0x7f1dcc101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:34.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.810+0000 7f1dd2d78700 1 --2- 172.21.15.71:0/184046795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dcc101b00 0x7f1dcc101f20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009910 tx=0x7f1dc0009c20 comp rx=0 tx=0).stop 2024-03-04T08:48:34.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.810+0000 7f1dd2d78700 1 -- 172.21.15.71:0/184046795 shutdown_connections 2024-03-04T08:48:34.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.810+0000 7f1dd2d78700 1 --2- 172.21.15.71:0/184046795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dcc101b00 0x7f1dcc101f20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:34.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.810+0000 7f1dd2d78700 1 -- 172.21.15.71:0/184046795 >> 172.21.15.71:0/184046795 conn(0x7f1dcc0fd0a0 msgr2=0x7f1dcc0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:34.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.810+0000 7f1dd2d78700 1 -- 172.21.15.71:0/184046795 shutdown_connections 2024-03-04T08:48:34.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.810+0000 7f1dd2d78700 1 -- 172.21.15.71:0/184046795 wait complete. 2024-03-04T08:48:34.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.810+0000 7f1dd2d78700 1 Processor -- start 2024-03-04T08:48:34.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.810+0000 7f1dd2d78700 1 -- start start 2024-03-04T08:48:34.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.810+0000 7f1dd2d78700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dcc101b00 0x7f1dcc075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:34.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.811+0000 7f1dd2d78700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc107b90 con 0x7f1dcc101b00 2024-03-04T08:48:34.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.811+0000 7f1dd0b14700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dcc101b00 0x7f1dcc075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:34.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.811+0000 7f1dd0b14700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dcc101b00 0x7f1dcc075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56880/0 (socket says 172.21.15.71:56880) 2024-03-04T08:48:34.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.811+0000 7f1dd0b14700 1 -- 172.21.15.71:0/122216785 learned_addr learned my addr 172.21.15.71:0/122216785 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:34.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.811+0000 7f1dd0b14700 1 -- 172.21.15.71:0/122216785 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dcc0762c0 con 0x7f1dcc101b00 2024-03-04T08:48:34.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.811+0000 7f1dd0b14700 1 --2- 172.21.15.71:0/122216785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dcc101b00 0x7f1dcc075d80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0006010 tx=0x7f1dc0005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:34.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.811+0000 7f1dc9ffb700 1 -- 172.21.15.71:0/122216785 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1dc0027070 con 0x7f1dcc101b00 2024-03-04T08:48:34.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.811+0000 7f1dd2d78700 1 -- 172.21.15.71:0/122216785 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dcc0744b0 con 0x7f1dcc101b00 2024-03-04T08:48:34.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.811+0000 7f1dc9ffb700 1 -- 172.21.15.71:0/122216785 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1dc000be40 con 0x7f1dcc101b00 2024-03-04T08:48:34.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.811+0000 7f1dc9ffb700 1 -- 172.21.15.71:0/122216785 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1dc000f460 con 0x7f1dcc101b00 2024-03-04T08:48:34.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.811+0000 7f1dd2d78700 1 -- 172.21.15.71:0/122216785 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dcc0749f0 con 0x7f1dcc101b00 2024-03-04T08:48:34.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.812+0000 7f1dc9ffb700 1 -- 172.21.15.71:0/122216785 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f1dc002a070 con 0x7f1dcc101b00 2024-03-04T08:48:34.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.812+0000 7f1dd2d78700 1 -- 172.21.15.71:0/122216785 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dcc108310 con 0x7f1dcc101b00 2024-03-04T08:48:34.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.812+0000 7f1dc9ffb700 1 --2- 172.21.15.71:0/122216785 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f1dbc038ca0 0x7f1dbc03b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:34.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.813+0000 7f1dc9ffb700 1 -- 172.21.15.71:0/122216785 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1dc001e030 con 0x7f1dcc101b00 2024-03-04T08:48:34.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.813+0000 7f1dcbfff700 1 --2- 172.21.15.71:0/122216785 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f1dbc038ca0 0x7f1dbc03b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:34.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.813+0000 7f1dcbfff700 1 --2- 172.21.15.71:0/122216785 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f1dbc038ca0 0x7f1dbc03b160 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1db8006fd0 tx=0x7f1db8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:34.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:34.817+0000 7f1dc9ffb700 1 -- 172.21.15.71:0/122216785 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1dc0013070 con 0x7f1dcc101b00 2024-03-04T08:48:34.854 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:34 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:34.854 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:34 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:34.854 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:34 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi071", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:48:34.854 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:34 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:34.855 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:34 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:48:34.855 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:34 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:35.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:35.020+0000 7f1dd2d78700 1 -- 172.21.15.71:0/122216785 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f1dcc190610 con 0x7f1dcc101b00 2024-03-04T08:48:35.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:35.023+0000 7f1dc9ffb700 1 -- 172.21.15.71:0/122216785 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f1dc002a360 con 0x7f1dcc101b00 2024-03-04T08:48:35.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:35.025+0000 7f1dd2d78700 1 -- 172.21.15.71:0/122216785 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f1dbc038ca0 msgr2=0x7f1dbc03b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:35.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:35.025+0000 7f1dd2d78700 1 --2- 172.21.15.71:0/122216785 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f1dbc038ca0 0x7f1dbc03b160 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1db8006fd0 tx=0x7f1db8006e40 comp rx=0 tx=0).stop 2024-03-04T08:48:35.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:35.025+0000 7f1dd2d78700 1 -- 172.21.15.71:0/122216785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dcc101b00 msgr2=0x7f1dcc075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:35.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:35.025+0000 7f1dd2d78700 1 --2- 172.21.15.71:0/122216785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dcc101b00 0x7f1dcc075d80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0006010 tx=0x7f1dc0005050 comp rx=0 tx=0).stop 2024-03-04T08:48:35.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:35.025+0000 7f1dd2d78700 1 -- 172.21.15.71:0/122216785 shutdown_connections 2024-03-04T08:48:35.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:35.025+0000 7f1dd2d78700 1 --2- 172.21.15.71:0/122216785 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f1dbc038ca0 0x7f1dbc03b160 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:35.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:35.025+0000 7f1dd2d78700 1 --2- 172.21.15.71:0/122216785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dcc101b00 0x7f1dcc075d80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:35.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:35.025+0000 7f1dd2d78700 1 -- 172.21.15.71:0/122216785 >> 172.21.15.71:0/122216785 conn(0x7f1dcc0fd0a0 msgr2=0x7f1dcc0fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:35.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:35.026+0000 7f1dd2d78700 1 -- 172.21.15.71:0/122216785 shutdown_connections 2024-03-04T08:48:35.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:35.026+0000 7f1dd2d78700 1 -- 172.21.15.71:0/122216785 wait complete. 2024-03-04T08:48:35.659 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-04T08:48:35.659 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-04T08:48:36.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:36 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/122216785' entity='client.admin' 2024-03-04T08:48:36.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:36 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:48:36.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:36 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:36.327 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:48:37.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.821+0000 7f97a9198700 1 -- 172.21.15.71:0/979515938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97a4101fa0 msgr2=0x7f97a41023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:37.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.821+0000 7f97a9198700 1 --2- 172.21.15.71:0/979515938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97a4101fa0 0x7f97a41023c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9794009910 tx=0x7f9794009c20 comp rx=0 tx=0).stop 2024-03-04T08:48:37.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.821+0000 7f97a9198700 1 -- 172.21.15.71:0/979515938 shutdown_connections 2024-03-04T08:48:37.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.821+0000 7f97a9198700 1 --2- 172.21.15.71:0/979515938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97a4101fa0 0x7f97a41023c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:37.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.821+0000 7f97a9198700 1 -- 172.21.15.71:0/979515938 >> 172.21.15.71:0/979515938 conn(0x7f97a40fb750 msgr2=0x7f97a40fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:37.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.821+0000 7f97a9198700 1 -- 172.21.15.71:0/979515938 shutdown_connections 2024-03-04T08:48:37.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.821+0000 7f97a9198700 1 -- 172.21.15.71:0/979515938 wait complete. 2024-03-04T08:48:37.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.822+0000 7f97a9198700 1 Processor -- start 2024-03-04T08:48:37.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.822+0000 7f97a9198700 1 -- start start 2024-03-04T08:48:37.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.822+0000 7f97a9198700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97a4101fa0 0x7f97a4197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:37.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.822+0000 7f97a9198700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a4106aa0 con 0x7f97a4101fa0 2024-03-04T08:48:37.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.822+0000 7f97a2d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97a4101fa0 0x7f97a4197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:37.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.823+0000 7f97a2d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97a4101fa0 0x7f97a4197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56248/0 (socket says 172.21.15.71:56248) 2024-03-04T08:48:37.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.823+0000 7f97a2d9d700 1 -- 172.21.15.71:0/3566635815 learned_addr learned my addr 172.21.15.71:0/3566635815 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:37.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.823+0000 7f97a2d9d700 1 -- 172.21.15.71:0/3566635815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a4197760 con 0x7f97a4101fa0 2024-03-04T08:48:37.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.823+0000 7f97a2d9d700 1 --2- 172.21.15.71:0/3566635815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97a4101fa0 0x7f97a4197220 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9794006010 tx=0x7f9794005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:37.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.823+0000 7f978bfff700 1 -- 172.21.15.71:0/3566635815 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9794027070 con 0x7f97a4101fa0 2024-03-04T08:48:37.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.823+0000 7f97a9198700 1 -- 172.21.15.71:0/3566635815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a41979f0 con 0x7f97a4101fa0 2024-03-04T08:48:37.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.823+0000 7f978bfff700 1 -- 172.21.15.71:0/3566635815 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f979400be40 con 0x7f97a4101fa0 2024-03-04T08:48:37.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.823+0000 7f978bfff700 1 -- 172.21.15.71:0/3566635815 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f979400f460 con 0x7f97a4101fa0 2024-03-04T08:48:37.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.823+0000 7f97a9198700 1 -- 172.21.15.71:0/3566635815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a419a6e0 con 0x7f97a4101fa0 2024-03-04T08:48:37.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.824+0000 7f978bfff700 1 -- 172.21.15.71:0/3566635815 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f979402a070 con 0x7f97a4101fa0 2024-03-04T08:48:37.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.824+0000 7f97a9198700 1 -- 172.21.15.71:0/3566635815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a4107220 con 0x7f97a4101fa0 2024-03-04T08:48:37.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.824+0000 7f978bfff700 1 --2- 172.21.15.71:0/3566635815 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f978c038c40 0x7f978c03b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:37.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.824+0000 7f978bfff700 1 -- 172.21.15.71:0/3566635815 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f979401e030 con 0x7f97a4101fa0 2024-03-04T08:48:37.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.824+0000 7f97a259c700 1 --2- 172.21.15.71:0/3566635815 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f978c038c40 0x7f978c03b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:37.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.825+0000 7f97a259c700 1 --2- 172.21.15.71:0/3566635815 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f978c038c40 0x7f978c03b100 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9798006fd0 tx=0x7f9798006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:37.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:37.829+0000 7f978bfff700 1 -- 172.21.15.71:0/3566635815 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9794013070 con 0x7f97a4101fa0 2024-03-04T08:48:38.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:38.027+0000 7f97a9198700 1 -- 172.21.15.71:0/3566635815 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f97a402dac0 con 0x7f978c038c40 2024-03-04T08:48:38.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:38.031+0000 7f978bfff700 1 -- 172.21.15.71:0/3566635815 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f97a402dac0 con 0x7f978c038c40 2024-03-04T08:48:38.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:38.033+0000 7f97a9198700 1 -- 172.21.15.71:0/3566635815 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f978c038c40 msgr2=0x7f978c03b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:38.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:38.033+0000 7f97a9198700 1 --2- 172.21.15.71:0/3566635815 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f978c038c40 0x7f978c03b100 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9798006fd0 tx=0x7f9798006e40 comp rx=0 tx=0).stop 2024-03-04T08:48:38.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:38.033+0000 7f97a9198700 1 -- 172.21.15.71:0/3566635815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97a4101fa0 msgr2=0x7f97a4197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:38.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:38.033+0000 7f97a9198700 1 --2- 172.21.15.71:0/3566635815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97a4101fa0 0x7f97a4197220 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9794006010 tx=0x7f9794005050 comp rx=0 tx=0).stop 2024-03-04T08:48:38.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:38.033+0000 7f97a9198700 1 -- 172.21.15.71:0/3566635815 shutdown_connections 2024-03-04T08:48:38.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:38.033+0000 7f97a9198700 1 --2- 172.21.15.71:0/3566635815 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f978c038c40 0x7f978c03b100 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:38.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:38.033+0000 7f97a9198700 1 --2- 172.21.15.71:0/3566635815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97a4101fa0 0x7f97a4197220 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:38.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:38.033+0000 7f97a9198700 1 -- 172.21.15.71:0/3566635815 >> 172.21.15.71:0/3566635815 conn(0x7f97a40fb750 msgr2=0x7f97a4100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:38.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:38.033+0000 7f97a9198700 1 -- 172.21.15.71:0/3566635815 shutdown_connections 2024-03-04T08:48:38.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:38.033+0000 7f97a9198700 1 -- 172.21.15.71:0/3566635815 wait complete. 2024-03-04T08:48:38.835 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi150 2024-03-04T08:48:38.836 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-03-04T08:48:38.836 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.conf 2024-03-04T08:48:38.855 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-03-04T08:48:38.855 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-04T08:48:38.912 INFO:tasks.cephadm:Adding host smithi150 to orchestrator... 2024-03-04T08:48:38.913 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph orch host add smithi150 2024-03-04T08:48:39.151 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:48:39.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:39 smithi071 ceph-mon[32789]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:48:39.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:39 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:39.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:39 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:48:39.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:39 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:48:39.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:39 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:48:39.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:39 smithi071 ceph-mon[32789]: Updating smithi071:/etc/ceph/ceph.conf 2024-03-04T08:48:39.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:39 smithi071 ceph-mon[32789]: Updating smithi071:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T08:48:39.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:39 smithi071 ceph-mon[32789]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-03-04T08:48:39.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:39 smithi071 ceph-mon[32789]: Updating smithi071:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T08:48:39.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:39 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:39.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:39 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:39.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:39 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:40.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.627+0000 7fbe8688c700 1 -- 172.21.15.71:0/1638938802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe80107aa0 msgr2=0x7fbe80107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:40.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.627+0000 7fbe8688c700 1 --2- 172.21.15.71:0/1638938802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe80107aa0 0x7fbe80107e80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009910 tx=0x7fbe74009c20 comp rx=0 tx=0).stop 2024-03-04T08:48:40.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.627+0000 7fbe8688c700 1 -- 172.21.15.71:0/1638938802 shutdown_connections 2024-03-04T08:48:40.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.627+0000 7fbe8688c700 1 --2- 172.21.15.71:0/1638938802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe80107aa0 0x7fbe80107e80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:40.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.627+0000 7fbe8688c700 1 -- 172.21.15.71:0/1638938802 >> 172.21.15.71:0/1638938802 conn(0x7fbe800fd5d0 msgr2=0x7fbe800ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:40.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.627+0000 7fbe8688c700 1 -- 172.21.15.71:0/1638938802 shutdown_connections 2024-03-04T08:48:40.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.627+0000 7fbe8688c700 1 -- 172.21.15.71:0/1638938802 wait complete. 2024-03-04T08:48:40.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.627+0000 7fbe8688c700 1 Processor -- start 2024-03-04T08:48:40.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.627+0000 7fbe8688c700 1 -- start start 2024-03-04T08:48:40.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.628+0000 7fbe8688c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe80107aa0 0x7fbe801975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:40.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.628+0000 7fbe8688c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8010c0e0 con 0x7fbe80107aa0 2024-03-04T08:48:40.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.628+0000 7fbe7ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe80107aa0 0x7fbe801975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:40.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.628+0000 7fbe7ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe80107aa0 0x7fbe801975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56260/0 (socket says 172.21.15.71:56260) 2024-03-04T08:48:40.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.628+0000 7fbe7ffff700 1 -- 172.21.15.71:0/4219112297 learned_addr learned my addr 172.21.15.71:0/4219112297 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:40.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.628+0000 7fbe7ffff700 1 -- 172.21.15.71:0/4219112297 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe80197b20 con 0x7fbe80107aa0 2024-03-04T08:48:40.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.628+0000 7fbe7ffff700 1 --2- 172.21.15.71:0/4219112297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe80107aa0 0x7fbe801975e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbe74005850 tx=0x7fbe74005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:40.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.628+0000 7fbe7d7fa700 1 -- 172.21.15.71:0/4219112297 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe74026070 con 0x7fbe80107aa0 2024-03-04T08:48:40.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.628+0000 7fbe8688c700 1 -- 172.21.15.71:0/4219112297 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe80197e10 con 0x7fbe80107aa0 2024-03-04T08:48:40.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.629+0000 7fbe7d7fa700 1 -- 172.21.15.71:0/4219112297 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe7400bbd0 con 0x7fbe80107aa0 2024-03-04T08:48:40.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.629+0000 7fbe7d7fa700 1 -- 172.21.15.71:0/4219112297 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe74003970 con 0x7fbe80107aa0 2024-03-04T08:48:40.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.629+0000 7fbe8688c700 1 -- 172.21.15.71:0/4219112297 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8019beb0 con 0x7fbe80107aa0 2024-03-04T08:48:40.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.629+0000 7fbe7d7fa700 1 -- 172.21.15.71:0/4219112297 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7fbe74029070 con 0x7fbe80107aa0 2024-03-04T08:48:40.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.629+0000 7fbe7d7fa700 1 --2- 172.21.15.71:0/4219112297 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fbe68038c40 0x7fbe6803b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:40.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.630+0000 7fbe7d7fa700 1 -- 172.21.15.71:0/4219112297 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbe7401e070 con 0x7fbe80107aa0 2024-03-04T08:48:40.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.630+0000 7fbe8688c700 1 -- 172.21.15.71:0/4219112297 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe801031e0 con 0x7fbe80107aa0 2024-03-04T08:48:40.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.630+0000 7fbe7f7fe700 1 --2- 172.21.15.71:0/4219112297 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fbe68038c40 0x7fbe6803b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:40.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.630+0000 7fbe7f7fe700 1 --2- 172.21.15.71:0/4219112297 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fbe68038c40 0x7fbe6803b100 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbe70006fd0 tx=0x7fbe70006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:40.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.635+0000 7fbe7d7fa700 1 -- 172.21.15.71:0/4219112297 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbe7404ce20 con 0x7fbe80107aa0 2024-03-04T08:48:40.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:40.832+0000 7fbe8688c700 1 -- 172.21.15.71:0/4219112297 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe800636f0 con 0x7fbe68038c40 2024-03-04T08:48:41.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:40 smithi071 ceph-mon[32789]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:48:42.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:41 smithi071 ceph-mon[32789]: Deploying cephadm binary to smithi150 2024-03-04T08:48:43.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:42 smithi071 ceph-mon[32789]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:48:45.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:45.022+0000 7fbe7d7fa700 1 -- 172.21.15.71:0/4219112297 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fbe800636f0 con 0x7fbe68038c40 2024-03-04T08:48:45.019 INFO:teuthology.orchestra.run.smithi071.stdout:Added host 'smithi150' with addr '172.21.15.150' 2024-03-04T08:48:45.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:45.025+0000 7fbe8688c700 1 -- 172.21.15.71:0/4219112297 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fbe68038c40 msgr2=0x7fbe6803b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:45.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:45.025+0000 7fbe8688c700 1 --2- 172.21.15.71:0/4219112297 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fbe68038c40 0x7fbe6803b100 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbe70006fd0 tx=0x7fbe70006e40 comp rx=0 tx=0).stop 2024-03-04T08:48:45.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:45.025+0000 7fbe8688c700 1 -- 172.21.15.71:0/4219112297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe80107aa0 msgr2=0x7fbe801975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:45.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:45.025+0000 7fbe8688c700 1 --2- 172.21.15.71:0/4219112297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe80107aa0 0x7fbe801975e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbe74005850 tx=0x7fbe74005030 comp rx=0 tx=0).stop 2024-03-04T08:48:45.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:45.025+0000 7fbe8688c700 1 -- 172.21.15.71:0/4219112297 shutdown_connections 2024-03-04T08:48:45.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:45.025+0000 7fbe8688c700 1 --2- 172.21.15.71:0/4219112297 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fbe68038c40 0x7fbe6803b100 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:45.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:45.025+0000 7fbe8688c700 1 --2- 172.21.15.71:0/4219112297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe80107aa0 0x7fbe801975e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:45.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:45.025+0000 7fbe8688c700 1 -- 172.21.15.71:0/4219112297 >> 172.21.15.71:0/4219112297 conn(0x7fbe800fd5d0 msgr2=0x7fbe80109610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:45.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:45.025+0000 7fbe8688c700 1 -- 172.21.15.71:0/4219112297 shutdown_connections 2024-03-04T08:48:45.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:45.025+0000 7fbe8688c700 1 -- 172.21.15.71:0/4219112297 wait complete. 2024-03-04T08:48:45.611 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph orch host ls --format=json 2024-03-04T08:48:45.846 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:48:45.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:45 smithi071 ceph-mon[32789]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:48:45.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:45 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:45.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:45 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:48:45.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:45 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:46.882 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:46 smithi071 ceph-mon[32789]: Added host smithi150 2024-03-04T08:48:46.882 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:46 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:47.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.382+0000 7f72c1f59700 1 -- 172.21.15.71:0/4274438460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72bc107aa0 msgr2=0x7f72bc107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:47.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.382+0000 7f72c1f59700 1 --2- 172.21.15.71:0/4274438460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72bc107aa0 0x7f72bc107e80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009910 tx=0x7f72ac009c20 comp rx=0 tx=0).stop 2024-03-04T08:48:47.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.382+0000 7f72c1f59700 1 -- 172.21.15.71:0/4274438460 shutdown_connections 2024-03-04T08:48:47.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.382+0000 7f72c1f59700 1 --2- 172.21.15.71:0/4274438460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72bc107aa0 0x7f72bc107e80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:47.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.382+0000 7f72c1f59700 1 -- 172.21.15.71:0/4274438460 >> 172.21.15.71:0/4274438460 conn(0x7f72bc0fd5b0 msgr2=0x7f72bc0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:47.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.382+0000 7f72c1f59700 1 -- 172.21.15.71:0/4274438460 shutdown_connections 2024-03-04T08:48:47.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.382+0000 7f72c1f59700 1 -- 172.21.15.71:0/4274438460 wait complete. 2024-03-04T08:48:47.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.382+0000 7f72c1f59700 1 Processor -- start 2024-03-04T08:48:47.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.383+0000 7f72c1f59700 1 -- start start 2024-03-04T08:48:47.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.383+0000 7f72c1f59700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72bc107aa0 0x7f72bc1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:47.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.383+0000 7f72c1f59700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc10c0e0 con 0x7f72bc107aa0 2024-03-04T08:48:47.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.383+0000 7f72bb7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72bc107aa0 0x7f72bc1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:47.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.383+0000 7f72bb7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72bc107aa0 0x7f72bc1975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37828/0 (socket says 172.21.15.71:37828) 2024-03-04T08:48:47.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.383+0000 7f72bb7fe700 1 -- 172.21.15.71:0/1607864728 learned_addr learned my addr 172.21.15.71:0/1607864728 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:47.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.383+0000 7f72bb7fe700 1 -- 172.21.15.71:0/1607864728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72bc197b20 con 0x7f72bc107aa0 2024-03-04T08:48:47.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.383+0000 7f72bb7fe700 1 --2- 172.21.15.71:0/1607864728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72bc107aa0 0x7f72bc1975e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f72ac005850 tx=0x7f72ac005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:47.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.384+0000 7f72b97fa700 1 -- 172.21.15.71:0/1607864728 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72ac026070 con 0x7f72bc107aa0 2024-03-04T08:48:47.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.384+0000 7f72b97fa700 1 -- 172.21.15.71:0/1607864728 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f72ac00bbd0 con 0x7f72bc107aa0 2024-03-04T08:48:47.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.384+0000 7f72c1f59700 1 -- 172.21.15.71:0/1607864728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72bc197e10 con 0x7f72bc107aa0 2024-03-04T08:48:47.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.384+0000 7f72b97fa700 1 -- 172.21.15.71:0/1607864728 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72ac003970 con 0x7f72bc107aa0 2024-03-04T08:48:47.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.384+0000 7f72c1f59700 1 -- 172.21.15.71:0/1607864728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72bc19beb0 con 0x7f72bc107aa0 2024-03-04T08:48:47.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.384+0000 7f72b97fa700 1 -- 172.21.15.71:0/1607864728 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f72ac029070 con 0x7f72bc107aa0 2024-03-04T08:48:47.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.385+0000 7f72c1f59700 1 -- 172.21.15.71:0/1607864728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72bc1031e0 con 0x7f72bc107aa0 2024-03-04T08:48:47.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.385+0000 7f72b97fa700 1 --2- 172.21.15.71:0/1607864728 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f72a8038c40 0x7f72a803b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:47.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.385+0000 7f72b97fa700 1 -- 172.21.15.71:0/1607864728 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f72ac01e070 con 0x7f72bc107aa0 2024-03-04T08:48:47.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.385+0000 7f72b2ffd700 1 --2- 172.21.15.71:0/1607864728 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f72a8038c40 0x7f72a803b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:47.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.385+0000 7f72b2ffd700 1 --2- 172.21.15.71:0/1607864728 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f72a8038c40 0x7f72a803b100 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f72a4006fd0 tx=0x7f72a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:47.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.390+0000 7f72b97fa700 1 -- 172.21.15.71:0/1607864728 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f72ac013070 con 0x7f72bc107aa0 2024-03-04T08:48:47.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.593+0000 7f72c1f59700 1 -- 172.21.15.71:0/1607864728 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f72bc198940 con 0x7f72a8038c40 2024-03-04T08:48:47.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.595+0000 7f72b97fa700 1 -- 172.21.15.71:0/1607864728 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f72bc198940 con 0x7f72a8038c40 2024-03-04T08:48:47.591 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:48:47.591 INFO:teuthology.orchestra.run.smithi071.stdout:[{"addr": "172.21.15.71", "hostname": "smithi071", "labels": [], "status": ""}, {"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}] 2024-03-04T08:48:47.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.597+0000 7f72c1f59700 1 -- 172.21.15.71:0/1607864728 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f72a8038c40 msgr2=0x7f72a803b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:47.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.597+0000 7f72c1f59700 1 --2- 172.21.15.71:0/1607864728 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f72a8038c40 0x7f72a803b100 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f72a4006fd0 tx=0x7f72a4006e40 comp rx=0 tx=0).stop 2024-03-04T08:48:47.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.597+0000 7f72c1f59700 1 -- 172.21.15.71:0/1607864728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72bc107aa0 msgr2=0x7f72bc1975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:47.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.597+0000 7f72c1f59700 1 --2- 172.21.15.71:0/1607864728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72bc107aa0 0x7f72bc1975e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f72ac005850 tx=0x7f72ac005030 comp rx=0 tx=0).stop 2024-03-04T08:48:47.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.597+0000 7f72c1f59700 1 -- 172.21.15.71:0/1607864728 shutdown_connections 2024-03-04T08:48:47.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.597+0000 7f72c1f59700 1 --2- 172.21.15.71:0/1607864728 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f72a8038c40 0x7f72a803b100 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:47.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.597+0000 7f72c1f59700 1 --2- 172.21.15.71:0/1607864728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72bc107aa0 0x7f72bc1975e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:47.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.597+0000 7f72c1f59700 1 -- 172.21.15.71:0/1607864728 >> 172.21.15.71:0/1607864728 conn(0x7f72bc0fd5b0 msgr2=0x7f72bc109620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:47.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.597+0000 7f72c1f59700 1 -- 172.21.15.71:0/1607864728 shutdown_connections 2024-03-04T08:48:47.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:47.597+0000 7f72c1f59700 1 -- 172.21.15.71:0/1607864728 wait complete. 2024-03-04T08:48:47.914 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:47 smithi071 ceph-mon[32789]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:48:47.914 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:47 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:48.240 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi158 2024-03-04T08:48:48.240 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2024-03-04T08:48:48.241 DEBUG:teuthology.orchestra.run.smithi158:> dd of=/etc/ceph/ceph.conf 2024-03-04T08:48:48.259 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2024-03-04T08:48:48.259 DEBUG:teuthology.orchestra.run.smithi158:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-04T08:48:48.317 INFO:tasks.cephadm:Adding host smithi158 to orchestrator... 2024-03-04T08:48:48.317 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph orch host add smithi158 2024-03-04T08:48:48.550 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:48:48.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:48 smithi071 ceph-mon[32789]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T08:48:49.874 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:49 smithi071 ceph-mon[32789]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:48:50.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.130+0000 7fc5c375d700 1 -- 172.21.15.71:0/277454533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5bc107aa0 msgr2=0x7fc5bc107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:50.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.130+0000 7fc5c375d700 1 --2- 172.21.15.71:0/277454533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5bc107aa0 0x7fc5bc107e80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8009910 tx=0x7fc5b8009c20 comp rx=0 tx=0).stop 2024-03-04T08:48:50.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.130+0000 7fc5c375d700 1 -- 172.21.15.71:0/277454533 shutdown_connections 2024-03-04T08:48:50.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.130+0000 7fc5c375d700 1 --2- 172.21.15.71:0/277454533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5bc107aa0 0x7fc5bc107e80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:50.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.130+0000 7fc5c375d700 1 -- 172.21.15.71:0/277454533 >> 172.21.15.71:0/277454533 conn(0x7fc5bc0fd5d0 msgr2=0x7fc5bc0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:50.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.131+0000 7fc5c375d700 1 -- 172.21.15.71:0/277454533 shutdown_connections 2024-03-04T08:48:50.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.131+0000 7fc5c375d700 1 -- 172.21.15.71:0/277454533 wait complete. 2024-03-04T08:48:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.131+0000 7fc5c375d700 1 Processor -- start 2024-03-04T08:48:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.131+0000 7fc5c375d700 1 -- start start 2024-03-04T08:48:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.132+0000 7fc5c375d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5bc107aa0 0x7fc5bc1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.132+0000 7fc5c375d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc10c0e0 con 0x7fc5bc107aa0 2024-03-04T08:48:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.132+0000 7fc5c14f9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5bc107aa0 0x7fc5bc1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.132+0000 7fc5c14f9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5bc107aa0 0x7fc5bc1975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37848/0 (socket says 172.21.15.71:37848) 2024-03-04T08:48:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.132+0000 7fc5c14f9700 1 -- 172.21.15.71:0/2422188583 learned_addr learned my addr 172.21.15.71:0/2422188583 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.132+0000 7fc5c14f9700 1 -- 172.21.15.71:0/2422188583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc197b20 con 0x7fc5bc107aa0 2024-03-04T08:48:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.132+0000 7fc5c14f9700 1 --2- 172.21.15.71:0/2422188583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5bc107aa0 0x7fc5bc1975e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8005850 tx=0x7fc5b8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:50.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.132+0000 7fc5b27fc700 1 -- 172.21.15.71:0/2422188583 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5b8026070 con 0x7fc5bc107aa0 2024-03-04T08:48:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.132+0000 7fc5b27fc700 1 -- 172.21.15.71:0/2422188583 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5b800bbd0 con 0x7fc5bc107aa0 2024-03-04T08:48:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.132+0000 7fc5c375d700 1 -- 172.21.15.71:0/2422188583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5bc197e10 con 0x7fc5bc107aa0 2024-03-04T08:48:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.132+0000 7fc5b27fc700 1 -- 172.21.15.71:0/2422188583 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5b8003970 con 0x7fc5bc107aa0 2024-03-04T08:48:50.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.132+0000 7fc5c375d700 1 -- 172.21.15.71:0/2422188583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5bc19beb0 con 0x7fc5bc107aa0 2024-03-04T08:48:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.133+0000 7fc5b27fc700 1 -- 172.21.15.71:0/2422188583 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7fc5b8029070 con 0x7fc5bc107aa0 2024-03-04T08:48:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.133+0000 7fc5c375d700 1 -- 172.21.15.71:0/2422188583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5bc1031e0 con 0x7fc5bc107aa0 2024-03-04T08:48:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.133+0000 7fc5b27fc700 1 --2- 172.21.15.71:0/2422188583 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc5a8038ca0 0x7fc5a803b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.133+0000 7fc5b27fc700 1 -- 172.21.15.71:0/2422188583 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc5b801e070 con 0x7fc5bc107aa0 2024-03-04T08:48:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.134+0000 7fc5c0cf8700 1 --2- 172.21.15.71:0/2422188583 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc5a8038ca0 0x7fc5a803b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:50.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.134+0000 7fc5c0cf8700 1 --2- 172.21.15.71:0/2422188583 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc5a8038ca0 0x7fc5a803b160 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac006fd0 tx=0x7fc5ac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:50.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.138+0000 7fc5b27fc700 1 -- 172.21.15.71:0/2422188583 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc5b8013070 con 0x7fc5bc107aa0 2024-03-04T08:48:50.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:50.334+0000 7fc5c375d700 1 -- 172.21.15.71:0/2422188583 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi158", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5bc198940 con 0x7fc5a8038ca0 2024-03-04T08:48:51.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:51 smithi071 ceph-mon[32789]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi158", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:48:51.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:51 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:51.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:51 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:51.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:51 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:51.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:51 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:51.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:51 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:48:51.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:51 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:48:51.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:51 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:48:51.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:51 smithi071 ceph-mon[32789]: Updating smithi150:/etc/ceph/ceph.conf 2024-03-04T08:48:51.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:51 smithi071 ceph-mon[32789]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:48:51.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:51 smithi071 ceph-mon[32789]: Updating smithi150:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T08:48:51.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:51 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:51.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:51 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:51.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:51 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:52.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:52 smithi071 ceph-mon[32789]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-03-04T08:48:52.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:52 smithi071 ceph-mon[32789]: Deploying cephadm binary to smithi158 2024-03-04T08:48:52.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:52 smithi071 ceph-mon[32789]: Updating smithi150:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T08:48:53.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:53 smithi071 ceph-mon[32789]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:48:54.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:54.414+0000 7fc5b27fc700 1 -- 172.21.15.71:0/2422188583 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fc5bc198940 con 0x7fc5a8038ca0 2024-03-04T08:48:54.411 INFO:teuthology.orchestra.run.smithi071.stdout:Added host 'smithi158' with addr '172.21.15.158' 2024-03-04T08:48:54.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:54.416+0000 7fc5c375d700 1 -- 172.21.15.71:0/2422188583 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc5a8038ca0 msgr2=0x7fc5a803b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:54.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:54.416+0000 7fc5c375d700 1 --2- 172.21.15.71:0/2422188583 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc5a8038ca0 0x7fc5a803b160 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac006fd0 tx=0x7fc5ac006e40 comp rx=0 tx=0).stop 2024-03-04T08:48:54.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:54.416+0000 7fc5c375d700 1 -- 172.21.15.71:0/2422188583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5bc107aa0 msgr2=0x7fc5bc1975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:54.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:54.416+0000 7fc5c375d700 1 --2- 172.21.15.71:0/2422188583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5bc107aa0 0x7fc5bc1975e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8005850 tx=0x7fc5b8005030 comp rx=0 tx=0).stop 2024-03-04T08:48:54.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:54.416+0000 7fc5c375d700 1 -- 172.21.15.71:0/2422188583 shutdown_connections 2024-03-04T08:48:54.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:54.416+0000 7fc5c375d700 1 --2- 172.21.15.71:0/2422188583 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc5a8038ca0 0x7fc5a803b160 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:54.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:54.416+0000 7fc5c375d700 1 --2- 172.21.15.71:0/2422188583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5bc107aa0 0x7fc5bc1975e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:54.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:54.416+0000 7fc5c375d700 1 -- 172.21.15.71:0/2422188583 >> 172.21.15.71:0/2422188583 conn(0x7fc5bc0fd5d0 msgr2=0x7fc5bc109610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:54.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:54.417+0000 7fc5c375d700 1 -- 172.21.15.71:0/2422188583 shutdown_connections 2024-03-04T08:48:54.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:54.417+0000 7fc5c375d700 1 -- 172.21.15.71:0/2422188583 wait complete. 2024-03-04T08:48:55.081 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph orch host ls --format=json 2024-03-04T08:48:55.316 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:48:55.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:55 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:55.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:55 smithi071 ceph-mon[32789]: Added host smithi158 2024-03-04T08:48:55.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:55 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:48:55.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:55 smithi071 ceph-mon[32789]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:48:55.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:55 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:56.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:56 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:56.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.919+0000 7f3fc6a09700 1 -- 172.21.15.71:0/4180432075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc0107aa0 msgr2=0x7f3fc0107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:56.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.919+0000 7f3fc6a09700 1 --2- 172.21.15.71:0/4180432075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc0107aa0 0x7f3fc0107e80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009910 tx=0x7f3fb0009c20 comp rx=0 tx=0).stop 2024-03-04T08:48:56.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.919+0000 7f3fc6a09700 1 -- 172.21.15.71:0/4180432075 shutdown_connections 2024-03-04T08:48:56.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.919+0000 7f3fc6a09700 1 --2- 172.21.15.71:0/4180432075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc0107aa0 0x7f3fc0107e80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:56.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.919+0000 7f3fc6a09700 1 -- 172.21.15.71:0/4180432075 >> 172.21.15.71:0/4180432075 conn(0x7f3fc00fd5d0 msgr2=0x7f3fc00ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:56.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.920+0000 7f3fc6a09700 1 -- 172.21.15.71:0/4180432075 shutdown_connections 2024-03-04T08:48:56.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.920+0000 7f3fc6a09700 1 -- 172.21.15.71:0/4180432075 wait complete. 2024-03-04T08:48:56.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.920+0000 7f3fc6a09700 1 Processor -- start 2024-03-04T08:48:56.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.920+0000 7f3fc6a09700 1 -- start start 2024-03-04T08:48:56.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.920+0000 7f3fc6a09700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc0107aa0 0x7f3fc0197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:56.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.920+0000 7f3fc6a09700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc010c0e0 con 0x7f3fc0107aa0 2024-03-04T08:48:56.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.921+0000 7f3fbffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc0107aa0 0x7f3fc0197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:56.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.921+0000 7f3fbffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc0107aa0 0x7f3fc0197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38108/0 (socket says 172.21.15.71:38108) 2024-03-04T08:48:56.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.921+0000 7f3fbffff700 1 -- 172.21.15.71:0/3983885305 learned_addr learned my addr 172.21.15.71:0/3983885305 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:56.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.921+0000 7f3fbffff700 1 -- 172.21.15.71:0/3983885305 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc0197b90 con 0x7f3fc0107aa0 2024-03-04T08:48:56.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.921+0000 7f3fbffff700 1 --2- 172.21.15.71:0/3983885305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc0107aa0 0x7f3fc0197650 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0006010 tx=0x7f3fb0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:56.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.921+0000 7f3fbd7fa700 1 -- 172.21.15.71:0/3983885305 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fb0026070 con 0x7f3fc0107aa0 2024-03-04T08:48:56.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.921+0000 7f3fc6a09700 1 -- 172.21.15.71:0/3983885305 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc0197e80 con 0x7f3fc0107aa0 2024-03-04T08:48:56.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.921+0000 7f3fbd7fa700 1 -- 172.21.15.71:0/3983885305 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fb000bbd0 con 0x7f3fc0107aa0 2024-03-04T08:48:56.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.921+0000 7f3fbd7fa700 1 -- 172.21.15.71:0/3983885305 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fb0003910 con 0x7f3fc0107aa0 2024-03-04T08:48:56.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.921+0000 7f3fc6a09700 1 -- 172.21.15.71:0/3983885305 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc019bf20 con 0x7f3fc0107aa0 2024-03-04T08:48:56.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.922+0000 7f3fbd7fa700 1 -- 172.21.15.71:0/3983885305 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f3fb0029070 con 0x7f3fc0107aa0 2024-03-04T08:48:56.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.922+0000 7f3fbd7fa700 1 --2- 172.21.15.71:0/3983885305 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3fa8038c40 0x7f3fa803b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:56.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.922+0000 7f3fbd7fa700 1 -- 172.21.15.71:0/3983885305 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3fb001e070 con 0x7f3fc0107aa0 2024-03-04T08:48:56.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.922+0000 7f3fbf7fe700 1 --2- 172.21.15.71:0/3983885305 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3fa8038c40 0x7f3fa803b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:56.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.922+0000 7f3fc6a09700 1 -- 172.21.15.71:0/3983885305 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc01031e0 con 0x7f3fc0107aa0 2024-03-04T08:48:56.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.923+0000 7f3fbf7fe700 1 --2- 172.21.15.71:0/3983885305 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3fa8038c40 0x7f3fa803b100 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4006fd0 tx=0x7f3fb4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:56.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:56.927+0000 7f3fbd7fa700 1 -- 172.21.15.71:0/3983885305 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3fb0030070 con 0x7f3fc0107aa0 2024-03-04T08:48:57.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:57.129+0000 7f3fc6a09700 1 -- 172.21.15.71:0/3983885305 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3fc00636f0 con 0x7f3fa8038c40 2024-03-04T08:48:57.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:57.131+0000 7f3fbd7fa700 1 -- 172.21.15.71:0/3983885305 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7f3fc00636f0 con 0x7f3fa8038c40 2024-03-04T08:48:57.127 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:48:57.127 INFO:teuthology.orchestra.run.smithi071.stdout:[{"addr": "172.21.15.71", "hostname": "smithi071", "labels": [], "status": ""}, {"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}, {"addr": "172.21.15.158", "hostname": "smithi158", "labels": [], "status": ""}] 2024-03-04T08:48:57.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:57.133+0000 7f3fc6a09700 1 -- 172.21.15.71:0/3983885305 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3fa8038c40 msgr2=0x7f3fa803b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:57.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:57.133+0000 7f3fc6a09700 1 --2- 172.21.15.71:0/3983885305 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3fa8038c40 0x7f3fa803b100 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4006fd0 tx=0x7f3fb4006e40 comp rx=0 tx=0).stop 2024-03-04T08:48:57.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:57.133+0000 7f3fc6a09700 1 -- 172.21.15.71:0/3983885305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc0107aa0 msgr2=0x7f3fc0197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:57.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:57.133+0000 7f3fc6a09700 1 --2- 172.21.15.71:0/3983885305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc0107aa0 0x7f3fc0197650 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0006010 tx=0x7f3fb0005030 comp rx=0 tx=0).stop 2024-03-04T08:48:57.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:57.133+0000 7f3fc6a09700 1 -- 172.21.15.71:0/3983885305 shutdown_connections 2024-03-04T08:48:57.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:57.133+0000 7f3fc6a09700 1 --2- 172.21.15.71:0/3983885305 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3fa8038c40 0x7f3fa803b100 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:57.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:57.133+0000 7f3fc6a09700 1 --2- 172.21.15.71:0/3983885305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc0107aa0 0x7f3fc0197650 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:57.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:57.133+0000 7f3fc6a09700 1 -- 172.21.15.71:0/3983885305 >> 172.21.15.71:0/3983885305 conn(0x7f3fc00fd5d0 msgr2=0x7f3fc0109650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:57.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:57.133+0000 7f3fc6a09700 1 -- 172.21.15.71:0/3983885305 shutdown_connections 2024-03-04T08:48:57.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:57.133+0000 7f3fc6a09700 1 -- 172.21.15.71:0/3983885305 wait complete. 2024-03-04T08:48:57.776 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:57 smithi071 ceph-mon[32789]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:48:57.776 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:57 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:48:57.807 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-04T08:48:57.808 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph osd crush tunables default 2024-03-04T08:48:58.044 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:48:58.774 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:58 smithi071 ceph-mon[32789]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T08:48:59.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.536+0000 7f855e561700 1 -- 172.21.15.71:0/3699122478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8558107aa0 msgr2=0x7f8558107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:48:59.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.536+0000 7f855e561700 1 --2- 172.21.15.71:0/3699122478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8558107aa0 0x7f8558107e80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8540009910 tx=0x7f8540009c20 comp rx=0 tx=0).stop 2024-03-04T08:48:59.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.536+0000 7f855e561700 1 -- 172.21.15.71:0/3699122478 shutdown_connections 2024-03-04T08:48:59.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.536+0000 7f855e561700 1 --2- 172.21.15.71:0/3699122478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8558107aa0 0x7f8558107e80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:59.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.536+0000 7f855e561700 1 -- 172.21.15.71:0/3699122478 >> 172.21.15.71:0/3699122478 conn(0x7f85580fd5d0 msgr2=0x7f85580ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:48:59.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.536+0000 7f855e561700 1 -- 172.21.15.71:0/3699122478 shutdown_connections 2024-03-04T08:48:59.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.536+0000 7f855e561700 1 -- 172.21.15.71:0/3699122478 wait complete. 2024-03-04T08:48:59.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.537+0000 7f855e561700 1 Processor -- start 2024-03-04T08:48:59.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.537+0000 7f855e561700 1 -- start start 2024-03-04T08:48:59.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.537+0000 7f855e561700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8558107aa0 0x7f85580743c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:59.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.537+0000 7f855e561700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855810c0e0 con 0x7f8558107aa0 2024-03-04T08:48:59.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.537+0000 7f8557fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8558107aa0 0x7f85580743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:59.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.537+0000 7f8557fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8558107aa0 0x7f85580743c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38122/0 (socket says 172.21.15.71:38122) 2024-03-04T08:48:59.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.537+0000 7f8557fff700 1 -- 172.21.15.71:0/3420679694 learned_addr learned my addr 172.21.15.71:0/3420679694 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:48:59.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.537+0000 7f8557fff700 1 -- 172.21.15.71:0/3420679694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85580762b0 con 0x7f8558107aa0 2024-03-04T08:48:59.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.538+0000 7f8557fff700 1 --2- 172.21.15.71:0/3420679694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8558107aa0 0x7f85580743c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8540006010 tx=0x7f8540005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:59.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.538+0000 7f85557fa700 1 -- 172.21.15.71:0/3420679694 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8540026070 con 0x7f8558107aa0 2024-03-04T08:48:59.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.538+0000 7f855e561700 1 -- 172.21.15.71:0/3420679694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8558074960 con 0x7f8558107aa0 2024-03-04T08:48:59.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.538+0000 7f85557fa700 1 -- 172.21.15.71:0/3420679694 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f854000bc30 con 0x7f8558107aa0 2024-03-04T08:48:59.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.538+0000 7f85557fa700 1 -- 172.21.15.71:0/3420679694 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f85400039d0 con 0x7f8558107aa0 2024-03-04T08:48:59.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.538+0000 7f855e561700 1 -- 172.21.15.71:0/3420679694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8558074e20 con 0x7f8558107aa0 2024-03-04T08:48:59.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.539+0000 7f85557fa700 1 -- 172.21.15.71:0/3420679694 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f8540029070 con 0x7f8558107aa0 2024-03-04T08:48:59.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.539+0000 7f855e561700 1 -- 172.21.15.71:0/3420679694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85581031e0 con 0x7f8558107aa0 2024-03-04T08:48:59.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.539+0000 7f85557fa700 1 --2- 172.21.15.71:0/3420679694 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f8544038c40 0x7f854403b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:59.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.539+0000 7f85557fa700 1 -- 172.21.15.71:0/3420679694 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f854001e070 con 0x7f8558107aa0 2024-03-04T08:48:59.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.539+0000 7f85577fe700 1 --2- 172.21.15.71:0/3420679694 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f8544038c40 0x7f854403b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:48:59.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.539+0000 7f85577fe700 1 --2- 172.21.15.71:0/3420679694 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f8544038c40 0x7f854403b100 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8548006fd0 tx=0x7f8548006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:48:59.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.544+0000 7f85557fa700 1 -- 172.21.15.71:0/3420679694 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8540013070 con 0x7f8558107aa0 2024-03-04T08:48:59.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:48:59.744+0000 7f855e561700 1 -- 172.21.15.71:0/3420679694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f85580636f0 con 0x7f8558107aa0 2024-03-04T08:48:59.740 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:48:59 smithi071 ceph-mon[32789]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:00.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:00.439+0000 7f85557fa700 1 -- 172.21.15.71:0/3420679694 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f8540030090 con 0x7f8558107aa0 2024-03-04T08:49:00.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:00.441+0000 7f855e561700 1 -- 172.21.15.71:0/3420679694 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f8544038c40 msgr2=0x7f854403b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:00.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:00.441+0000 7f855e561700 1 --2- 172.21.15.71:0/3420679694 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f8544038c40 0x7f854403b100 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8548006fd0 tx=0x7f8548006e40 comp rx=0 tx=0).stop 2024-03-04T08:49:00.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:00.441+0000 7f855e561700 1 -- 172.21.15.71:0/3420679694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8558107aa0 msgr2=0x7f85580743c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:00.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:00.441+0000 7f855e561700 1 --2- 172.21.15.71:0/3420679694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8558107aa0 0x7f85580743c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8540006010 tx=0x7f8540005030 comp rx=0 tx=0).stop 2024-03-04T08:49:00.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:00.441+0000 7f855e561700 1 -- 172.21.15.71:0/3420679694 shutdown_connections 2024-03-04T08:49:00.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:00.441+0000 7f855e561700 1 --2- 172.21.15.71:0/3420679694 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f8544038c40 0x7f854403b100 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:00.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:00.441+0000 7f855e561700 1 --2- 172.21.15.71:0/3420679694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8558107aa0 0x7f85580743c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:00.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:00.442+0000 7f855e561700 1 -- 172.21.15.71:0/3420679694 >> 172.21.15.71:0/3420679694 conn(0x7f85580fd5d0 msgr2=0x7f8558109610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:49:00.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:00.442+0000 7f855e561700 1 -- 172.21.15.71:0/3420679694 shutdown_connections 2024-03-04T08:49:00.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:00.442+0000 7f855e561700 1 -- 172.21.15.71:0/3420679694 wait complete. 2024-03-04T08:49:00.439 INFO:teuthology.orchestra.run.smithi071.stderr:adjusted tunables profile to default 2024-03-04T08:49:00.798 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:00 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3420679694' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-04T08:49:00.799 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:00 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:00.799 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:00 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:00.799 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:00 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:00.799 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:00 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:00.799 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:00 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi158", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:49:00.799 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:00 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:00.799 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:00 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:49:01.108 INFO:tasks.cephadm:Adding mon.a on smithi071 2024-03-04T08:49:01.109 INFO:tasks.cephadm:Adding mon.b on smithi150 2024-03-04T08:49:01.109 INFO:tasks.cephadm:Adding mon.c on smithi158 2024-03-04T08:49:01.109 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph orch apply mon '3;smithi071:172.21.15.71=a;smithi150:172.21.15.150=b;smithi158:172.21.15.158=c' 2024-03-04T08:49:01.333 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T08:49:01.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:01 smithi071 ceph-mon[32789]: Updating smithi158:/etc/ceph/ceph.conf 2024-03-04T08:49:01.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:01 smithi071 ceph-mon[32789]: Updating smithi158:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T08:49:01.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:01 smithi071 ceph-mon[32789]: Updating smithi158:/etc/ceph/ceph.client.admin.keyring 2024-03-04T08:49:01.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:01 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3420679694' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-04T08:49:01.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:01 smithi071 ceph-mon[32789]: osdmap e4: 0 total, 0 up, 0 in 2024-03-04T08:49:01.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:01 smithi071 ceph-mon[32789]: Updating smithi158:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T08:49:01.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:01 smithi071 ceph-mon[32789]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:01.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:01 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:01.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:01 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:01.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:01 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:02.710 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.713+0000 7f11bbfcf700 1 -- 172.21.15.158:0/1401503209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b4102c60 msgr2=0x7f11b4103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:02.710 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.713+0000 7f11bbfcf700 1 --2- 172.21.15.158:0/1401503209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b4102c60 0x7f11b4103040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f11b0009910 tx=0x7f11b0009c20 comp rx=0 tx=0).stop 2024-03-04T08:49:02.710 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.713+0000 7f11bbfcf700 1 -- 172.21.15.158:0/1401503209 shutdown_connections 2024-03-04T08:49:02.710 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.713+0000 7f11bbfcf700 1 --2- 172.21.15.158:0/1401503209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b4102c60 0x7f11b4103040 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:02.710 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.713+0000 7f11bbfcf700 1 -- 172.21.15.158:0/1401503209 >> 172.21.15.158:0/1401503209 conn(0x7f11b40fc4b0 msgr2=0x7f11b40fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:49:02.710 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.713+0000 7f11bbfcf700 1 -- 172.21.15.158:0/1401503209 shutdown_connections 2024-03-04T08:49:02.711 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.713+0000 7f11bbfcf700 1 -- 172.21.15.158:0/1401503209 wait complete. 2024-03-04T08:49:02.711 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.714+0000 7f11bbfcf700 1 Processor -- start 2024-03-04T08:49:02.711 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.714+0000 7f11bbfcf700 1 -- start start 2024-03-04T08:49:02.712 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.714+0000 7f11bbfcf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b4102c60 0x7f11b4197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:49:02.712 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.714+0000 7f11bbfcf700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b410c150 con 0x7f11b4102c60 2024-03-04T08:49:02.712 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.715+0000 7f11b9d6b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b4102c60 0x7f11b4197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:49:02.712 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.715+0000 7f11b9d6b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b4102c60 0x7f11b4197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.158:48822/0 (socket says 172.21.15.158:48822) 2024-03-04T08:49:02.713 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.715+0000 7f11b9d6b700 1 -- 172.21.15.158:0/2085096692 learned_addr learned my addr 172.21.15.158:0/2085096692 (peer_addr_for_me v2:172.21.15.158:0/0) 2024-03-04T08:49:02.713 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.715+0000 7f11b9d6b700 1 -- 172.21.15.158:0/2085096692 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b4197b50 con 0x7f11b4102c60 2024-03-04T08:49:02.713 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.715+0000 7f11b9d6b700 1 --2- 172.21.15.158:0/2085096692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b4102c60 0x7f11b4197610 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f11b0006010 tx=0x7f11b0005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:49:02.713 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.715+0000 7f11aaffd700 1 -- 172.21.15.158:0/2085096692 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11b0026070 con 0x7f11b4102c60 2024-03-04T08:49:02.713 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.715+0000 7f11bbfcf700 1 -- 172.21.15.158:0/2085096692 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b4197de0 con 0x7f11b4102c60 2024-03-04T08:49:02.713 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.715+0000 7f11aaffd700 1 -- 172.21.15.158:0/2085096692 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f11b000bc30 con 0x7f11b4102c60 2024-03-04T08:49:02.713 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.715+0000 7f11aaffd700 1 -- 172.21.15.158:0/2085096692 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11b0003ab0 con 0x7f11b4102c60 2024-03-04T08:49:02.713 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.716+0000 7f11bbfcf700 1 -- 172.21.15.158:0/2085096692 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b4198240 con 0x7f11b4102c60 2024-03-04T08:49:02.714 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.716+0000 7f11bbfcf700 1 -- 172.21.15.158:0/2085096692 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b406a820 con 0x7f11b4102c60 2024-03-04T08:49:02.714 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.716+0000 7f11aaffd700 1 -- 172.21.15.158:0/2085096692 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f11b0029070 con 0x7f11b4102c60 2024-03-04T08:49:02.714 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.717+0000 7f11aaffd700 1 --2- 172.21.15.158:0/2085096692 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f11a0038c40 0x7f11a003b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:49:02.714 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.717+0000 7f11aaffd700 1 -- 172.21.15.158:0/2085096692 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f11b001e070 con 0x7f11b4102c60 2024-03-04T08:49:02.715 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.718+0000 7f11b956a700 1 --2- 172.21.15.158:0/2085096692 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f11a0038c40 0x7f11a003b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:49:02.715 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.719+0000 7f11b956a700 1 --2- 172.21.15.158:0/2085096692 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f11a0038c40 0x7f11a003b100 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f11a4006fd0 tx=0x7f11a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:49:02.718 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.722+0000 7f11aaffd700 1 -- 172.21.15.158:0/2085096692 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f11b0013070 con 0x7f11b4102c60 2024-03-04T08:49:02.925 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.929+0000 7f11bbfcf700 1 -- 172.21.15.158:0/2085096692 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi071:172.21.15.71=a;smithi150:172.21.15.150=b;smithi158:172.21.15.158=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f11b4104120 con 0x7f11a0038c40 2024-03-04T08:49:02.932 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.936+0000 7f11aaffd700 1 -- 172.21.15.158:0/2085096692 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f11b4104120 con 0x7f11a0038c40 2024-03-04T08:49:02.932 INFO:teuthology.orchestra.run.smithi158.stdout:Scheduled mon update... 2024-03-04T08:49:02.934 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.938+0000 7f11bbfcf700 1 -- 172.21.15.158:0/2085096692 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f11a0038c40 msgr2=0x7f11a003b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:02.935 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.938+0000 7f11bbfcf700 1 --2- 172.21.15.158:0/2085096692 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f11a0038c40 0x7f11a003b100 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f11a4006fd0 tx=0x7f11a4006e40 comp rx=0 tx=0).stop 2024-03-04T08:49:02.935 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.938+0000 7f11bbfcf700 1 -- 172.21.15.158:0/2085096692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b4102c60 msgr2=0x7f11b4197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:02.935 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.938+0000 7f11bbfcf700 1 --2- 172.21.15.158:0/2085096692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b4102c60 0x7f11b4197610 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f11b0006010 tx=0x7f11b0005050 comp rx=0 tx=0).stop 2024-03-04T08:49:02.935 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.938+0000 7f11bbfcf700 1 -- 172.21.15.158:0/2085096692 shutdown_connections 2024-03-04T08:49:02.935 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.938+0000 7f11bbfcf700 1 --2- 172.21.15.158:0/2085096692 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f11a0038c40 0x7f11a003b100 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:02.935 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.938+0000 7f11bbfcf700 1 --2- 172.21.15.158:0/2085096692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b4102c60 0x7f11b4197610 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:02.935 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.938+0000 7f11bbfcf700 1 -- 172.21.15.158:0/2085096692 >> 172.21.15.158:0/2085096692 conn(0x7f11b40fc4b0 msgr2=0x7f11b4103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:49:02.935 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.938+0000 7f11bbfcf700 1 -- 172.21.15.158:0/2085096692 shutdown_connections 2024-03-04T08:49:02.935 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:02.938+0000 7f11bbfcf700 1 -- 172.21.15.158:0/2085096692 wait complete. 2024-03-04T08:49:03.492 DEBUG:teuthology.orchestra.run.smithi150:mon.b> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mon.b.service 2024-03-04T08:49:03.495 DEBUG:teuthology.orchestra.run.smithi158:mon.c> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mon.c.service 2024-03-04T08:49:03.497 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-04T08:49:03.497 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T08:49:03.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:03 smithi071 ceph-mon[32789]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:03.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:03 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:03.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:03 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:49:03.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:03 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:03.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:03 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:49:03.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:03 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:03.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:03 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T08:49:03.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:03 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:04.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:04 smithi071 ceph-mon[32789]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi071:172.21.15.71=a;smithi150:172.21.15.150=b;smithi158:172.21.15.158=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:49:04.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:04 smithi071 ceph-mon[32789]: Saving service mon spec with placement smithi071:172.21.15.71=a;smithi150:172.21.15.150=b;smithi158:172.21.15.158=c;count:3 2024-03-04T08:49:04.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:04 smithi071 ceph-mon[32789]: Deploying daemon mon.c on smithi158 2024-03-04T08:49:05.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:05 smithi071 ceph-mon[32789]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:07.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:07 smithi071 ceph-mon[32789]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:09.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:09 smithi071 ceph-mon[32789]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:11.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:11 smithi071 ceph-mon[32789]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:13.442 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T08:49:13.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:13 smithi071 ceph-mon[32789]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:15.413 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.402+0000 7f7f5c704700 1 -- 172.21.15.158:0/1517932149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f54101620 msgr2=0x7f7f54101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:15.414 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.402+0000 7f7f5c704700 1 --2- 172.21.15.158:0/1517932149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f54101620 0x7f7f54101a00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7f50009910 tx=0x7f7f50009c20 comp rx=0 tx=0).stop 2024-03-04T08:49:15.414 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.402+0000 7f7f5c704700 1 -- 172.21.15.158:0/1517932149 shutdown_connections 2024-03-04T08:49:15.414 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.402+0000 7f7f5c704700 1 --2- 172.21.15.158:0/1517932149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f54101620 0x7f7f54101a00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:15.414 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.402+0000 7f7f5c704700 1 -- 172.21.15.158:0/1517932149 >> 172.21.15.158:0/1517932149 conn(0x7f7f54077080 msgr2=0x7f7f54077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:49:15.414 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.402+0000 7f7f5c704700 1 -- 172.21.15.158:0/1517932149 shutdown_connections 2024-03-04T08:49:15.414 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.402+0000 7f7f5c704700 1 -- 172.21.15.158:0/1517932149 wait complete. 2024-03-04T08:49:15.414 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.403+0000 7f7f5c704700 1 Processor -- start 2024-03-04T08:49:15.414 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.403+0000 7f7f5c704700 1 -- start start 2024-03-04T08:49:15.414 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.403+0000 7f7f5c704700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f54101620 0x7f7f540fd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:49:15.415 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.403+0000 7f7f5c704700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5410b0f0 con 0x7f7f54101620 2024-03-04T08:49:15.415 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.403+0000 7f7f5a4a0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f54101620 0x7f7f540fd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:49:15.415 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.404+0000 7f7f5a4a0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f54101620 0x7f7f540fd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.158:44122/0 (socket says 172.21.15.158:44122) 2024-03-04T08:49:15.415 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.404+0000 7f7f5a4a0700 1 -- 172.21.15.158:0/389262239 learned_addr learned my addr 172.21.15.158:0/389262239 (peer_addr_for_me v2:172.21.15.158:0/0) 2024-03-04T08:49:15.415 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.404+0000 7f7f5a4a0700 1 -- 172.21.15.158:0/389262239 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f540ff200 con 0x7f7f54101620 2024-03-04T08:49:15.415 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.404+0000 7f7f5a4a0700 1 --2- 172.21.15.158:0/389262239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f54101620 0x7f7f540fd360 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7f50005850 tx=0x7f7f50005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:49:15.415 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.405+0000 7f7f4b7fe700 1 -- 172.21.15.158:0/389262239 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7f50026070 con 0x7f7f54101620 2024-03-04T08:49:15.415 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.405+0000 7f7f5c704700 1 -- 172.21.15.158:0/389262239 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f540fd8a0 con 0x7f7f54101620 2024-03-04T08:49:15.415 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.405+0000 7f7f5c704700 1 -- 172.21.15.158:0/389262239 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f540fdd60 con 0x7f7f54101620 2024-03-04T08:49:15.415 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.405+0000 7f7f4b7fe700 1 -- 172.21.15.158:0/389262239 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f5000bbd0 con 0x7f7f54101620 2024-03-04T08:49:15.415 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.405+0000 7f7f4b7fe700 1 -- 172.21.15.158:0/389262239 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7f500039b0 con 0x7f7f54101620 2024-03-04T08:49:15.415 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.405+0000 7f7f5c704700 1 -- 172.21.15.158:0/389262239 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f541023d0 con 0x7f7f54101620 2024-03-04T08:49:15.416 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.406+0000 7f7f4b7fe700 1 -- 172.21.15.158:0/389262239 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f7f50029070 con 0x7f7f54101620 2024-03-04T08:49:15.416 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.406+0000 7f7f4b7fe700 1 --2- 172.21.15.158:0/389262239 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f7f40038c40 0x7f7f4003b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:49:15.416 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.406+0000 7f7f4b7fe700 1 -- 172.21.15.158:0/389262239 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7f5001e070 con 0x7f7f54101620 2024-03-04T08:49:15.416 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.406+0000 7f7f59c9f700 1 --2- 172.21.15.158:0/389262239 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f7f40038c40 0x7f7f4003b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:49:15.416 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.407+0000 7f7f59c9f700 1 --2- 172.21.15.158:0/389262239 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f7f40038c40 0x7f7f4003b100 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7f44006fd0 tx=0x7f7f44006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:49:15.416 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.411+0000 7f7f4b7fe700 1 -- 172.21.15.158:0/389262239 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7f50013070 con 0x7f7f54101620 2024-03-04T08:49:15.498 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 podman[37187]: 2024-03-04 08:49:15.502038132 +0000 UTC m=+0.294006452 container create 78dd30fa6ca723f20dd601f927fd348f9d18406e2ebb15468bcba74089501d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-c, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD) 2024-03-04T08:49:15.686 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.689+0000 7f7f5c704700 1 -- 172.21.15.158:0/389262239 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7f540fe840 con 0x7f7f54101620 2024-03-04T08:49:15.686 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.689+0000 7f7f4b7fe700 1 -- 172.21.15.158:0/389262239 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+745 (secure 0 0 0) 0x7f7f50015350 con 0x7f7f54101620 2024-03-04T08:49:15.686 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-03-04T08:49:15.686 INFO:teuthology.orchestra.run.smithi158.stdout:{"epoch":1,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","modified":"2024-03-04T08:47:22.873299Z","created":"2024-03-04T08:47:22.873299Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-04T08:49:15.688 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.691+0000 7f7f5c704700 1 -- 172.21.15.158:0/389262239 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f7f40038c40 msgr2=0x7f7f4003b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:15.688 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.691+0000 7f7f5c704700 1 --2- 172.21.15.158:0/389262239 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f7f40038c40 0x7f7f4003b100 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7f44006fd0 tx=0x7f7f44006e40 comp rx=0 tx=0).stop 2024-03-04T08:49:15.688 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.691+0000 7f7f5c704700 1 -- 172.21.15.158:0/389262239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f54101620 msgr2=0x7f7f540fd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:15.688 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.691+0000 7f7f5c704700 1 --2- 172.21.15.158:0/389262239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f54101620 0x7f7f540fd360 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7f50005850 tx=0x7f7f50005030 comp rx=0 tx=0).stop 2024-03-04T08:49:15.688 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.691+0000 7f7f5c704700 1 -- 172.21.15.158:0/389262239 shutdown_connections 2024-03-04T08:49:15.688 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.691+0000 7f7f5c704700 1 --2- 172.21.15.158:0/389262239 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f7f40038c40 0x7f7f4003b100 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:15.688 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.691+0000 7f7f5c704700 1 --2- 172.21.15.158:0/389262239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f54101620 0x7f7f540fd360 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:15.688 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.691+0000 7f7f5c704700 1 -- 172.21.15.158:0/389262239 >> 172.21.15.158:0/389262239 conn(0x7f7f54077080 msgr2=0x7f7f54104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:49:15.688 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.692+0000 7f7f5c704700 1 -- 172.21.15.158:0/389262239 shutdown_connections 2024-03-04T08:49:15.689 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:15.692+0000 7f7f5c704700 1 -- 172.21.15.158:0/389262239 wait complete. 2024-03-04T08:49:15.689 INFO:teuthology.orchestra.run.smithi158.stderr:dumped monmap epoch 1 2024-03-04T08:49:15.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:15 smithi071 ceph-mon[32789]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:16.085 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 podman[37187]: 2024-03-04 08:49:15.830261306 +0000 UTC m=+0.622229634 container init 78dd30fa6ca723f20dd601f927fd348f9d18406e2ebb15468bcba74089501d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T08:49:16.086 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 podman[37187]: 2024-03-04 08:49:15.833627526 +0000 UTC m=+0.625595854 container start 78dd30fa6ca723f20dd601f927fd348f9d18406e2ebb15468bcba74089501d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-c, ceph=True, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T08:49:16.086 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: set uid:gid to 167:167 (ceph:ceph) 2024-03-04T08:49:16.086 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: ceph version 19.0.0-1739-g1bc250c7 (1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd) squid (dev), process ceph-mon, pid 2 2024-03-04T08:49:16.086 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: pidfile_write: ignore empty --pid-file 2024-03-04T08:49:16.086 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: load: jerasure load: lrc 2024-03-04T08:49:16.086 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: RocksDB version: 7.9.2 2024-03-04T08:49:16.086 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Git sha 0 2024-03-04T08:49:16.086 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Compile date 2024-03-03 23:16:35 2024-03-04T08:49:16.086 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: DB SUMMARY 2024-03-04T08:49:16.087 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: DB Session ID: B91PG562F0II5ICC8EIQ 2024-03-04T08:49:16.087 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: CURRENT file: CURRENT 2024-03-04T08:49:16.087 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: IDENTITY file: IDENTITY 2024-03-04T08:49:16.087 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-04T08:49:16.087 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-04T08:49:16.087 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-04T08:49:16.087 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.error_if_exists: 0 2024-03-04T08:49:16.087 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.create_if_missing: 0 2024-03-04T08:49:16.087 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.paranoid_checks: 1 2024-03-04T08:49:16.087 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-04T08:49:16.087 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-04T08:49:16.088 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-04T08:49:16.088 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.env: 0x55e1d1e7ab20 2024-03-04T08:49:16.088 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.fs: PosixFileSystem 2024-03-04T08:49:16.088 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.info_log: 0x55e1d3e35920 2024-03-04T08:49:16.088 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_file_opening_threads: 16 2024-03-04T08:49:16.090 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.statistics: (nil) 2024-03-04T08:49:16.090 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.use_fsync: 0 2024-03-04T08:49:16.090 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_log_file_size: 0 2024-03-04T08:49:16.090 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-04T08:49:16.092 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-04T08:49:16.092 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.keep_log_file_num: 1000 2024-03-04T08:49:16.092 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.recycle_log_file_num: 0 2024-03-04T08:49:16.092 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.allow_fallocate: 1 2024-03-04T08:49:16.092 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.allow_mmap_reads: 0 2024-03-04T08:49:16.092 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.allow_mmap_writes: 0 2024-03-04T08:49:16.093 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.use_direct_reads: 0 2024-03-04T08:49:16.093 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-04T08:49:16.093 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.create_missing_column_families: 0 2024-03-04T08:49:16.093 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.db_log_dir: 2024-03-04T08:49:16.093 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.wal_dir: 2024-03-04T08:49:16.093 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-04T08:49:16.093 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-04T08:49:16.093 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-04T08:49:16.093 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-04T08:49:16.093 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-04T08:49:16.093 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-04T08:49:16.093 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.advise_random_on_open: 1 2024-03-04T08:49:16.094 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.db_write_buffer_size: 0 2024-03-04T08:49:16.094 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.write_buffer_manager: 0x55e1d3e09400 2024-03-04T08:49:16.094 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-04T08:49:16.094 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-04T08:49:16.094 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-04T08:49:16.094 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.rate_limiter: (nil) 2024-03-04T08:49:16.094 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-04T08:49:16.094 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.wal_recovery_mode: 2 2024-03-04T08:49:16.094 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.enable_thread_tracking: 0 2024-03-04T08:49:16.094 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.enable_pipelined_write: 0 2024-03-04T08:49:16.094 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.unordered_write: 0 2024-03-04T08:49:16.095 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-04T08:49:16.095 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-04T08:49:16.095 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-04T08:49:16.095 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-04T08:49:16.095 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.row_cache: None 2024-03-04T08:49:16.095 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.wal_filter: None 2024-03-04T08:49:16.095 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-04T08:49:16.095 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.allow_ingest_behind: 0 2024-03-04T08:49:16.095 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.two_write_queues: 0 2024-03-04T08:49:16.095 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.manual_wal_flush: 0 2024-03-04T08:49:16.095 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.wal_compression: 0 2024-03-04T08:49:16.095 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.atomic_flush: 0 2024-03-04T08:49:16.096 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-04T08:49:16.096 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-04T08:49:16.096 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-04T08:49:16.096 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.log_readahead_size: 0 2024-03-04T08:49:16.096 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-04T08:49:16.096 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.best_efforts_recovery: 0 2024-03-04T08:49:16.096 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-04T08:49:16.096 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-04T08:49:16.096 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.allow_data_in_errors: 0 2024-03-04T08:49:16.096 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.db_host_id: __hostname__ 2024-03-04T08:49:16.096 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-04T08:49:16.097 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_background_jobs: 2 2024-03-04T08:49:16.097 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_background_compactions: -1 2024-03-04T08:49:16.097 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_subcompactions: 1 2024-03-04T08:49:16.097 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-04T08:49:16.097 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-04T08:49:16.097 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-04T08:49:16.097 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_total_wal_size: 0 2024-03-04T08:49:16.097 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-04T08:49:16.097 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-04T08:49:16.097 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-04T08:49:16.097 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-04T08:49:16.097 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_open_files: -1 2024-03-04T08:49:16.097 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.bytes_per_sync: 0 2024-03-04T08:49:16.098 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-04T08:49:16.098 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-04T08:49:16.098 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.compaction_readahead_size: 0 2024-03-04T08:49:16.098 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_background_flushes: -1 2024-03-04T08:49:16.098 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Compression algorithms supported: 2024-03-04T08:49:16.098 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-04T08:49:16.098 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: kZSTD supported: 0 2024-03-04T08:49:16.098 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: kXpressCompression supported: 0 2024-03-04T08:49:16.098 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: kLZ4HCCompression supported: 1 2024-03-04T08:49:16.098 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: kZlibCompression supported: 1 2024-03-04T08:49:16.098 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: kSnappyCompression supported: 1 2024-03-04T08:49:16.099 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: kLZ4Compression supported: 1 2024-03-04T08:49:16.099 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: kBZip2Compression supported: 0 2024-03-04T08:49:16.099 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-04T08:49:16.099 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-04T08:49:16.099 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-04T08:49:16.099 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-04T08:49:16.099 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-04T08:49:16.099 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.merge_operator: 2024-03-04T08:49:16.099 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.compaction_filter: None 2024-03-04T08:49:16.099 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.compaction_filter_factory: None 2024-03-04T08:49:16.099 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.sst_partitioner_factory: None 2024-03-04T08:49:16.099 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-04T08:49:16.100 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-04T08:49:16.100 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e1d3e35a20) 2024-03-04T08:49:16.100 INFO:journalctl@ceph.mon.c.smithi158.stdout: cache_index_and_filter_blocks: 1 2024-03-04T08:49:16.100 INFO:journalctl@ceph.mon.c.smithi158.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-04T08:49:16.100 INFO:journalctl@ceph.mon.c.smithi158.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-04T08:49:16.100 INFO:journalctl@ceph.mon.c.smithi158.stdout: pin_top_level_index_and_filter: 1 2024-03-04T08:49:16.100 INFO:journalctl@ceph.mon.c.smithi158.stdout: index_type: 0 2024-03-04T08:49:16.100 INFO:journalctl@ceph.mon.c.smithi158.stdout: data_block_index_type: 0 2024-03-04T08:49:16.100 INFO:journalctl@ceph.mon.c.smithi158.stdout: index_shortening: 1 2024-03-04T08:49:16.100 INFO:journalctl@ceph.mon.c.smithi158.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-04T08:49:16.101 INFO:journalctl@ceph.mon.c.smithi158.stdout: checksum: 4 2024-03-04T08:49:16.101 INFO:journalctl@ceph.mon.c.smithi158.stdout: no_block_cache: 0 2024-03-04T08:49:16.101 INFO:journalctl@ceph.mon.c.smithi158.stdout: block_cache: 0x55e1d3e39090 2024-03-04T08:49:16.101 INFO:journalctl@ceph.mon.c.smithi158.stdout: block_cache_name: BinnedLRUCache 2024-03-04T08:49:16.101 INFO:journalctl@ceph.mon.c.smithi158.stdout: block_cache_options: 2024-03-04T08:49:16.101 INFO:journalctl@ceph.mon.c.smithi158.stdout: capacity : 536870912 2024-03-04T08:49:16.101 INFO:journalctl@ceph.mon.c.smithi158.stdout: num_shard_bits : 4 2024-03-04T08:49:16.101 INFO:journalctl@ceph.mon.c.smithi158.stdout: strict_capacity_limit : 0 2024-03-04T08:49:16.101 INFO:journalctl@ceph.mon.c.smithi158.stdout: high_pri_pool_ratio: 0.000 2024-03-04T08:49:16.101 INFO:journalctl@ceph.mon.c.smithi158.stdout: block_cache_compressed: (nil) 2024-03-04T08:49:16.101 INFO:journalctl@ceph.mon.c.smithi158.stdout: persistent_cache: (nil) 2024-03-04T08:49:16.102 INFO:journalctl@ceph.mon.c.smithi158.stdout: block_size: 4096 2024-03-04T08:49:16.102 INFO:journalctl@ceph.mon.c.smithi158.stdout: block_size_deviation: 10 2024-03-04T08:49:16.102 INFO:journalctl@ceph.mon.c.smithi158.stdout: block_restart_interval: 16 2024-03-04T08:49:16.102 INFO:journalctl@ceph.mon.c.smithi158.stdout: index_block_restart_interval: 1 2024-03-04T08:49:16.102 INFO:journalctl@ceph.mon.c.smithi158.stdout: metadata_block_size: 4096 2024-03-04T08:49:16.102 INFO:journalctl@ceph.mon.c.smithi158.stdout: partition_filters: 0 2024-03-04T08:49:16.102 INFO:journalctl@ceph.mon.c.smithi158.stdout: use_delta_encoding: 1 2024-03-04T08:49:16.102 INFO:journalctl@ceph.mon.c.smithi158.stdout: filter_policy: bloomfilter 2024-03-04T08:49:16.102 INFO:journalctl@ceph.mon.c.smithi158.stdout: whole_key_filtering: 1 2024-03-04T08:49:16.102 INFO:journalctl@ceph.mon.c.smithi158.stdout: verify_compression: 0 2024-03-04T08:49:16.102 INFO:journalctl@ceph.mon.c.smithi158.stdout: read_amp_bytes_per_bit: 0 2024-03-04T08:49:16.102 INFO:journalctl@ceph.mon.c.smithi158.stdout: format_version: 5 2024-03-04T08:49:16.103 INFO:journalctl@ceph.mon.c.smithi158.stdout: enable_index_compression: 1 2024-03-04T08:49:16.103 INFO:journalctl@ceph.mon.c.smithi158.stdout: block_align: 0 2024-03-04T08:49:16.103 INFO:journalctl@ceph.mon.c.smithi158.stdout: max_auto_readahead_size: 262144 2024-03-04T08:49:16.103 INFO:journalctl@ceph.mon.c.smithi158.stdout: prepopulate_block_cache: 0 2024-03-04T08:49:16.103 INFO:journalctl@ceph.mon.c.smithi158.stdout: initial_auto_readahead_size: 8192 2024-03-04T08:49:16.103 INFO:journalctl@ceph.mon.c.smithi158.stdout: num_file_reads_for_auto_readahead: 2 2024-03-04T08:49:16.103 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.write_buffer_size: 33554432 2024-03-04T08:49:16.103 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_write_buffer_number: 2 2024-03-04T08:49:16.103 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.compression: NoCompression 2024-03-04T08:49:16.103 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.bottommost_compression: Disabled 2024-03-04T08:49:16.103 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.prefix_extractor: nullptr 2024-03-04T08:49:16.103 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-04T08:49:16.103 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.num_levels: 7 2024-03-04T08:49:16.104 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-04T08:49:16.104 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-04T08:49:16.104 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-04T08:49:16.104 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-04T08:49:16.104 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-04T08:49:16.104 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-04T08:49:16.104 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-04T08:49:16.104 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-04T08:49:16.104 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-04T08:49:16.104 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-04T08:49:16.104 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-04T08:49:16.104 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-04T08:49:16.104 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-04T08:49:16.104 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.compression_opts.level: 32767 2024-03-04T08:49:16.105 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.compression_opts.strategy: 0 2024-03-04T08:49:16.105 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-04T08:49:16.105 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-04T08:49:16.105 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-04T08:49:16.105 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-04T08:49:16.105 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.compression_opts.enabled: false 2024-03-04T08:49:16.105 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-04T08:49:16.105 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-04T08:49:16.105 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-04T08:49:16.105 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-04T08:49:16.105 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.target_file_size_base: 67108864 2024-03-04T08:49:16.105 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-04T08:49:16.105 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-04T08:49:16.105 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-04T08:49:16.105 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-04T08:49:16.106 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-04T08:49:16.106 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-04T08:49:16.106 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-04T08:49:16.106 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-04T08:49:16.106 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-04T08:49:16.106 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-04T08:49:16.106 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-04T08:49:16.106 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-04T08:49:16.106 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-04T08:49:16.106 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-04T08:49:16.106 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.arena_block_size: 1048576 2024-03-04T08:49:16.106 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-04T08:49:16.106 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-04T08:49:16.106 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.disable_auto_compactions: 0 2024-03-04T08:49:16.107 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-04T08:49:16.107 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-04T08:49:16.107 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-04T08:49:16.107 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-04T08:49:16.107 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-04T08:49:16.107 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-04T08:49:16.107 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-04T08:49:16.107 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-04T08:49:16.107 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-04T08:49:16.107 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-04T08:49:16.107 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.table_properties_collectors: 2024-03-04T08:49:16.107 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.inplace_update_support: 0 2024-03-04T08:49:16.107 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-04T08:49:16.107 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-04T08:49:16.108 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-04T08:49:16.108 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-04T08:49:16.108 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.bloom_locality: 0 2024-03-04T08:49:16.108 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.max_successive_merges: 0 2024-03-04T08:49:16.108 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-04T08:49:16.108 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.paranoid_file_checks: 0 2024-03-04T08:49:16.108 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.force_consistency_checks: 1 2024-03-04T08:49:16.108 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.report_bg_io_stats: 0 2024-03-04T08:49:16.108 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.ttl: 2592000 2024-03-04T08:49:16.108 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-04T08:49:16.108 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-04T08:49:16.108 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-04T08:49:16.108 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.enable_blob_files: false 2024-03-04T08:49:16.108 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.min_blob_size: 0 2024-03-04T08:49:16.109 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.blob_file_size: 268435456 2024-03-04T08:49:16.109 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-04T08:49:16.109 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-04T08:49:16.109 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-04T08:49:16.109 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-04T08:49:16.109 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-04T08:49:16.109 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.blob_file_starting_level: 0 2024-03-04T08:49:16.109 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-04T08:49:16.109 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-04T08:49:16.109 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-04T08:49:16.109 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 417360b7-0dbe-4386-b4f5-f80bd33bdc5a 2024-03-04T08:49:16.109 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709542155863850, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-04T08:49:16.109 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-04T08:49:16.109 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709542155864267, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1709542155, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "417360b7-0dbe-4386-b4f5-f80bd33bdc5a", "db_session_id": "B91PG562F0II5ICC8EIQ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-04T08:49:16.110 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709542155864347, "job": 1, "event": "recovery_finished"} 2024-03-04T08:49:16.110 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-04T08:49:16.110 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-04T08:49:16.110 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e1d3f2a000 2024-03-04T08:49:16.110 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: DB pointer 0x55e1d3f14000 2024-03-04T08:49:16.110 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-04T08:49:16.110 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-04T08:49:16.110 INFO:journalctl@ceph.mon.c.smithi158.stdout: ** DB Stats ** 2024-03-04T08:49:16.110 INFO:journalctl@ceph.mon.c.smithi158.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-04T08:49:16.110 INFO:journalctl@ceph.mon.c.smithi158.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-04T08:49:16.110 INFO:journalctl@ceph.mon.c.smithi158.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-04T08:49:16.110 INFO:journalctl@ceph.mon.c.smithi158.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-04T08:49:16.111 INFO:journalctl@ceph.mon.c.smithi158.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-04T08:49:16.111 INFO:journalctl@ceph.mon.c.smithi158.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-04T08:49:16.111 INFO:journalctl@ceph.mon.c.smithi158.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-04T08:49:16.111 INFO:journalctl@ceph.mon.c.smithi158.stdout: 2024-03-04T08:49:16.111 INFO:journalctl@ceph.mon.c.smithi158.stdout: ** Compaction Stats [default] ** 2024-03-04T08:49:16.111 INFO:journalctl@ceph.mon.c.smithi158.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-04T08:49:16.111 INFO:journalctl@ceph.mon.c.smithi158.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-04T08:49:16.111 INFO:journalctl@ceph.mon.c.smithi158.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-04T08:49:16.111 INFO:journalctl@ceph.mon.c.smithi158.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-04T08:49:16.111 INFO:journalctl@ceph.mon.c.smithi158.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-04T08:49:16.111 INFO:journalctl@ceph.mon.c.smithi158.stdout: 2024-03-04T08:49:16.111 INFO:journalctl@ceph.mon.c.smithi158.stdout: ** Compaction Stats [default] ** 2024-03-04T08:49:16.111 INFO:journalctl@ceph.mon.c.smithi158.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-04T08:49:16.112 INFO:journalctl@ceph.mon.c.smithi158.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-04T08:49:16.112 INFO:journalctl@ceph.mon.c.smithi158.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-04T08:49:16.112 INFO:journalctl@ceph.mon.c.smithi158.stdout: 2024-03-04T08:49:16.112 INFO:journalctl@ceph.mon.c.smithi158.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-04T08:49:16.112 INFO:journalctl@ceph.mon.c.smithi158.stdout: 2024-03-04T08:49:16.112 INFO:journalctl@ceph.mon.c.smithi158.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-04T08:49:16.112 INFO:journalctl@ceph.mon.c.smithi158.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-04T08:49:16.112 INFO:journalctl@ceph.mon.c.smithi158.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-04T08:49:16.112 INFO:journalctl@ceph.mon.c.smithi158.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-04T08:49:16.112 INFO:journalctl@ceph.mon.c.smithi158.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-04T08:49:16.112 INFO:journalctl@ceph.mon.c.smithi158.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-04T08:49:16.112 INFO:journalctl@ceph.mon.c.smithi158.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-04T08:49:16.112 INFO:journalctl@ceph.mon.c.smithi158.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-04T08:49:16.113 INFO:journalctl@ceph.mon.c.smithi158.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-04T08:49:16.113 INFO:journalctl@ceph.mon.c.smithi158.stdout: Block cache BinnedLRUCache@0x55e1d3e39090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-04T08:49:16.113 INFO:journalctl@ceph.mon.c.smithi158.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-04T08:49:16.113 INFO:journalctl@ceph.mon.c.smithi158.stdout: 2024-03-04T08:49:16.113 INFO:journalctl@ceph.mon.c.smithi158.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-04T08:49:16.113 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-04T08:49:16.113 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: using public_addr v2:172.21.15.158:0/0 -> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] 2024-03-04T08:49:16.113 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: starting mon.c rank -1 at public addrs [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] at bind addrs [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T08:49:16.113 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: mon.c@-1(???) e0 preinit fsid b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T08:49:16.113 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: mon.c@-1(synchronizing).mds e1 new map 2024-03-04T08:49:16.113 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-04T08:49:16.113 INFO:journalctl@ceph.mon.c.smithi158.stdout: e1 2024-03-04T08:49:16.114 INFO:journalctl@ceph.mon.c.smithi158.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T08:49:16.114 INFO:journalctl@ceph.mon.c.smithi158.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-04T08:49:16.114 INFO:journalctl@ceph.mon.c.smithi158.stdout: legacy client fscid: -1 2024-03-04T08:49:16.114 INFO:journalctl@ceph.mon.c.smithi158.stdout: 2024-03-04T08:49:16.114 INFO:journalctl@ceph.mon.c.smithi158.stdout: No filesystems configured 2024-03-04T08:49:16.114 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-04T08:49:16.114 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-04T08:49:16.114 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-04T08:49:16.114 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-04T08:49:16.114 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-04T08:49:16.114 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-04T08:49:16.114 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-04T08:49:16.115 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-04T08:49:16.115 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-04T08:49:16.115 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-04T08:49:16.115 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:16.115 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:16.115 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T08:49:16.115 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:16.115 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi158", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:49:16.115 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:16.115 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:16.115 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:16.115 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:16.115 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:49:16.116 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:16.116 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:49:16.116 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: Updating smithi150:/etc/ceph/ceph.conf 2024-03-04T08:49:16.116 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:16.116 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: Updating smithi150:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T08:49:16.116 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:16.116 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:16.116 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:16.116 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-03-04T08:49:16.116 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: Deploying cephadm binary to smithi158 2024-03-04T08:49:16.116 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: Updating smithi150:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T08:49:16.117 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:16.117 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:16.117 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: Added host smithi158 2024-03-04T08:49:16.117 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:49:16.117 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:16.117 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:16.117 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:16.117 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:16.117 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:16.117 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T08:49:16.117 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:16.117 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3420679694' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-04T08:49:16.118 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:16.118 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:16.118 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:16.118 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:16.118 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi158", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:49:16.118 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:16.118 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:49:16.118 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: Updating smithi158:/etc/ceph/ceph.conf 2024-03-04T08:49:16.118 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: Updating smithi158:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T08:49:16.118 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: Updating smithi158:/etc/ceph/ceph.client.admin.keyring 2024-03-04T08:49:16.118 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3420679694' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-04T08:49:16.119 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: osdmap e4: 0 total, 0 up, 0 in 2024-03-04T08:49:16.119 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: Updating smithi158:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T08:49:16.119 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:16.119 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:16.119 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:16.119 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:16.119 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:16.119 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:16.119 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:49:16.119 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:16.119 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:49:16.119 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:16.120 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T08:49:16.120 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:16.120 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi071:172.21.15.71=a;smithi150:172.21.15.150=b;smithi158:172.21.15.158=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:49:16.120 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: Saving service mon spec with placement smithi071:172.21.15.71=a;smithi150:172.21.15.150=b;smithi158:172.21.15.158=c;count:3 2024-03-04T08:49:16.120 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: Deploying daemon mon.c on smithi158 2024-03-04T08:49:16.120 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:16.120 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:16.120 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:16.120 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:16.120 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:16.120 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:16.120 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:15 smithi158 ceph-mon[37241]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-04T08:49:16.121 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:16 smithi158 bash[37187]: 78dd30fa6ca723f20dd601f927fd348f9d18406e2ebb15468bcba74089501d59 2024-03-04T08:49:16.121 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:16 smithi158 systemd[1]: Started Ceph mon.c for b3c85c40-da03-11ee-95c6-87774f69a715. 2024-03-04T08:49:17.491 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-04T08:49:17.491 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T08:49:17.736 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T08:49:21.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:20 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T08:49:21.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:20 smithi158 ceph-mon[37241]: mon.a calling monitor election 2024-03-04T08:49:21.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:20 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:49:21.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:20 smithi158 ceph-mon[37241]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:21.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:20 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:49:21.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:20 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:49:21.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:20 smithi158 ceph-mon[37241]: mon.c calling monitor election 2024-03-04T08:49:21.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:20 smithi158 ceph-mon[37241]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:21.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:20 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:49:21.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:20 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:49:21.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:20 smithi158 ceph-mon[37241]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:21.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:20 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:49:21.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:20 smithi158 ceph-mon[37241]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-04T08:49:21.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:20 smithi158 ceph-mon[37241]: monmap e2: 2 mons at {a=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],c=[v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T08:49:21.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:20 smithi158 ceph-mon[37241]: fsmap 2024-03-04T08:49:21.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:20 smithi158 ceph-mon[37241]: osdmap e4: 0 total, 0 up, 0 in 2024-03-04T08:49:21.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:20 smithi158 ceph-mon[37241]: mgrmap e13: x(active, since 58s) 2024-03-04T08:49:21.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:20 smithi158 ceph-mon[37241]: overall HEALTH_OK 2024-03-04T08:49:21.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:20 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:21.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:20 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:21.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:20 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T08:49:21.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:20 smithi071 ceph-mon[32789]: mon.a calling monitor election 2024-03-04T08:49:21.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:20 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:49:21.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:20 smithi071 ceph-mon[32789]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:21.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:20 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:49:21.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:20 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:49:21.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:20 smithi071 ceph-mon[32789]: mon.c calling monitor election 2024-03-04T08:49:21.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:20 smithi071 ceph-mon[32789]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:21.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:20 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:49:21.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:20 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:49:21.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:20 smithi071 ceph-mon[32789]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:21.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:20 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:49:21.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:20 smithi071 ceph-mon[32789]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-04T08:49:21.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:20 smithi071 ceph-mon[32789]: monmap e2: 2 mons at {a=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],c=[v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T08:49:21.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:20 smithi071 ceph-mon[32789]: fsmap 2024-03-04T08:49:21.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:20 smithi071 ceph-mon[32789]: osdmap e4: 0 total, 0 up, 0 in 2024-03-04T08:49:21.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:20 smithi071 ceph-mon[32789]: mgrmap e13: x(active, since 58s) 2024-03-04T08:49:21.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:20 smithi071 ceph-mon[32789]: overall HEALTH_OK 2024-03-04T08:49:21.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:20 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:21.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:20 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:22.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:21 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:22.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:21 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T08:49:22.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:21 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:22.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:21 smithi158 ceph-mon[37241]: Deploying daemon mon.b on smithi150 2024-03-04T08:49:22.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:21 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:49:22.308 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.311+0000 7fcf05644700 1 -- 172.21.15.158:0/1956291118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee80035c0 msgr2=0x7fcee8005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:22.308 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.311+0000 7fcf05644700 1 --2- 172.21.15.158:0/1956291118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee80035c0 0x7fcee8005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcf00101330 tx=0x7fcef000feb0 comp rx=0 tx=0).stop 2024-03-04T08:49:22.308 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.311+0000 7fcf05644700 1 -- 172.21.15.158:0/1956291118 shutdown_connections 2024-03-04T08:49:22.308 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.311+0000 7fcf05644700 1 --2- 172.21.15.158:0/1956291118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee80035c0 0x7fcee8005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:22.308 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.311+0000 7fcf05644700 1 -- 172.21.15.158:0/1956291118 >> 172.21.15.158:0/1956291118 conn(0x7fcf000771b0 msgr2=0x7fcf000775c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:49:22.308 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.311+0000 7fcf05644700 1 -- 172.21.15.158:0/1956291118 shutdown_connections 2024-03-04T08:49:22.309 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.311+0000 7fcf05644700 1 -- 172.21.15.158:0/1956291118 wait complete. 2024-03-04T08:49:22.309 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.312+0000 7fcf05644700 1 Processor -- start 2024-03-04T08:49:22.311 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.313+0000 7fcf05644700 1 -- start start 2024-03-04T08:49:22.311 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.313+0000 7fcf05644700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee80035c0 0x7fcf001a2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:49:22.311 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.313+0000 7fcf05644700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf001a2680 0x7fcf001a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:49:22.311 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.313+0000 7fcf05644700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0006c010 con 0x7fcee80035c0 2024-03-04T08:49:22.311 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.313+0000 7fcf05644700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0006be90 con 0x7fcf001a2680 2024-03-04T08:49:22.311 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.313+0000 7fcefe7fc700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf001a2680 0x7fcf001a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:49:22.311 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.313+0000 7fcefe7fc700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf001a2680 0x7fcf001a6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.158:52912/0 (socket says 172.21.15.158:52912) 2024-03-04T08:49:22.312 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.313+0000 7fcefe7fc700 1 -- 172.21.15.158:0/2029795663 learned_addr learned my addr 172.21.15.158:0/2029795663 (peer_addr_for_me v2:172.21.15.158:0/0) 2024-03-04T08:49:22.312 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.313+0000 7fcefeffd700 1 --2- 172.21.15.158:0/2029795663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee80035c0 0x7fcf001a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:49:22.312 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.313+0000 7fcefe7fc700 1 -- 172.21.15.158:0/2029795663 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf001a2680 msgr2=0x7fcf001a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-04T08:49:22.312 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.313+0000 7fcefe7fc700 1 -- 172.21.15.158:0/2029795663 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf001a2680 msgr2=0x7fcf001a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-04T08:49:22.312 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.313+0000 7fcefe7fc700 1 --2- 172.21.15.158:0/2029795663 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf001a2680 0x7fcf001a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-04T08:49:22.312 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.313+0000 7fcefe7fc700 1 --2- 172.21.15.158:0/2029795663 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf001a2680 0x7fcf001a6a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:49:22.312 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.314+0000 7fcefeffd700 1 -- 172.21.15.158:0/2029795663 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf001a2680 msgr2=0x7fcf001a6a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T08:49:22.312 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.314+0000 7fcefeffd700 1 --2- 172.21.15.158:0/2029795663 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf001a2680 0x7fcf001a6a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:22.312 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.314+0000 7fcefeffd700 1 -- 172.21.15.158:0/2029795663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf001a2cc0 con 0x7fcee80035c0 2024-03-04T08:49:22.312 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.314+0000 7fcefeffd700 1 --2- 172.21.15.158:0/2029795663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee80035c0 0x7fcf001a2140 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcef00059d0 tx=0x7fcef00057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:49:22.312 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.314+0000 7fcee7fff700 1 -- 172.21.15.158:0/2029795663 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fcef0004b20 con 0x7fcee80035c0 2024-03-04T08:49:22.312 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.314+0000 7fcee7fff700 1 -- 172.21.15.158:0/2029795663 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcef0013e40 con 0x7fcee80035c0 2024-03-04T08:49:22.312 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.314+0000 7fcee7fff700 1 -- 172.21.15.158:0/2029795663 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fcef0011cd0 con 0x7fcee80035c0 2024-03-04T08:49:22.313 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.314+0000 7fcf05644700 1 -- 172.21.15.158:0/2029795663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf001a6f50 con 0x7fcee80035c0 2024-03-04T08:49:22.313 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.314+0000 7fcf05644700 1 -- 172.21.15.158:0/2029795663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf001a74b0 con 0x7fcee80035c0 2024-03-04T08:49:22.314 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.315+0000 7fcee7fff700 1 -- 172.21.15.158:0/2029795663 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7fcef0023070 con 0x7fcee80035c0 2024-03-04T08:49:22.314 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.315+0000 7fcf05644700 1 -- 172.21.15.158:0/2029795663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf0006a000 con 0x7fcee80035c0 2024-03-04T08:49:22.314 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.316+0000 7fcee7fff700 1 --2- 172.21.15.158:0/2029795663 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fceec0389e0 0x7fceec03aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:49:22.314 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.316+0000 7fcee7fff700 1 -- 172.21.15.158:0/2029795663 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcef0020050 con 0x7fcee80035c0 2024-03-04T08:49:22.314 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.316+0000 7fcefe7fc700 1 --2- 172.21.15.158:0/2029795663 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fceec0389e0 0x7fceec03aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:49:22.314 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.316+0000 7fcefe7fc700 1 --2- 172.21.15.158:0/2029795663 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fceec0389e0 0x7fceec03aea0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009180 tx=0x7fcef4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:49:22.317 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.321+0000 7fcee7fff700 1 -- 172.21.15.158:0/2029795663 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fcef0031020 con 0x7fcee80035c0 2024-03-04T08:49:22.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:21 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:22.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:21 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T08:49:22.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:21 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:22.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:21 smithi071 ceph-mon[32789]: Deploying daemon mon.b on smithi150 2024-03-04T08:49:22.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:21 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:49:22.593 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.597+0000 7fcf05644700 1 -- 172.21.15.158:0/2029795663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcf001a7760 con 0x7fcee80035c0 2024-03-04T08:49:22.594 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.598+0000 7fcee7fff700 1 -- 172.21.15.158:0/2029795663 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7fcef0028030 con 0x7fcee80035c0 2024-03-04T08:49:22.594 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-03-04T08:49:22.594 INFO:teuthology.orchestra.run.smithi158.stdout:{"epoch":2,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","modified":"2024-03-04T08:49:15.889345Z","created":"2024-03-04T08:47:22.873299Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:3300","nonce":0},{"type":"v1","addr":"172.21.15.158:6789","nonce":0}]},"addr":"172.21.15.158:6789/0","public_addr":"172.21.15.158:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-04T08:49:22.596 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.600+0000 7fcf05644700 1 -- 172.21.15.158:0/2029795663 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fceec0389e0 msgr2=0x7fceec03aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:22.596 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.600+0000 7fcf05644700 1 --2- 172.21.15.158:0/2029795663 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fceec0389e0 0x7fceec03aea0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009180 tx=0x7fcef4008040 comp rx=0 tx=0).stop 2024-03-04T08:49:22.596 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.600+0000 7fcf05644700 1 -- 172.21.15.158:0/2029795663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee80035c0 msgr2=0x7fcf001a2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:22.596 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.600+0000 7fcf05644700 1 --2- 172.21.15.158:0/2029795663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee80035c0 0x7fcf001a2140 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcef00059d0 tx=0x7fcef00057d0 comp rx=0 tx=0).stop 2024-03-04T08:49:22.596 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.600+0000 7fcf05644700 1 -- 172.21.15.158:0/2029795663 shutdown_connections 2024-03-04T08:49:22.596 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.600+0000 7fcf05644700 1 --2- 172.21.15.158:0/2029795663 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fceec0389e0 0x7fceec03aea0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:22.596 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.600+0000 7fcf05644700 1 --2- 172.21.15.158:0/2029795663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcee80035c0 0x7fcf001a2140 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:22.596 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.600+0000 7fcf05644700 1 --2- 172.21.15.158:0/2029795663 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fcf001a2680 0x7fcf001a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:22.596 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.600+0000 7fcf05644700 1 -- 172.21.15.158:0/2029795663 >> 172.21.15.158:0/2029795663 conn(0x7fcf000771b0 msgr2=0x7fcf000ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:49:22.596 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.600+0000 7fcf05644700 1 -- 172.21.15.158:0/2029795663 shutdown_connections 2024-03-04T08:49:22.597 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:22.600+0000 7fcf05644700 1 -- 172.21.15.158:0/2029795663 wait complete. 2024-03-04T08:49:22.597 INFO:teuthology.orchestra.run.smithi158.stderr:dumped monmap epoch 2 2024-03-04T08:49:23.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:23 smithi158 ceph-mon[37241]: from='client.? 172.21.15.158:0/2029795663' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T08:49:23.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:23 smithi158 ceph-mon[37241]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:23.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:23 smithi071 ceph-mon[32789]: from='client.? 172.21.15.158:0/2029795663' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T08:49:23.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:23 smithi071 ceph-mon[32789]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:24.452 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-04T08:49:24.453 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T08:49:24.697 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T08:49:25.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:24 smithi150 systemd[1]: Started Ceph mon.b for b3c85c40-da03-11ee-95c6-87774f69a715. 2024-03-04T08:49:29.917 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.921+0000 7f6b44a26700 1 -- 172.21.15.158:0/2920684184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b2c0057d0 msgr2=0x7f6b2c007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:29.918 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.921+0000 7f6b44a26700 1 --2- 172.21.15.158:0/2920684184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b2c0057d0 0x7f6b2c007c60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6b401009f0 tx=0x7f6b30009e80 comp rx=0 tx=0).stop 2024-03-04T08:49:29.918 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.921+0000 7f6b44a26700 1 -- 172.21.15.158:0/2920684184 shutdown_connections 2024-03-04T08:49:29.918 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.921+0000 7f6b44a26700 1 --2- 172.21.15.158:0/2920684184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b2c0057d0 0x7f6b2c007c60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:29.918 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.921+0000 7f6b44a26700 1 -- 172.21.15.158:0/2920684184 >> 172.21.15.158:0/2920684184 conn(0x7f6b400fc5d0 msgr2=0x7f6b400fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:49:29.918 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.921+0000 7f6b44a26700 1 -- 172.21.15.158:0/2920684184 shutdown_connections 2024-03-04T08:49:29.918 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.921+0000 7f6b44a26700 1 -- 172.21.15.158:0/2920684184 wait complete. 2024-03-04T08:49:29.920 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.922+0000 7f6b44a26700 1 Processor -- start 2024-03-04T08:49:29.920 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.922+0000 7f6b44a26700 1 -- start start 2024-03-04T08:49:29.920 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.923+0000 7f6b44a26700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b2c0057d0 0x7f6b40195600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:49:29.920 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.923+0000 7f6b44a26700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6b40195b40 0x7f6b40199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:49:29.920 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.923+0000 7f6b44a26700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b4019a3c0 0x7f6b4019a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:49:29.920 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.923+0000 7f6b44a26700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b40107ae0 con 0x7f6b2c0057d0 2024-03-04T08:49:29.921 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.923+0000 7f6b44a26700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b40107960 con 0x7f6b4019a3c0 2024-03-04T08:49:29.921 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.923+0000 7f6b44a26700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b40107c60 con 0x7f6b40195b40 2024-03-04T08:49:29.921 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.923+0000 7f6b3e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b2c0057d0 0x7f6b40195600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:49:29.921 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.923+0000 7f6b3dd9b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6b40195b40 0x7f6b40199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:49:29.921 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.923+0000 7f6b3e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b2c0057d0 0x7f6b40195600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.158:39228/0 (socket says 172.21.15.158:39228) 2024-03-04T08:49:29.921 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.923+0000 7f6b3e59c700 1 -- 172.21.15.158:0/813607965 learned_addr learned my addr 172.21.15.158:0/813607965 (peer_addr_for_me v2:172.21.15.158:0/0) 2024-03-04T08:49:29.921 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.923+0000 7f6b3ed9d700 1 --2- 172.21.15.158:0/813607965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b4019a3c0 0x7f6b4019a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:49:29.921 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.923+0000 7f6b3dd9b700 1 -- 172.21.15.158:0/813607965 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6b40195b40 msgr2=0x7f6b40199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-04T08:49:29.921 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.923+0000 7f6b3dd9b700 1 -- 172.21.15.158:0/813607965 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6b40195b40 msgr2=0x7f6b40199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-04T08:49:29.921 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.923+0000 7f6b3dd9b700 1 --2- 172.21.15.158:0/813607965 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6b40195b40 0x7f6b40199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-04T08:49:29.921 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.923+0000 7f6b3dd9b700 1 --2- 172.21.15.158:0/813607965 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6b40195b40 0x7f6b40199e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:49:29.921 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.923+0000 7f6b3ed9d700 1 -- 172.21.15.158:0/813607965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b4019a3c0 msgr2=0x7f6b4019a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-03-04T08:49:29.921 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.923+0000 7f6b3ed9d700 1 -- 172.21.15.158:0/813607965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b4019a3c0 msgr2=0x7f6b4019a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-04T08:49:29.922 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.923+0000 7f6b3ed9d700 1 --2- 172.21.15.158:0/813607965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b4019a3c0 0x7f6b4019a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-04T08:49:29.922 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.923+0000 7f6b3ed9d700 1 --2- 172.21.15.158:0/813607965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b4019a3c0 0x7f6b4019a870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:49:29.922 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.923+0000 7f6b3e59c700 1 -- 172.21.15.158:0/813607965 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6b40195b40 msgr2=0x7f6b40199e80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T08:49:29.922 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.923+0000 7f6b3e59c700 1 --2- 172.21.15.158:0/813607965 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6b40195b40 0x7f6b40199e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:29.922 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.923+0000 7f6b3e59c700 1 -- 172.21.15.158:0/813607965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b4019a3c0 msgr2=0x7f6b4019a870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T08:49:29.922 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.923+0000 7f6b3e59c700 1 --2- 172.21.15.158:0/813607965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b4019a3c0 0x7f6b4019a870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:29.922 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.923+0000 7f6b3e59c700 1 -- 172.21.15.158:0/813607965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b401a2130 con 0x7f6b2c0057d0 2024-03-04T08:49:29.922 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.924+0000 7f6b3e59c700 1 --2- 172.21.15.158:0/813607965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b2c0057d0 0x7f6b40195600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6b300199d0 tx=0x7f6b30019ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:49:29.922 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.924+0000 7f6b377fe700 1 -- 172.21.15.158:0/813607965 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3002e070 con 0x7f6b2c0057d0 2024-03-04T08:49:29.922 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.924+0000 7f6b377fe700 1 -- 172.21.15.158:0/813607965 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b30005930 con 0x7f6b2c0057d0 2024-03-04T08:49:29.922 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.924+0000 7f6b377fe700 1 -- 172.21.15.158:0/813607965 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b30029c20 con 0x7f6b2c0057d0 2024-03-04T08:49:29.923 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.924+0000 7f6b44a26700 1 -- 172.21.15.158:0/813607965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b401a2300 con 0x7f6b2c0057d0 2024-03-04T08:49:29.923 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.924+0000 7f6b44a26700 1 -- 172.21.15.158:0/813607965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b401a2890 con 0x7f6b2c0057d0 2024-03-04T08:49:29.924 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.925+0000 7f6b377fe700 1 -- 172.21.15.158:0/813607965 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f6b30031070 con 0x7f6b2c0057d0 2024-03-04T08:49:29.924 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.925+0000 7f6b44a26700 1 -- 172.21.15.158:0/813607965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b400636f0 con 0x7f6b2c0057d0 2024-03-04T08:49:29.925 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.925+0000 7f6b377fe700 1 --2- 172.21.15.158:0/813607965 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6b200390d0 0x7f6b2003b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:49:29.925 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.925+0000 7f6b377fe700 1 -- 172.21.15.158:0/813607965 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6b3004e540 con 0x7f6b2c0057d0 2024-03-04T08:49:29.925 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.925+0000 7f6b3dd9b700 1 --2- 172.21.15.158:0/813607965 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6b200390d0 0x7f6b2003b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:49:29.925 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.926+0000 7f6b3dd9b700 1 --2- 172.21.15.158:0/813607965 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6b200390d0 0x7f6b2003b590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6b28009850 tx=0x7f6b28008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:49:29.926 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:29.930+0000 7f6b377fe700 1 -- 172.21.15.158:0/813607965 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6b3001d030 con 0x7f6b2c0057d0 2024-03-04T08:49:30.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:30.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T08:49:30.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: mon.c calling monitor election 2024-03-04T08:49:30.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: mon.a calling monitor election 2024-03-04T08:49:30.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:49:30.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:49:30.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:49:30.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:30.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:49:30.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:49:30.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:30.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:49:30.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:49:30.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-04T08:49:30.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: monmap e3: 3 mons at {a=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],b=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0],c=[v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T08:49:30.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: fsmap 2024-03-04T08:49:30.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: osdmap e4: 0 total, 0 up, 0 in 2024-03-04T08:49:30.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: mgrmap e13: x(active, since 67s) 2024-03-04T08:49:30.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-04T08:49:30.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-04T08:49:30.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-04T08:49:30.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: mon.b (rank 2) addr [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] is down (out of quorum) 2024-03-04T08:49:30.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:30.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:30.174 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:30.174 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:29 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:30.216 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:30.220+0000 7f6b44a26700 1 -- 172.21.15.158:0/813607965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6b401a2af0 con 0x7f6b2c0057d0 2024-03-04T08:49:30.217 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:30.217 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T08:49:30.217 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: mon.c calling monitor election 2024-03-04T08:49:30.217 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: mon.a calling monitor election 2024-03-04T08:49:30.217 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:49:30.217 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:49:30.218 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:49:30.218 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:30.218 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:49:30.218 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:49:30.218 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:30.218 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:49:30.218 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:49:30.218 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-04T08:49:30.218 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: monmap e3: 3 mons at {a=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],b=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0],c=[v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T08:49:30.218 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: fsmap 2024-03-04T08:49:30.218 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: osdmap e4: 0 total, 0 up, 0 in 2024-03-04T08:49:30.218 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: mgrmap e13: x(active, since 67s) 2024-03-04T08:49:30.219 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-04T08:49:30.219 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-04T08:49:30.219 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-04T08:49:30.219 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: mon.b (rank 2) addr [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] is down (out of quorum) 2024-03-04T08:49:30.219 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:30.219 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:30.219 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:30.219 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:29 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:30.219 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:30.221+0000 7f6b377fe700 1 -- 172.21.15.158:0/813607965 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1283 (secure 0 0 0) 0x7f6b30038520 con 0x7f6b2c0057d0 2024-03-04T08:49:30.220 INFO:teuthology.orchestra.run.smithi158.stdout: 2024-03-04T08:49:30.220 INFO:teuthology.orchestra.run.smithi158.stdout:{"epoch":3,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","modified":"2024-03-04T08:49:24.828036Z","created":"2024-03-04T08:47:22.873299Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:3300","nonce":0},{"type":"v1","addr":"172.21.15.158:6789","nonce":0}]},"addr":"172.21.15.158:6789/0","public_addr":"172.21.15.158:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:3300","nonce":0},{"type":"v1","addr":"172.21.15.150:6789","nonce":0}]},"addr":"172.21.15.150:6789/0","public_addr":"172.21.15.150:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-04T08:49:30.221 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:30.223+0000 7f6b44a26700 1 -- 172.21.15.158:0/813607965 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6b200390d0 msgr2=0x7f6b2003b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:30.221 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:30.223+0000 7f6b44a26700 1 --2- 172.21.15.158:0/813607965 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6b200390d0 0x7f6b2003b590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6b28009850 tx=0x7f6b28008040 comp rx=0 tx=0).stop 2024-03-04T08:49:30.221 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:30.223+0000 7f6b44a26700 1 -- 172.21.15.158:0/813607965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b2c0057d0 msgr2=0x7f6b40195600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:30.221 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:30.223+0000 7f6b44a26700 1 --2- 172.21.15.158:0/813607965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b2c0057d0 0x7f6b40195600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6b300199d0 tx=0x7f6b30019ab0 comp rx=0 tx=0).stop 2024-03-04T08:49:30.221 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:30.223+0000 7f6b44a26700 1 -- 172.21.15.158:0/813607965 shutdown_connections 2024-03-04T08:49:30.221 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:30.223+0000 7f6b44a26700 1 --2- 172.21.15.158:0/813607965 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6b200390d0 0x7f6b2003b590 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:30.221 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:30.223+0000 7f6b44a26700 1 --2- 172.21.15.158:0/813607965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b4019a3c0 0x7f6b4019a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:30.221 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:30.223+0000 7f6b44a26700 1 --2- 172.21.15.158:0/813607965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b2c0057d0 0x7f6b40195600 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:30.221 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:30.223+0000 7f6b44a26700 1 --2- 172.21.15.158:0/813607965 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6b40195b40 0x7f6b40199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:30.222 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:30.223+0000 7f6b44a26700 1 -- 172.21.15.158:0/813607965 >> 172.21.15.158:0/813607965 conn(0x7f6b400fc5d0 msgr2=0x7f6b400fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:49:30.222 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:30.223+0000 7f6b44a26700 1 -- 172.21.15.158:0/813607965 shutdown_connections 2024-03-04T08:49:30.222 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:30.223+0000 7f6b44a26700 1 -- 172.21.15.158:0/813607965 wait complete. 2024-03-04T08:49:30.222 INFO:teuthology.orchestra.run.smithi158.stderr:dumped monmap epoch 3 2024-03-04T08:49:30.827 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-04T08:49:30.827 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph config generate-minimal-conf 2024-03-04T08:49:31.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:30 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:49:31.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:30 smithi158 ceph-mon[37241]: from='client.? 172.21.15.158:0/813607965' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T08:49:31.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:30 smithi158 ceph-mon[37241]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:31.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:30 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:49:31.060 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:49:31.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:30 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:49:31.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:30 smithi071 ceph-mon[32789]: from='client.? 172.21.15.158:0/813607965' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T08:49:31.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:30 smithi071 ceph-mon[32789]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:31.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:30 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:49:32.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:32.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T08:49:32.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: mon.c calling monitor election 2024-03-04T08:49:32.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: mon.a calling monitor election 2024-03-04T08:49:32.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:49:32.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:49:32.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:49:32.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:32.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:49:32.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:49:32.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:32.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:49:32.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:49:32.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-04T08:49:32.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: monmap e3: 3 mons at {a=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],b=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0],c=[v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T08:49:32.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: fsmap 2024-03-04T08:49:32.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: osdmap e4: 0 total, 0 up, 0 in 2024-03-04T08:49:32.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: mgrmap e13: x(active, since 67s) 2024-03-04T08:49:32.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-04T08:49:32.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-04T08:49:32.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-04T08:49:32.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: mon.b (rank 2) addr [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] is down (out of quorum) 2024-03-04T08:49:32.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:32.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:32.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:32.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:32.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:49:32.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: from='client.? 172.21.15.158:0/813607965' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T08:49:32.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:32.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:31 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:49:32.733 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: mon.b calling monitor election 2024-03-04T08:49:32.733 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: mon.b calling monitor election 2024-03-04T08:49:32.733 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: mon.c calling monitor election 2024-03-04T08:49:32.733 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: mon.a calling monitor election 2024-03-04T08:49:32.733 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-04T08:49:32.733 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: monmap e3: 3 mons at {a=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],b=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0],c=[v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T08:49:32.733 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: fsmap 2024-03-04T08:49:32.733 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: osdmap e4: 0 total, 0 up, 0 in 2024-03-04T08:49:32.734 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: mgrmap e13: x(active, since 69s) 2024-03-04T08:49:32.734 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-04T08:49:32.734 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: Cluster is now healthy 2024-03-04T08:49:32.734 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: overall HEALTH_OK 2024-03-04T08:49:32.734 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:32.734 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:32.734 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:32.734 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:32.734 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:32.734 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:32.734 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:32.734 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:32.734 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:32.734 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:32.735 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:32.735 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T08:49:32.735 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T08:49:32.735 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:32.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.762+0000 7fe14b068700 1 -- 172.21.15.71:0/3264257349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe144100380 msgr2=0x7fe144100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:32.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.762+0000 7fe14b068700 1 --2- 172.21.15.71:0/3264257349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe144100380 0x7fe144100760 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe13800aa20 tx=0x7fe13800ad30 comp rx=0 tx=0).stop 2024-03-04T08:49:32.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.762+0000 7fe14b068700 1 -- 172.21.15.71:0/3264257349 shutdown_connections 2024-03-04T08:49:32.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.762+0000 7fe14b068700 1 --2- 172.21.15.71:0/3264257349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe144100380 0x7fe144100760 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:32.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.762+0000 7fe14b068700 1 -- 172.21.15.71:0/3264257349 >> 172.21.15.71:0/3264257349 conn(0x7fe1440fbc10 msgr2=0x7fe1440fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:49:32.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.762+0000 7fe14b068700 1 -- 172.21.15.71:0/3264257349 shutdown_connections 2024-03-04T08:49:32.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.762+0000 7fe14b068700 1 -- 172.21.15.71:0/3264257349 wait complete. 2024-03-04T08:49:32.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.762+0000 7fe14b068700 1 Processor -- start 2024-03-04T08:49:32.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.762+0000 7fe14b068700 1 -- start start 2024-03-04T08:49:32.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.762+0000 7fe14b068700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe144100380 0x7fe14419d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:49:32.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.763+0000 7fe14b068700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe14419db10 0x7fe1441976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:49:32.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.763+0000 7fe14b068700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe144197be0 0x7fe144198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:49:32.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.763+0000 7fe14b068700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe144109e70 con 0x7fe144100380 2024-03-04T08:49:32.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.763+0000 7fe14b068700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe144109cf0 con 0x7fe14419db10 2024-03-04T08:49:32.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.763+0000 7fe14b068700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe144109ff0 con 0x7fe144197be0 2024-03-04T08:49:32.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.763+0000 7fe148e04700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe144100380 0x7fe14419d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:49:32.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.763+0000 7fe148e04700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe144100380 0x7fe14419d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54740/0 (socket says 172.21.15.71:54740) 2024-03-04T08:49:32.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.763+0000 7fe148e04700 1 -- 172.21.15.71:0/1303504277 learned_addr learned my addr 172.21.15.71:0/1303504277 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:49:32.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.763+0000 7fe143fff700 1 --2- 172.21.15.71:0/1303504277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe14419db10 0x7fe1441976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:49:32.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.763+0000 7fe149605700 1 --2- 172.21.15.71:0/1303504277 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe144197be0 0x7fe144198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:49:32.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.763+0000 7fe148e04700 1 -- 172.21.15.71:0/1303504277 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe144197be0 msgr2=0x7fe144198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:32.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.763+0000 7fe148e04700 1 --2- 172.21.15.71:0/1303504277 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe144197be0 0x7fe144198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:32.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.763+0000 7fe148e04700 1 -- 172.21.15.71:0/1303504277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe14419db10 msgr2=0x7fe1441976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:32.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.763+0000 7fe148e04700 1 --2- 172.21.15.71:0/1303504277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe14419db10 0x7fe1441976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:32.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.763+0000 7fe148e04700 1 -- 172.21.15.71:0/1303504277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1441a4380 con 0x7fe144100380 2024-03-04T08:49:32.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.763+0000 7fe148e04700 1 --2- 172.21.15.71:0/1303504277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe144100380 0x7fe14419d5d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe138005e00 tx=0x7fe13800cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:49:32.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.763+0000 7fe141ffb700 1 -- 172.21.15.71:0/1303504277 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138025070 con 0x7fe144100380 2024-03-04T08:49:32.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.763+0000 7fe141ffb700 1 -- 172.21.15.71:0/1303504277 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1380054a0 con 0x7fe144100380 2024-03-04T08:49:32.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.763+0000 7fe14b068700 1 -- 172.21.15.71:0/1303504277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1441a4610 con 0x7fe144100380 2024-03-04T08:49:32.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.763+0000 7fe141ffb700 1 -- 172.21.15.71:0/1303504277 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13800e6b0 con 0x7fe144100380 2024-03-04T08:49:32.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.764+0000 7fe14b068700 1 -- 172.21.15.71:0/1303504277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1441a4b50 con 0x7fe144100380 2024-03-04T08:49:32.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.764+0000 7fe141ffb700 1 -- 172.21.15.71:0/1303504277 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7fe138028070 con 0x7fe144100380 2024-03-04T08:49:32.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.764+0000 7fe14b068700 1 -- 172.21.15.71:0/1303504277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe144101860 con 0x7fe144100380 2024-03-04T08:49:32.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.765+0000 7fe141ffb700 1 --2- 172.21.15.71:0/1303504277 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe1340390d0 0x7fe13403b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:49:32.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.765+0000 7fe141ffb700 1 -- 172.21.15.71:0/1303504277 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe13804c770 con 0x7fe144100380 2024-03-04T08:49:32.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.765+0000 7fe143fff700 1 --2- 172.21.15.71:0/1303504277 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe1340390d0 0x7fe13403b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:49:32.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.765+0000 7fe143fff700 1 --2- 172.21.15.71:0/1303504277 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe1340390d0 0x7fe13403b590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe130009ad0 tx=0x7fe130009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:49:32.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.770+0000 7fe141ffb700 1 -- 172.21.15.71:0/1303504277 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe138005650 con 0x7fe144100380 2024-03-04T08:49:32.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.970+0000 7fe14b068700 1 -- 172.21.15.71:0/1303504277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fe1441a4db0 con 0x7fe144100380 2024-03-04T08:49:32.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.970+0000 7fe141ffb700 1 -- 172.21.15.71:0/1303504277 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7fe13802f020 con 0x7fe144100380 2024-03-04T08:49:32.967 INFO:teuthology.orchestra.run.smithi071.stdout:# minimal ceph.conf for b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T08:49:32.967 INFO:teuthology.orchestra.run.smithi071.stdout:[global] 2024-03-04T08:49:32.967 INFO:teuthology.orchestra.run.smithi071.stdout: fsid = b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T08:49:32.968 INFO:teuthology.orchestra.run.smithi071.stdout: mon_host = [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] 2024-03-04T08:49:32.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.972+0000 7fe14b068700 1 -- 172.21.15.71:0/1303504277 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe1340390d0 msgr2=0x7fe13403b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:32.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.972+0000 7fe14b068700 1 --2- 172.21.15.71:0/1303504277 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe1340390d0 0x7fe13403b590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe130009ad0 tx=0x7fe130009210 comp rx=0 tx=0).stop 2024-03-04T08:49:32.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.972+0000 7fe14b068700 1 -- 172.21.15.71:0/1303504277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe144100380 msgr2=0x7fe14419d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:32.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.972+0000 7fe14b068700 1 --2- 172.21.15.71:0/1303504277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe144100380 0x7fe14419d5d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe138005e00 tx=0x7fe13800cd10 comp rx=0 tx=0).stop 2024-03-04T08:49:32.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.973+0000 7fe14b068700 1 -- 172.21.15.71:0/1303504277 shutdown_connections 2024-03-04T08:49:32.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.973+0000 7fe14b068700 1 --2- 172.21.15.71:0/1303504277 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe1340390d0 0x7fe13403b590 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:32.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.973+0000 7fe14b068700 1 --2- 172.21.15.71:0/1303504277 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe144197be0 0x7fe144198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:32.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.973+0000 7fe14b068700 1 --2- 172.21.15.71:0/1303504277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe144100380 0x7fe14419d5d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:32.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.973+0000 7fe14b068700 1 --2- 172.21.15.71:0/1303504277 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe14419db10 0x7fe1441976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:32.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.973+0000 7fe14b068700 1 -- 172.21.15.71:0/1303504277 >> 172.21.15.71:0/1303504277 conn(0x7fe1440fbc10 msgr2=0x7fe1441080d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:49:32.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.973+0000 7fe14b068700 1 -- 172.21.15.71:0/1303504277 shutdown_connections 2024-03-04T08:49:32.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:32.973+0000 7fe14b068700 1 -- 172.21.15.71:0/1303504277 wait complete. 2024-03-04T08:49:33.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: mon.b calling monitor election 2024-03-04T08:49:33.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: mon.b calling monitor election 2024-03-04T08:49:33.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: mon.c calling monitor election 2024-03-04T08:49:33.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: mon.a calling monitor election 2024-03-04T08:49:33.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-04T08:49:33.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: monmap e3: 3 mons at {a=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],b=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0],c=[v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T08:49:33.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: fsmap 2024-03-04T08:49:33.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: osdmap e4: 0 total, 0 up, 0 in 2024-03-04T08:49:33.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: mgrmap e13: x(active, since 69s) 2024-03-04T08:49:33.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-04T08:49:33.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: Cluster is now healthy 2024-03-04T08:49:33.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: overall HEALTH_OK 2024-03-04T08:49:33.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:33.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:33.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:33.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:33.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:33.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:33.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:33.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:33.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:33.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:33.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:33.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T08:49:33.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T08:49:33.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:33.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: mon.b calling monitor election 2024-03-04T08:49:33.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: mon.b calling monitor election 2024-03-04T08:49:33.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: mon.c calling monitor election 2024-03-04T08:49:33.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: mon.a calling monitor election 2024-03-04T08:49:33.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-04T08:49:33.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: monmap e3: 3 mons at {a=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],b=[v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0],c=[v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T08:49:33.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: fsmap 2024-03-04T08:49:33.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: osdmap e4: 0 total, 0 up, 0 in 2024-03-04T08:49:33.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: mgrmap e13: x(active, since 69s) 2024-03-04T08:49:33.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-04T08:49:33.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: Cluster is now healthy 2024-03-04T08:49:33.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: overall HEALTH_OK 2024-03-04T08:49:33.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:33.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:33.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:33.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:33.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:33.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:33.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:33.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:33.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:33.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:33.055 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:33.055 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T08:49:33.055 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T08:49:33.055 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:33.608 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-04T08:49:33.609 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-04T08:49:33.609 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.conf 2024-03-04T08:49:33.640 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-04T08:49:33.640 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-04T08:49:33.709 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-03-04T08:49:33.709 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.conf 2024-03-04T08:49:33.739 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-03-04T08:49:33.739 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-04T08:49:33.806 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2024-03-04T08:49:33.806 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/etc/ceph/ceph.conf 2024-03-04T08:49:33.836 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2024-03-04T08:49:33.837 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-04T08:49:33.903 INFO:tasks.cephadm:Adding mgr.x on smithi071 2024-03-04T08:49:33.903 INFO:tasks.cephadm:Adding mgr.y on smithi150 2024-03-04T08:49:33.904 INFO:tasks.cephadm:Adding mgr.z on smithi158 2024-03-04T08:49:33.904 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph orch apply mgr '3;smithi071=x;smithi150=y;smithi158=z' 2024-03-04T08:49:33.907 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:33 smithi071 ceph-mon[32789]: Reconfiguring mon.a (unknown last config time)... 2024-03-04T08:49:33.907 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:33 smithi071 ceph-mon[32789]: Reconfiguring daemon mon.a on smithi071 2024-03-04T08:49:33.907 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:33 smithi071 ceph-mon[32789]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:33.907 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:33 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:33.908 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:33 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:49:33.908 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:33 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1303504277' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:33.967 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:33 smithi158 ceph-mon[37241]: Reconfiguring mon.a (unknown last config time)... 2024-03-04T08:49:33.967 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:33 smithi158 ceph-mon[37241]: Reconfiguring daemon mon.a on smithi071 2024-03-04T08:49:33.967 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:33 smithi158 ceph-mon[37241]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:33.967 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:33 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:33.968 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:33 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:49:33.968 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:33 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1303504277' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:34.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:33 smithi150 ceph-mon[36807]: Reconfiguring mon.a (unknown last config time)... 2024-03-04T08:49:34.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:33 smithi150 ceph-mon[36807]: Reconfiguring daemon mon.a on smithi071 2024-03-04T08:49:34.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:33 smithi150 ceph-mon[36807]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:34.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:33 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:34.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:33 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:49:34.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:33 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1303504277' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:34.174 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T08:49:35.621 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.625+0000 7f482aeb1700 1 -- 172.21.15.158:0/792122099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4824103300 msgr2=0x7f4824103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:35.621 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.625+0000 7f482aeb1700 1 --2- 172.21.15.158:0/792122099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4824103300 0x7f4824103720 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4818009910 tx=0x7f4818009c20 comp rx=0 tx=0).stop 2024-03-04T08:49:35.621 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.625+0000 7f482aeb1700 1 -- 172.21.15.158:0/792122099 shutdown_connections 2024-03-04T08:49:35.621 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.625+0000 7f482aeb1700 1 --2- 172.21.15.158:0/792122099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4824103300 0x7f4824103720 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:35.622 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.625+0000 7f482aeb1700 1 -- 172.21.15.158:0/792122099 >> 172.21.15.158:0/792122099 conn(0x7f48240fd1c0 msgr2=0x7f48240ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:49:35.622 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.625+0000 7f482aeb1700 1 -- 172.21.15.158:0/792122099 shutdown_connections 2024-03-04T08:49:35.622 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.625+0000 7f482aeb1700 1 -- 172.21.15.158:0/792122099 wait complete. 2024-03-04T08:49:35.624 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.626+0000 7f482aeb1700 1 Processor -- start 2024-03-04T08:49:35.624 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.626+0000 7f482aeb1700 1 -- start start 2024-03-04T08:49:35.624 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.626+0000 7f482aeb1700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4824103300 0x7f4824197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:49:35.624 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.626+0000 7f482aeb1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4824197a90 0x7f482419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:49:35.624 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.626+0000 7f482aeb1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4824197f10 0x7f482419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:49:35.624 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.626+0000 7f482aeb1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4824107d70 con 0x7f4824197a90 2024-03-04T08:49:35.625 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.626+0000 7f482aeb1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4824107bf0 con 0x7f4824197f10 2024-03-04T08:49:35.625 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.626+0000 7f482aeb1700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4824107ef0 con 0x7f4824103300 2024-03-04T08:49:35.625 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.626+0000 7f4828c4d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4824103300 0x7f4824197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:49:35.625 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.626+0000 7f4828c4d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4824103300 0x7f4824197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.158:54976/0 (socket says 172.21.15.158:54976) 2024-03-04T08:49:35.625 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.626+0000 7f4828c4d700 1 -- 172.21.15.158:0/69263034 learned_addr learned my addr 172.21.15.158:0/69263034 (peer_addr_for_me v2:172.21.15.158:0/0) 2024-03-04T08:49:35.625 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.626+0000 7f482944e700 1 --2- 172.21.15.158:0/69263034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4824197f10 0x7f482419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:49:35.625 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.626+0000 7f4823fff700 1 --2- 172.21.15.158:0/69263034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4824197a90 0x7f482419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:49:35.625 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.626+0000 7f4828c4d700 1 -- 172.21.15.158:0/69263034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4824197f10 msgr2=0x7f482419f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:35.625 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.626+0000 7f4828c4d700 1 --2- 172.21.15.158:0/69263034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4824197f10 0x7f482419f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:35.625 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.626+0000 7f4828c4d700 1 -- 172.21.15.158:0/69263034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4824197a90 msgr2=0x7f482419cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:35.625 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.626+0000 7f4828c4d700 1 --2- 172.21.15.158:0/69263034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4824197a90 0x7f482419cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:35.625 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.626+0000 7f4828c4d700 1 -- 172.21.15.158:0/69263034 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482419f960 con 0x7f4824103300 2024-03-04T08:49:35.625 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.626+0000 7f482944e700 1 --2- 172.21.15.158:0/69263034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4824197f10 0x7f482419f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:49:35.626 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.626+0000 7f4828c4d700 1 --2- 172.21.15.158:0/69263034 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4824103300 0x7f4824197550 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4818000c00 tx=0x7f48180050b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:49:35.626 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.627+0000 7f4821ffb700 1 -- 172.21.15.158:0/69263034 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4818027070 con 0x7f4824103300 2024-03-04T08:49:35.626 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.627+0000 7f482aeb1700 1 -- 172.21.15.158:0/69263034 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f482419fbf0 con 0x7f4824103300 2024-03-04T08:49:35.626 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.627+0000 7f4821ffb700 1 -- 172.21.15.158:0/69263034 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4818022470 con 0x7f4824103300 2024-03-04T08:49:35.626 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.627+0000 7f4821ffb700 1 -- 172.21.15.158:0/69263034 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481800f6c0 con 0x7f4824103300 2024-03-04T08:49:35.626 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.627+0000 7f482aeb1700 1 -- 172.21.15.158:0/69263034 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48241a0120 con 0x7f4824103300 2024-03-04T08:49:35.626 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.628+0000 7f4821ffb700 1 -- 172.21.15.158:0/69263034 <== mon.1 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f481802a070 con 0x7f4824103300 2024-03-04T08:49:35.626 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.628+0000 7f4821ffb700 1 --2- 172.21.15.158:0/69263034 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f48140390d0 0x7f481403b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:49:35.626 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.628+0000 7f4821ffb700 1 -- 172.21.15.158:0/69263034 <== mon.1 v2:172.21.15.158:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f481804ddc0 con 0x7f4824103300 2024-03-04T08:49:35.626 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.628+0000 7f482aeb1700 1 -- 172.21.15.158:0/69263034 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48240636f0 con 0x7f4824103300 2024-03-04T08:49:35.627 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.628+0000 7f4823fff700 1 --2- 172.21.15.158:0/69263034 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f48140390d0 0x7f481403b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:49:35.627 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.629+0000 7f4823fff700 1 --2- 172.21.15.158:0/69263034 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f48140390d0 0x7f481403b590 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4810009850 tx=0x7f4810008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:49:35.629 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.633+0000 7f4821ffb700 1 -- 172.21.15.158:0/69263034 <== mon.1 v2:172.21.15.158:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4818013070 con 0x7f4824103300 2024-03-04T08:49:35.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:35 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:35.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:35 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:35.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:35 smithi150 ceph-mon[36807]: Reconfiguring mon.b (monmap changed)... 2024-03-04T08:49:35.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:35 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T08:49:35.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:35 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T08:49:35.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:35 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:35.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:35 smithi150 ceph-mon[36807]: Reconfiguring daemon mon.b on smithi150 2024-03-04T08:49:35.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:35 smithi150 ceph-mon[36807]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:35.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:35 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:35.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:35 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:35.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:35 smithi158 ceph-mon[37241]: Reconfiguring mon.b (monmap changed)... 2024-03-04T08:49:35.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:35 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T08:49:35.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:35 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T08:49:35.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:35 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:35.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:35 smithi158 ceph-mon[37241]: Reconfiguring daemon mon.b on smithi150 2024-03-04T08:49:35.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:35 smithi158 ceph-mon[37241]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:35.835 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.839+0000 7f482aeb1700 1 -- 172.21.15.158:0/69263034 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi071=x;smithi150=y;smithi158=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f4824062500 con 0x7f48140390d0 2024-03-04T08:49:35.841 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.845+0000 7f4821ffb700 1 -- 172.21.15.158:0/69263034 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4824062500 con 0x7f48140390d0 2024-03-04T08:49:35.841 INFO:teuthology.orchestra.run.smithi158.stdout:Scheduled mgr update... 2024-03-04T08:49:35.844 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.847+0000 7f482aeb1700 1 -- 172.21.15.158:0/69263034 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f48140390d0 msgr2=0x7f481403b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:35.844 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.847+0000 7f482aeb1700 1 --2- 172.21.15.158:0/69263034 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f48140390d0 0x7f481403b590 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4810009850 tx=0x7f4810008040 comp rx=0 tx=0).stop 2024-03-04T08:49:35.844 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.847+0000 7f482aeb1700 1 -- 172.21.15.158:0/69263034 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4824103300 msgr2=0x7f4824197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:35.844 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.847+0000 7f482aeb1700 1 --2- 172.21.15.158:0/69263034 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4824103300 0x7f4824197550 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4818000c00 tx=0x7f48180050b0 comp rx=0 tx=0).stop 2024-03-04T08:49:35.844 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.847+0000 7f482aeb1700 1 -- 172.21.15.158:0/69263034 shutdown_connections 2024-03-04T08:49:35.844 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.847+0000 7f482aeb1700 1 --2- 172.21.15.158:0/69263034 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f48140390d0 0x7f481403b590 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:35.844 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.847+0000 7f482aeb1700 1 --2- 172.21.15.158:0/69263034 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4824103300 0x7f4824197550 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:35.844 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.847+0000 7f482aeb1700 1 --2- 172.21.15.158:0/69263034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4824197f10 0x7f482419f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:35.844 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.847+0000 7f482aeb1700 1 --2- 172.21.15.158:0/69263034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4824197a90 0x7f482419cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:35.844 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.847+0000 7f482aeb1700 1 -- 172.21.15.158:0/69263034 >> 172.21.15.158:0/69263034 conn(0x7f48240fd1c0 msgr2=0x7f48240fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:49:35.844 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.847+0000 7f482aeb1700 1 -- 172.21.15.158:0/69263034 shutdown_connections 2024-03-04T08:49:35.845 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:49:35.847+0000 7f482aeb1700 1 -- 172.21.15.158:0/69263034 wait complete. 2024-03-04T08:49:35.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:35 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:35.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:35 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:35.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:35 smithi071 ceph-mon[32789]: Reconfiguring mon.b (monmap changed)... 2024-03-04T08:49:35.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:35 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T08:49:35.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:35 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T08:49:35.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:35 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:35.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:35 smithi071 ceph-mon[32789]: Reconfiguring daemon mon.b on smithi150 2024-03-04T08:49:35.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:35 smithi071 ceph-mon[32789]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:36.450 DEBUG:teuthology.orchestra.run.smithi150:mgr.y> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mgr.y.service 2024-03-04T08:49:36.453 DEBUG:teuthology.orchestra.run.smithi158:mgr.z> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mgr.z.service 2024-03-04T08:49:36.493 INFO:tasks.cephadm:Deploying OSDs... 2024-03-04T08:49:36.493 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-04T08:49:36.493 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/scratch_devs of=/dev/stdout 2024-03-04T08:49:36.513 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-04T08:49:36.513 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_1 2024-03-04T08:49:36.570 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-04T08:49:36.571 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T08:49:36.571 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-03-04T08:49:36.571 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T08:49:36.571 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T08:49:36.571 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-03-04 08:48:35.355556207 +0000 2024-03-04T08:49:36.571 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-03-04 08:43:38.858798536 +0000 2024-03-04T08:49:36.571 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-03-04 08:43:38.858798536 +0000 2024-03-04T08:49:36.571 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-03-04 08:43:38.858798536 +0000 2024-03-04T08:49:36.571 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-04T08:49:36.642 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-03-04T08:49:36.642 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-03-04T08:49:36.642 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000230302 s, 2.2 MB/s 2024-03-04T08:49:36.643 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-04T08:49:36.701 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_2 2024-03-04T08:49:36.758 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-04T08:49:36.758 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T08:49:36.758 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-04T08:49:36.758 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T08:49:36.758 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T08:49:36.758 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-03-04 08:48:35.389555608 +0000 2024-03-04T08:49:36.758 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-03-04 08:43:39.198792573 +0000 2024-03-04T08:49:36.758 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-03-04 08:43:39.198792573 +0000 2024-03-04T08:49:36.758 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-03-04 08:43:39.198792573 +0000 2024-03-04T08:49:36.758 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-04T08:49:36.827 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-03-04T08:49:36.827 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-03-04T08:49:36.827 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.00021731 s, 2.4 MB/s 2024-03-04T08:49:36.828 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-04T08:49:36.886 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_3 2024-03-04T08:49:36.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:36 smithi071 ceph-mon[32789]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi071=x;smithi150=y;smithi158=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:49:36.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:36 smithi071 ceph-mon[32789]: Saving service mgr spec with placement smithi071=x;smithi150=y;smithi158=z;count:3 2024-03-04T08:49:36.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:36 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:36.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:36 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:36.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:36 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:36.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:36 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T08:49:36.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:36 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T08:49:36.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:36 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:36.937 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-04T08:49:36.937 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T08:49:36.937 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-03-04T08:49:36.937 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T08:49:36.938 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T08:49:36.938 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-03-04 08:48:35.421555043 +0000 2024-03-04T08:49:36.938 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-03-04 08:43:39.545786487 +0000 2024-03-04T08:49:36.938 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-03-04 08:43:39.545786487 +0000 2024-03-04T08:49:36.938 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-03-04 08:43:39.545786487 +0000 2024-03-04T08:49:36.938 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-04T08:49:37.004 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-03-04T08:49:37.005 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-03-04T08:49:37.005 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000171728 s, 3.0 MB/s 2024-03-04T08:49:37.005 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-04T08:49:37.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:36 smithi158 ceph-mon[37241]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi071=x;smithi150=y;smithi158=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:49:37.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:36 smithi158 ceph-mon[37241]: Saving service mgr spec with placement smithi071=x;smithi150=y;smithi158=z;count:3 2024-03-04T08:49:37.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:36 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:37.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:36 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:37.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:36 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:37.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:36 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T08:49:37.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:36 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T08:49:37.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:36 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:37.061 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_4 2024-03-04T08:49:37.116 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-04T08:49:37.116 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T08:49:37.116 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-03-04T08:49:37.117 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T08:49:37.117 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T08:49:37.117 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-03-04 08:48:35.453554478 +0000 2024-03-04T08:49:37.117 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-03-04 08:43:39.876780682 +0000 2024-03-04T08:49:37.117 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-03-04 08:43:39.876780682 +0000 2024-03-04T08:49:37.117 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-03-04 08:43:39.876780682 +0000 2024-03-04T08:49:37.117 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-04T08:49:37.183 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-03-04T08:49:37.183 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-03-04T08:49:37.183 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000184652 s, 2.8 MB/s 2024-03-04T08:49:37.184 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-04T08:49:37.240 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-03-04T08:49:37.240 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2024-03-04T08:49:37.259 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-04T08:49:37.259 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2024-03-04T08:49:37.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:36 smithi150 ceph-mon[36807]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi071=x;smithi150=y;smithi158=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:49:37.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:36 smithi150 ceph-mon[36807]: Saving service mgr spec with placement smithi071=x;smithi150=y;smithi158=z;count:3 2024-03-04T08:49:37.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:36 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:37.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:36 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:37.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:36 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:37.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:36 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T08:49:37.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:36 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T08:49:37.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:36 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:37.297 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-04T08:49:37.297 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T08:49:37.297 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-03-04T08:49:37.297 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T08:49:37.297 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T08:49:37.297 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-03-04 08:49:24.287944727 +0000 2024-03-04T08:49:37.297 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-03-04 08:42:47.584256545 +0000 2024-03-04T08:49:37.297 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-03-04 08:42:47.584256545 +0000 2024-03-04T08:49:37.297 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-03-04 08:42:47.584256545 +0000 2024-03-04T08:49:37.298 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-04T08:49:37.366 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-03-04T08:49:37.366 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-03-04T08:49:37.366 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000230185 s, 2.2 MB/s 2024-03-04T08:49:37.368 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-04T08:49:37.429 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2024-03-04T08:49:37.485 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-04T08:49:37.486 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T08:49:37.486 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-03-04T08:49:37.486 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T08:49:37.486 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T08:49:37.486 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-03-04 08:49:24.288944708 +0000 2024-03-04T08:49:37.486 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-03-04 08:42:47.923249730 +0000 2024-03-04T08:49:37.486 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-03-04 08:42:47.923249730 +0000 2024-03-04T08:49:37.486 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-03-04 08:42:47.923249730 +0000 2024-03-04T08:49:37.486 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-04T08:49:37.554 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-03-04T08:49:37.554 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-03-04T08:49:37.554 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000203976 s, 2.5 MB/s 2024-03-04T08:49:37.555 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-04T08:49:37.616 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2024-03-04T08:49:37.673 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-04T08:49:37.673 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T08:49:37.673 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-03-04T08:49:37.673 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T08:49:37.673 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T08:49:37.673 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-03-04 08:49:24.288944708 +0000 2024-03-04T08:49:37.673 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-03-04 08:42:48.283242493 +0000 2024-03-04T08:49:37.673 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-03-04 08:42:48.283242493 +0000 2024-03-04T08:49:37.673 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-03-04 08:42:48.283242493 +0000 2024-03-04T08:49:37.674 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-04T08:49:37.742 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-03-04T08:49:37.742 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-03-04T08:49:37.743 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.00020885 s, 2.5 MB/s 2024-03-04T08:49:37.744 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-04T08:49:37.800 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2024-03-04T08:49:37.857 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-04T08:49:37.857 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T08:49:37.857 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-03-04T08:49:37.857 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T08:49:37.857 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T08:49:37.857 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-03-04 08:49:24.288944708 +0000 2024-03-04T08:49:37.857 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-03-04 08:42:48.640235316 +0000 2024-03-04T08:49:37.857 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-03-04 08:42:48.640235316 +0000 2024-03-04T08:49:37.858 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-03-04 08:42:48.640235316 +0000 2024-03-04T08:49:37.858 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-04T08:49:37.926 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-03-04T08:49:37.926 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-03-04T08:49:37.926 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.00025904 s, 2.0 MB/s 2024-03-04T08:49:37.927 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-04T08:49:37.986 DEBUG:teuthology.orchestra.run.smithi158:> set -ex 2024-03-04T08:49:37.986 DEBUG:teuthology.orchestra.run.smithi158:> dd if=/scratch_devs of=/dev/stdout 2024-03-04T08:49:38.002 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-04T08:49:38.002 DEBUG:teuthology.orchestra.run.smithi158:> stat /dev/vg_nvme/lv_1 2024-03-04T08:49:38.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:37 smithi150 ceph-mon[36807]: Reconfiguring mon.c (monmap changed)... 2024-03-04T08:49:38.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:37 smithi150 ceph-mon[36807]: Reconfiguring daemon mon.c on smithi158 2024-03-04T08:49:38.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:37 smithi150 ceph-mon[36807]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:38.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:37 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:38.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:37 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:38.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:37 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:49:38.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:37 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:38.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:37 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:49:38.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:37 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:38.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:37 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T08:49:38.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:37 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-04T08:49:38.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:37 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T08:49:38.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:37 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:38.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:37 smithi150 ceph-mon[36807]: Deploying daemon mgr.z on smithi158 2024-03-04T08:49:38.058 INFO:teuthology.orchestra.run.smithi158.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-04T08:49:38.058 INFO:teuthology.orchestra.run.smithi158.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T08:49:38.058 INFO:teuthology.orchestra.run.smithi158.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-03-04T08:49:38.059 INFO:teuthology.orchestra.run.smithi158.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T08:49:38.059 INFO:teuthology.orchestra.run.smithi158.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T08:49:38.059 INFO:teuthology.orchestra.run.smithi158.stdout:Access: 2024-03-04 08:49:14.527601133 +0000 2024-03-04T08:49:38.059 INFO:teuthology.orchestra.run.smithi158.stdout:Modify: 2024-03-04 08:43:09.882563880 +0000 2024-03-04T08:49:38.059 INFO:teuthology.orchestra.run.smithi158.stdout:Change: 2024-03-04 08:43:09.882563880 +0000 2024-03-04T08:49:38.059 INFO:teuthology.orchestra.run.smithi158.stdout: Birth: 2024-03-04 08:43:09.882563880 +0000 2024-03-04T08:49:38.059 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-04T08:49:38.126 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:37 smithi158 ceph-mon[37241]: Reconfiguring mon.c (monmap changed)... 2024-03-04T08:49:38.126 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:37 smithi158 ceph-mon[37241]: Reconfiguring daemon mon.c on smithi158 2024-03-04T08:49:38.127 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:37 smithi158 ceph-mon[37241]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:38.127 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:37 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:38.127 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:37 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:38.127 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:37 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:49:38.127 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:37 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:38.127 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:37 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:49:38.127 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:37 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:38.127 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:37 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T08:49:38.127 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:37 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-04T08:49:38.127 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:37 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T08:49:38.127 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:37 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:38.127 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:37 smithi158 ceph-mon[37241]: Deploying daemon mgr.z on smithi158 2024-03-04T08:49:38.129 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records in 2024-03-04T08:49:38.129 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records out 2024-03-04T08:49:38.129 INFO:teuthology.orchestra.run.smithi158.stderr:512 bytes copied, 0.000101916 s, 5.0 MB/s 2024-03-04T08:49:38.131 DEBUG:teuthology.orchestra.run.smithi158:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-04T08:49:38.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:37 smithi071 ceph-mon[32789]: Reconfiguring mon.c (monmap changed)... 2024-03-04T08:49:38.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:37 smithi071 ceph-mon[32789]: Reconfiguring daemon mon.c on smithi158 2024-03-04T08:49:38.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:37 smithi071 ceph-mon[32789]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:38.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:37 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:38.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:37 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:38.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:37 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:49:38.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:37 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:38.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:37 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:49:38.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:37 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:38.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:37 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T08:49:38.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:37 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-04T08:49:38.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:37 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T08:49:38.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:37 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:38.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:37 smithi071 ceph-mon[32789]: Deploying daemon mgr.z on smithi158 2024-03-04T08:49:38.187 DEBUG:teuthology.orchestra.run.smithi158:> stat /dev/vg_nvme/lv_2 2024-03-04T08:49:38.241 INFO:teuthology.orchestra.run.smithi158.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-04T08:49:38.242 INFO:teuthology.orchestra.run.smithi158.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T08:49:38.242 INFO:teuthology.orchestra.run.smithi158.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-03-04T08:49:38.242 INFO:teuthology.orchestra.run.smithi158.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T08:49:38.242 INFO:teuthology.orchestra.run.smithi158.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T08:49:38.242 INFO:teuthology.orchestra.run.smithi158.stdout:Access: 2024-03-04 08:49:14.528601122 +0000 2024-03-04T08:49:38.242 INFO:teuthology.orchestra.run.smithi158.stdout:Modify: 2024-03-04 08:43:10.214560289 +0000 2024-03-04T08:49:38.242 INFO:teuthology.orchestra.run.smithi158.stdout:Change: 2024-03-04 08:43:10.214560289 +0000 2024-03-04T08:49:38.242 INFO:teuthology.orchestra.run.smithi158.stdout: Birth: 2024-03-04 08:43:10.214560289 +0000 2024-03-04T08:49:38.242 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-04T08:49:38.307 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records in 2024-03-04T08:49:38.307 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records out 2024-03-04T08:49:38.307 INFO:teuthology.orchestra.run.smithi158.stderr:512 bytes copied, 9.2391e-05 s, 5.5 MB/s 2024-03-04T08:49:38.308 DEBUG:teuthology.orchestra.run.smithi158:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-04T08:49:38.364 DEBUG:teuthology.orchestra.run.smithi158:> stat /dev/vg_nvme/lv_3 2024-03-04T08:49:38.419 INFO:teuthology.orchestra.run.smithi158.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-04T08:49:38.420 INFO:teuthology.orchestra.run.smithi158.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T08:49:38.420 INFO:teuthology.orchestra.run.smithi158.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-03-04T08:49:38.420 INFO:teuthology.orchestra.run.smithi158.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T08:49:38.420 INFO:teuthology.orchestra.run.smithi158.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T08:49:38.420 INFO:teuthology.orchestra.run.smithi158.stdout:Access: 2024-03-04 08:49:14.528601122 +0000 2024-03-04T08:49:38.420 INFO:teuthology.orchestra.run.smithi158.stdout:Modify: 2024-03-04 08:43:10.543556730 +0000 2024-03-04T08:49:38.420 INFO:teuthology.orchestra.run.smithi158.stdout:Change: 2024-03-04 08:43:10.543556730 +0000 2024-03-04T08:49:38.421 INFO:teuthology.orchestra.run.smithi158.stdout: Birth: 2024-03-04 08:43:10.543556730 +0000 2024-03-04T08:49:38.421 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-04T08:49:38.485 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records in 2024-03-04T08:49:38.485 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records out 2024-03-04T08:49:38.485 INFO:teuthology.orchestra.run.smithi158.stderr:512 bytes copied, 8.6357e-05 s, 5.9 MB/s 2024-03-04T08:49:38.487 DEBUG:teuthology.orchestra.run.smithi158:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-04T08:49:38.542 DEBUG:teuthology.orchestra.run.smithi158:> stat /dev/vg_nvme/lv_4 2024-03-04T08:49:38.598 INFO:teuthology.orchestra.run.smithi158.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-04T08:49:38.598 INFO:teuthology.orchestra.run.smithi158.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T08:49:38.598 INFO:teuthology.orchestra.run.smithi158.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-03-04T08:49:38.598 INFO:teuthology.orchestra.run.smithi158.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T08:49:38.598 INFO:teuthology.orchestra.run.smithi158.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T08:49:38.598 INFO:teuthology.orchestra.run.smithi158.stdout:Access: 2024-03-04 08:49:14.529601111 +0000 2024-03-04T08:49:38.598 INFO:teuthology.orchestra.run.smithi158.stdout:Modify: 2024-03-04 08:43:10.877553117 +0000 2024-03-04T08:49:38.599 INFO:teuthology.orchestra.run.smithi158.stdout:Change: 2024-03-04 08:43:10.877553117 +0000 2024-03-04T08:49:38.599 INFO:teuthology.orchestra.run.smithi158.stdout: Birth: 2024-03-04 08:43:10.877553117 +0000 2024-03-04T08:49:38.599 DEBUG:teuthology.orchestra.run.smithi158:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-04T08:49:38.664 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records in 2024-03-04T08:49:38.664 INFO:teuthology.orchestra.run.smithi158.stderr:1+0 records out 2024-03-04T08:49:38.664 INFO:teuthology.orchestra.run.smithi158.stderr:512 bytes copied, 0.000104481 s, 4.9 MB/s 2024-03-04T08:49:38.666 DEBUG:teuthology.orchestra.run.smithi158:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-04T08:49:38.721 INFO:tasks.cephadm:Deploying osd.0 on smithi071 with /dev/vg_nvme/lv_4... 2024-03-04T08:49:38.721 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-04T08:49:38.971 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:49:39.092 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:38 smithi071 ceph-mon[32789]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:39.237 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:38 smithi158 ceph-mon[37241]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:39.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:38 smithi150 ceph-mon[36807]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:40.303 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:39 smithi158 podman[39093]: 2024-03-04 08:49:39.852224552 +0000 UTC m=+0.017812897 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T08:49:40.967 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-04T08:49:40.968 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-04T08:49:40.968 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-03-04T08:49:40.968 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-03-04T08:49:40.968 INFO:teuthology.orchestra.run.smithi071.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0128674 s, 815 MB/s 2024-03-04T08:49:40.968 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-03-04T08:49:41.534 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi071:vg_nvme/lv_4 2024-03-04T08:49:41.770 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:49:41.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:41 smithi071 ceph-mon[32789]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:42.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:41 smithi150 ceph-mon[36807]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:42.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:41 smithi158 ceph-mon[37241]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:43.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.274+0000 7f5f0f616700 1 -- 172.21.15.71:0/348189108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f081020e0 msgr2=0x7f5f08102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:43.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.274+0000 7f5f0f616700 1 --2- 172.21.15.71:0/348189108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f081020e0 0x7f5f08102500 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80099c0 tx=0x7f5ef8009cd0 comp rx=0 tx=0).stop 2024-03-04T08:49:43.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.274+0000 7f5f0f616700 1 -- 172.21.15.71:0/348189108 shutdown_connections 2024-03-04T08:49:43.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.274+0000 7f5f0f616700 1 --2- 172.21.15.71:0/348189108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f08103c80 0x7f5f0810a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:43.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.274+0000 7f5f0f616700 1 --2- 172.21.15.71:0/348189108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f081020e0 0x7f5f08102500 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:43.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.274+0000 7f5f0f616700 1 --2- 172.21.15.71:0/348189108 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5f081032e0 0x7f5f08103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:43.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.274+0000 7f5f0f616700 1 -- 172.21.15.71:0/348189108 >> 172.21.15.71:0/348189108 conn(0x7f5f080fd6a0 msgr2=0x7f5f080ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:49:43.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.274+0000 7f5f0f616700 1 -- 172.21.15.71:0/348189108 shutdown_connections 2024-03-04T08:49:43.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.274+0000 7f5f0f616700 1 -- 172.21.15.71:0/348189108 wait complete. 2024-03-04T08:49:43.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.275+0000 7f5f0f616700 1 Processor -- start 2024-03-04T08:49:43.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.275+0000 7f5f0f616700 1 -- start start 2024-03-04T08:49:43.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.275+0000 7f5f0f616700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f081020e0 0x7f5f08198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:49:43.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.275+0000 7f5f0f616700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5f081032e0 0x7f5f081989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:49:43.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.275+0000 7f5f0f616700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f08103c80 0x7f5f0819fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:49:43.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.275+0000 7f5f0f616700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0810dda0 con 0x7f5f08103c80 2024-03-04T08:49:43.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.275+0000 7f5f0f616700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0810dc20 con 0x7f5f081020e0 2024-03-04T08:49:43.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.275+0000 7f5f0f616700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0810df20 con 0x7f5f081032e0 2024-03-04T08:49:43.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.275+0000 7f5f0dbb3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f08103c80 0x7f5f0819fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:49:43.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.275+0000 7f5f0dbb3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f08103c80 0x7f5f0819fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35824/0 (socket says 172.21.15.71:35824) 2024-03-04T08:49:43.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.275+0000 7f5f0dbb3700 1 -- 172.21.15.71:0/1720266234 learned_addr learned my addr 172.21.15.71:0/1720266234 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:49:43.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.275+0000 7f5f0d3b2700 1 --2- 172.21.15.71:0/1720266234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f081020e0 0x7f5f08198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:49:43.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.275+0000 7f5f0cbb1700 1 --2- 172.21.15.71:0/1720266234 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5f081032e0 0x7f5f081989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:49:43.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.275+0000 7f5f0dbb3700 1 -- 172.21.15.71:0/1720266234 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5f081032e0 msgr2=0x7f5f081989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:43.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.275+0000 7f5f0dbb3700 1 --2- 172.21.15.71:0/1720266234 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5f081032e0 0x7f5f081989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:43.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.275+0000 7f5f0dbb3700 1 -- 172.21.15.71:0/1720266234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f081020e0 msgr2=0x7f5f08198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:49:43.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.275+0000 7f5f0dbb3700 1 --2- 172.21.15.71:0/1720266234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f081020e0 0x7f5f08198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:49:43.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.275+0000 7f5f0dbb3700 1 -- 172.21.15.71:0/1720266234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f081a00c0 con 0x7f5f08103c80 2024-03-04T08:49:43.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.276+0000 7f5f0dbb3700 1 --2- 172.21.15.71:0/1720266234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f08103c80 0x7f5f0819fa20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000b9a0 tx=0x7f5f0000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:49:43.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.276+0000 7f5efe7fc700 1 -- 172.21.15.71:0/1720266234 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f00019070 con 0x7f5f08103c80 2024-03-04T08:49:43.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.276+0000 7f5f0f616700 1 -- 172.21.15.71:0/1720266234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f081a03b0 con 0x7f5f08103c80 2024-03-04T08:49:43.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.276+0000 7f5efe7fc700 1 -- 172.21.15.71:0/1720266234 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f00014440 con 0x7f5f08103c80 2024-03-04T08:49:43.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.276+0000 7f5efe7fc700 1 -- 172.21.15.71:0/1720266234 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f000075d0 con 0x7f5f08103c80 2024-03-04T08:49:43.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.276+0000 7f5f0f616700 1 -- 172.21.15.71:0/1720266234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f081a0920 con 0x7f5f08103c80 2024-03-04T08:49:43.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.277+0000 7f5efe7fc700 1 -- 172.21.15.71:0/1720266234 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f5f0001c070 con 0x7f5f08103c80 2024-03-04T08:49:43.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.277+0000 7f5efe7fc700 1 --2- 172.21.15.71:0/1720266234 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f5ef4038e80 0x7f5ef403b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:49:43.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.277+0000 7f5f0f616700 1 -- 172.21.15.71:0/1720266234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f08068d80 con 0x7f5f08103c80 2024-03-04T08:49:43.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.277+0000 7f5efe7fc700 1 -- 172.21.15.71:0/1720266234 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5f0004b4f0 con 0x7f5f08103c80 2024-03-04T08:49:43.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.277+0000 7f5f0d3b2700 1 --2- 172.21.15.71:0/1720266234 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f5ef4038e80 0x7f5ef403b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:49:43.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.277+0000 7f5f0d3b2700 1 --2- 172.21.15.71:0/1720266234 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f5ef4038e80 0x7f5ef403b340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800b580 tx=0x7f5ef8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:49:43.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.282+0000 7f5efe7fc700 1 -- 172.21.15.71:0/1720266234 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5f000112a0 con 0x7f5f08103c80 2024-03-04T08:49:43.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:43.483+0000 7f5f0f616700 1 -- 172.21.15.71:0/1720266234 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f08062500 con 0x7f5ef4038e80 2024-03-04T08:49:43.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:43 smithi071 ceph-mon[32789]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:43.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:43 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:49:43.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:43 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:49:43.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:43 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:44.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:43 smithi150 ceph-mon[36807]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:44.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:43 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:49:44.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:43 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:49:44.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:43 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:44.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:43 smithi158 ceph-mon[37241]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:44.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:43 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:49:44.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:43 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:49:44.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:43 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:44.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:44 smithi071 ceph-mon[32789]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:49:45.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:44 smithi150 ceph-mon[36807]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:49:45.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:44 smithi158 ceph-mon[37241]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:49:48.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:47 smithi071 ceph-mon[32789]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:48.185 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:47 smithi158 ceph-mon[37241]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:48.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:47 smithi150 ceph-mon[36807]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:48.444 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:48 smithi158 podman[39093]: 2024-03-04 08:49:48.188020884 +0000 UTC m=+8.353609255 container create 4f4b699bb6120cc86b0423567dbe9d5fac28ceb3d1b2c2a982564f4fc041ce7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-04T08:49:48.445 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:48 smithi158 podman[39093]: 2024-03-04 08:49:48.415484216 +0000 UTC m=+8.581072548 container init 4f4b699bb6120cc86b0423567dbe9d5fac28ceb3d1b2c2a982564f4fc041ce7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-04T08:49:48.445 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:48 smithi158 podman[39093]: 2024-03-04 08:49:48.418360185 +0000 UTC m=+8.583948526 container start 4f4b699bb6120cc86b0423567dbe9d5fac28ceb3d1b2c2a982564f4fc041ce7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T08:49:48.802 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:48 smithi158 ceph-mgr[39112]: -- 172.21.15.158:0/2850850217 <== mon.1 v2:172.21.15.158:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555c2c4085a0 con 0x555c2c3dd000 2024-03-04T08:49:48.802 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:48 smithi158 bash[39093]: 4f4b699bb6120cc86b0423567dbe9d5fac28ceb3d1b2c2a982564f4fc041ce7f 2024-03-04T08:49:48.802 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:48 smithi158 systemd[1]: Started Ceph mgr.z for b3c85c40-da03-11ee-95c6-87774f69a715. 2024-03-04T08:49:49.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:48 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1639938755' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e43b47d-8888-4631-bc96-8a96fcf8cd46"}]: dispatch 2024-03-04T08:49:49.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:48 smithi071 ceph-mon[32789]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:49.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:48 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1639938755' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e43b47d-8888-4631-bc96-8a96fcf8cd46"}]': finished 2024-03-04T08:49:49.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:48 smithi071 ceph-mon[32789]: osdmap e5: 1 total, 0 up, 1 in 2024-03-04T08:49:49.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:48 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T08:49:49.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:48 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3637172688' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:49:49.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:48 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:49.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:48 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:49.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:48 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:49.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:48 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T08:49:49.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:48 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-04T08:49:49.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:48 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T08:49:49.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:48 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:49.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:48 smithi071 ceph-mon[32789]: Deploying daemon mgr.y on smithi150 2024-03-04T08:49:49.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:48 smithi071 ceph-mon[32789]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:49.200 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:48 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1639938755' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e43b47d-8888-4631-bc96-8a96fcf8cd46"}]: dispatch 2024-03-04T08:49:49.200 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:48 smithi150 ceph-mon[36807]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:49.200 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:48 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1639938755' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e43b47d-8888-4631-bc96-8a96fcf8cd46"}]': finished 2024-03-04T08:49:49.200 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:48 smithi150 ceph-mon[36807]: osdmap e5: 1 total, 0 up, 1 in 2024-03-04T08:49:49.200 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:48 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T08:49:49.201 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:48 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3637172688' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:49:49.201 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:48 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:49.201 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:48 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:49.201 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:48 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:49.201 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:48 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T08:49:49.201 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:48 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-04T08:49:49.201 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:48 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T08:49:49.201 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:48 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:49.201 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:48 smithi150 ceph-mon[36807]: Deploying daemon mgr.y on smithi150 2024-03-04T08:49:49.201 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:48 smithi150 ceph-mon[36807]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:49.302 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:48 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:48.965+0000 7ffb4804e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T08:49:49.302 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:49 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:49.058+0000 7ffb4804e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T08:49:49.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:48 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1639938755' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e43b47d-8888-4631-bc96-8a96fcf8cd46"}]: dispatch 2024-03-04T08:49:49.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:48 smithi158 ceph-mon[37241]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:49.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:48 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1639938755' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e43b47d-8888-4631-bc96-8a96fcf8cd46"}]': finished 2024-03-04T08:49:49.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:48 smithi158 ceph-mon[37241]: osdmap e5: 1 total, 0 up, 1 in 2024-03-04T08:49:49.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:48 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T08:49:49.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:48 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3637172688' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:49:49.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:48 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:49.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:48 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:49.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:48 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:49.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:48 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T08:49:49.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:48 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-04T08:49:49.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:48 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T08:49:49.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:48 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:49.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:48 smithi158 ceph-mon[37241]: Deploying daemon mgr.y on smithi150 2024-03-04T08:49:49.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:48 smithi158 ceph-mon[37241]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:50.029 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:49 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:49.700+0000 7ffb4804e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T08:49:50.029 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:49 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:49.794+0000 7ffb4804e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T08:49:50.029 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:49 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:49.937+0000 7ffb4804e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T08:49:50.302 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:50 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:50.033+0000 7ffb4804e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T08:49:50.302 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:50 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:50.116+0000 7ffb4804e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T08:49:50.665 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:50 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:50.383+0000 7ffb4804e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-04T08:49:50.665 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:50 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:50.491+0000 7ffb4804e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T08:49:50.665 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:50 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:50.576+0000 7ffb4804e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T08:49:50.665 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:50 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:50.669+0000 7ffb4804e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T08:49:50.780 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:50 smithi150 systemd[1]: Starting Ceph mgr.y for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T08:49:51.052 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:50 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:50.843+0000 7ffb4804e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T08:49:51.071 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:50 smithi150 podman[38181]: 2024-03-04 08:49:50.790220968 +0000 UTC m=+0.102029538 container create 39baab92d66cee74f7b95e4b036bc8bdeb7b996f314addd358c01b65fbad5a67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-04T08:49:51.071 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:50 smithi150 podman[38181]: 2024-03-04 08:49:50.705643088 +0000 UTC m=+0.017451660 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T08:49:51.071 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:51 smithi150 podman[38181]: 2024-03-04 08:49:51.001058942 +0000 UTC m=+0.312867531 container init 39baab92d66cee74f7b95e4b036bc8bdeb7b996f314addd358c01b65fbad5a67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y, RELEASE=wip-lusov-qdb-1bc250c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-04T08:49:51.072 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:51 smithi150 podman[38181]: 2024-03-04 08:49:51.003595363 +0000 UTC m=+0.315403935 container start 39baab92d66cee74f7b95e4b036bc8bdeb7b996f314addd358c01b65fbad5a67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y, org.label-schema.build-date=20240222, ceph=True, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-04T08:49:51.072 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:51 smithi150 ceph-mgr[38200]: -- 172.21.15.150:0/3792532120 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e3549985a0 con 0x55e35496cc00 2024-03-04T08:49:51.072 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:51 smithi150 bash[38181]: 39baab92d66cee74f7b95e4b036bc8bdeb7b996f314addd358c01b65fbad5a67 2024-03-04T08:49:51.529 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:51 smithi150 systemd[1]: Started Ceph mgr.y for b3c85c40-da03-11ee-95c6-87774f69a715. 2024-03-04T08:49:51.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:51 smithi071 ceph-mon[32789]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:51.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:51 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:51.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:51 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:51.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:51 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:51.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:51 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:51.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:51 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:49:51.982 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:51 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:51.846+0000 7ffb4804e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-04T08:49:51.982 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:51 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:51.985+0000 7ffb4804e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T08:49:51.982 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:51 smithi158 ceph-mon[37241]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:51.982 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:51 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:51.982 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:51 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:51.982 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:51 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:51.982 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:51 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:51.982 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:51 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:49:52.029 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:51 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:51.575+0000 7f0034481200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T08:49:52.030 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:51 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:51.673+0000 7f0034481200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T08:49:52.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:51 smithi150 ceph-mon[36807]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:52.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:51 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:52.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:51 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:52.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:51 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:52.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:51 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:52.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:51 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:49:52.552 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:52 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:52.327+0000 7ffb4804e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T08:49:52.552 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:52 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:52.417+0000 7ffb4804e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T08:49:52.709 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:52 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:52.365+0000 7f0034481200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T08:49:52.709 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:52 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:52.458+0000 7f0034481200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T08:49:52.709 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:52 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:52.610+0000 7f0034481200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T08:49:52.709 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:52 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:52.712+0000 7f0034481200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T08:49:53.029 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:52 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:52.801+0000 7f0034481200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T08:49:53.052 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:52 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:52.618+0000 7ffb4804e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T08:49:53.052 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:52 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:52.799+0000 7ffb4804e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T08:49:53.336 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:53 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:53.078+0000 7ffb4804e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T08:49:53.336 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:53 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:53.336+0000 7ffb4804e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T08:49:53.377 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:53 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:53.076+0000 7f0034481200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-04T08:49:53.378 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:53 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:53.193+0000 7f0034481200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T08:49:53.378 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:53 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:53.281+0000 7f0034481200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T08:49:53.379 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:53 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:53.379+0000 7f0034481200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T08:49:53.615 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:53 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:53.569+0000 7ffb4804e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T08:49:53.780 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:53 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:53.556+0000 7f0034481200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T08:49:53.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:53 smithi150 ceph-mon[36807]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:53.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:53 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:53.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:53 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:53.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:53 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:53.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:53 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:53.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:53 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:53.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:53 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:53.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:53 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:49:53.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:53 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:53.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:53 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T08:49:53.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:53 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T08:49:53.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:53 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:53.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:53 smithi071 ceph-mon[32789]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:53.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:53 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:53.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:53 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:53.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:53 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:53.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:53 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:53.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:53 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:53.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:53 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:53.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:53 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:49:53.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:53 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:53.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:53 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T08:49:53.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:53 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T08:49:53.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:53 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:54.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:53 smithi158 ceph-mon[37241]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:54.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:53 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:54.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:53 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:54.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:53 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:54.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:53 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:54.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:53 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:54.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:53 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:54.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:53 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:49:54.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:53 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:54.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:53 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T08:49:54.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:53 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T08:49:54.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:53 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:54.618 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:54 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:54.356+0000 7ffb4804e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T08:49:54.913 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:54 smithi158 ceph-mon[37241]: Reconfiguring mgr.x (unknown last config time)... 2024-03-04T08:49:54.913 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:54 smithi158 ceph-mon[37241]: Reconfiguring daemon mgr.x on smithi071 2024-03-04T08:49:54.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:54 smithi071 ceph-mon[32789]: Reconfiguring mgr.x (unknown last config time)... 2024-03-04T08:49:54.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:54 smithi071 ceph-mon[32789]: Reconfiguring daemon mgr.x on smithi071 2024-03-04T08:49:55.012 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:54 smithi150 ceph-mon[36807]: Reconfiguring mgr.x (unknown last config time)... 2024-03-04T08:49:55.012 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:54 smithi150 ceph-mon[36807]: Reconfiguring daemon mgr.x on smithi071 2024-03-04T08:49:55.012 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:54 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:54.547+0000 7f0034481200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-04T08:49:55.013 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:54 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:54.680+0000 7f0034481200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T08:49:55.013 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:55 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:55.016+0000 7f0034481200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T08:49:55.219 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:54 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:54.917+0000 7ffb4804e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T08:49:55.220 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:55 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:55.013+0000 7ffb4804e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T08:49:55.220 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:49:55 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:49:55.223+0000 7ffb4804e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T08:49:55.279 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:55 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:55.105+0000 7f0034481200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T08:49:55.619 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:55 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:55.300+0000 7f0034481200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T08:49:55.619 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:55 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:55.488+0000 7f0034481200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T08:49:55.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:55.630+0000 7f5efe7fc700 1 -- 172.21.15.71:0/1720266234 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 14) v1 ==== 91342+0+0 (secure 0 0 0) 0x7f5f0001d3b0 con 0x7f5f08103c80 2024-03-04T08:49:55.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:55 smithi071 ceph-mon[32789]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:55.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:55 smithi071 ceph-mon[32789]: Standby manager daemon z started 2024-03-04T08:49:55.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:55 smithi071 ceph-mon[32789]: from='mgr.? 172.21.15.158:0/932881617' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-04T08:49:55.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:55 smithi071 ceph-mon[32789]: from='mgr.? 172.21.15.158:0/932881617' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T08:49:55.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:55 smithi071 ceph-mon[32789]: from='mgr.? 172.21.15.158:0/932881617' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-04T08:49:55.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:55 smithi071 ceph-mon[32789]: from='mgr.? 172.21.15.158:0/932881617' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T08:49:55.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:55 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-04T08:49:55.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:55 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:56.030 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:55 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:55.777+0000 7f0034481200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T08:49:56.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:55 smithi150 ceph-mon[36807]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:56.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:55 smithi150 ceph-mon[36807]: Standby manager daemon z started 2024-03-04T08:49:56.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:55 smithi150 ceph-mon[36807]: from='mgr.? 172.21.15.158:0/932881617' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-04T08:49:56.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:55 smithi150 ceph-mon[36807]: from='mgr.? 172.21.15.158:0/932881617' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T08:49:56.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:55 smithi150 ceph-mon[36807]: from='mgr.? 172.21.15.158:0/932881617' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-04T08:49:56.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:55 smithi150 ceph-mon[36807]: from='mgr.? 172.21.15.158:0/932881617' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T08:49:56.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:55 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-04T08:49:56.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:55 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:56.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:55 smithi158 ceph-mon[37241]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:56.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:55 smithi158 ceph-mon[37241]: Standby manager daemon z started 2024-03-04T08:49:56.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:55 smithi158 ceph-mon[37241]: from='mgr.? 172.21.15.158:0/932881617' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-04T08:49:56.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:55 smithi158 ceph-mon[37241]: from='mgr.? 172.21.15.158:0/932881617' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T08:49:56.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:55 smithi158 ceph-mon[37241]: from='mgr.? 172.21.15.158:0/932881617' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-04T08:49:56.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:55 smithi158 ceph-mon[37241]: from='mgr.? 172.21.15.158:0/932881617' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T08:49:56.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:55 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-04T08:49:56.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:55 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:56.529 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:56 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:56.040+0000 7f0034481200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T08:49:56.530 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:56 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:56.278+0000 7f0034481200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T08:49:56.790 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:56 smithi071 ceph-mon[32789]: Deploying daemon osd.0 on smithi071 2024-03-04T08:49:56.790 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:56 smithi071 ceph-mon[32789]: mgrmap e14: x(active, since 93s), standbys: z 2024-03-04T08:49:56.791 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:56 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-04T08:49:56.791 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:56 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:56.791 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:56 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:56.791 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:56 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:49:56.791 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:56 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:56.791 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:56 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:49:56.791 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:56 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:57.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:56 smithi150 ceph-mon[36807]: Deploying daemon osd.0 on smithi071 2024-03-04T08:49:57.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:56 smithi150 ceph-mon[36807]: mgrmap e14: x(active, since 93s), standbys: z 2024-03-04T08:49:57.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:56 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-04T08:49:57.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:56 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:57.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:56 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:57.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:56 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:49:57.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:56 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:57.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:56 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:49:57.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:56 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:57.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:56 smithi158 ceph-mon[37241]: Deploying daemon osd.0 on smithi071 2024-03-04T08:49:57.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:56 smithi158 ceph-mon[37241]: mgrmap e14: x(active, since 93s), standbys: z 2024-03-04T08:49:57.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:56 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-04T08:49:57.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:56 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:57.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:56 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:57.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:56 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:49:57.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:56 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:49:57.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:56 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:49:57.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:56 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:49:57.531 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:57 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:57.088+0000 7f0034481200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T08:49:57.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:57 smithi071 ceph-mon[32789]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:57.972 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:57 smithi150 ceph-mon[36807]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:57.973 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:57 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:57.661+0000 7f0034481200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T08:49:57.973 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:57 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:57.760+0000 7f0034481200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T08:49:57.973 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:49:57 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:49:57.975+0000 7f0034481200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T08:49:58.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:57 smithi158 ceph-mon[37241]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:58.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:49:58.641+0000 7f5efe7fc700 1 -- 172.21.15.71:0/1720266234 <== mon.0 v2:172.21.15.71:3300/0 8 ==== mgrmap(e 15) v1 ==== 136700+0+0 (secure 0 0 0) 0x7f5f000225f0 con 0x7f5f08103c80 2024-03-04T08:49:58.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:58 smithi071 ceph-mon[32789]: Standby manager daemon y started 2024-03-04T08:49:58.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:58 smithi071 ceph-mon[32789]: from='mgr.? 172.21.15.150:0/334908401' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-04T08:49:58.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:58 smithi071 ceph-mon[32789]: from='mgr.? 172.21.15.150:0/334908401' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T08:49:58.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:58 smithi071 ceph-mon[32789]: from='mgr.? 172.21.15.150:0/334908401' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-04T08:49:58.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:58 smithi071 ceph-mon[32789]: from='mgr.? 172.21.15.150:0/334908401' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T08:49:59.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:58 smithi150 ceph-mon[36807]: Standby manager daemon y started 2024-03-04T08:49:59.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:58 smithi150 ceph-mon[36807]: from='mgr.? 172.21.15.150:0/334908401' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-04T08:49:59.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:58 smithi150 ceph-mon[36807]: from='mgr.? 172.21.15.150:0/334908401' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T08:49:59.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:58 smithi150 ceph-mon[36807]: from='mgr.? 172.21.15.150:0/334908401' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-04T08:49:59.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:58 smithi150 ceph-mon[36807]: from='mgr.? 172.21.15.150:0/334908401' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T08:49:59.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:58 smithi158 ceph-mon[37241]: Standby manager daemon y started 2024-03-04T08:49:59.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:58 smithi158 ceph-mon[37241]: from='mgr.? 172.21.15.150:0/334908401' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-04T08:49:59.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:58 smithi158 ceph-mon[37241]: from='mgr.? 172.21.15.150:0/334908401' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T08:49:59.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:58 smithi158 ceph-mon[37241]: from='mgr.? 172.21.15.150:0/334908401' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-04T08:49:59.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:58 smithi158 ceph-mon[37241]: from='mgr.? 172.21.15.150:0/334908401' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T08:49:59.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:59 smithi071 ceph-mon[32789]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:49:59.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:59 smithi071 ceph-mon[32789]: mgrmap e15: x(active, since 96s), standbys: z, y 2024-03-04T08:49:59.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:49:59 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-04T08:50:00.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:59 smithi150 ceph-mon[36807]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:50:00.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:59 smithi150 ceph-mon[36807]: mgrmap e15: x(active, since 96s), standbys: z, y 2024-03-04T08:50:00.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:49:59 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-04T08:50:00.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:59 smithi158 ceph-mon[37241]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:50:00.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:59 smithi158 ceph-mon[37241]: mgrmap e15: x(active, since 96s), standbys: z, y 2024-03-04T08:50:00.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:49:59 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-04T08:50:00.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:00 smithi071 ceph-mon[32789]: overall HEALTH_OK 2024-03-04T08:50:01.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:00 smithi150 ceph-mon[36807]: overall HEALTH_OK 2024-03-04T08:50:01.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:00 smithi158 ceph-mon[37241]: overall HEALTH_OK 2024-03-04T08:50:01.833 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:01 smithi071 ceph-mon[32789]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:50:01.833 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:01 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:01.833 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:01 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:01.833 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:01 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:50:02.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:01 smithi150 ceph-mon[36807]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:50:02.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:01 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:02.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:01 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:02.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:01 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:50:02.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:01 smithi158 ceph-mon[37241]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:50:02.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:01 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:02.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:01 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:02.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:01 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:50:03.730 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:03 smithi071 ceph-mon[32789]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:50:04.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:03 smithi150 ceph-mon[36807]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:50:04.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:03 smithi158 ceph-mon[37241]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:50:04.392 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 0 on host 'smithi071' 2024-03-04T08:50:04.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:50:04.394+0000 7f5efe7fc700 1 -- 172.21.15.71:0/1720266234 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5f08062500 con 0x7f5ef4038e80 2024-03-04T08:50:04.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:50:04.397+0000 7f5f0f616700 1 -- 172.21.15.71:0/1720266234 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f5ef4038e80 msgr2=0x7f5ef403b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:50:04.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:50:04.397+0000 7f5f0f616700 1 --2- 172.21.15.71:0/1720266234 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f5ef4038e80 0x7f5ef403b340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800b580 tx=0x7f5ef8005fb0 comp rx=0 tx=0).stop 2024-03-04T08:50:04.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:50:04.397+0000 7f5f0f616700 1 -- 172.21.15.71:0/1720266234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f08103c80 msgr2=0x7f5f0819fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:50:04.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:50:04.397+0000 7f5f0f616700 1 --2- 172.21.15.71:0/1720266234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f08103c80 0x7f5f0819fa20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000b9a0 tx=0x7f5f0000bd60 comp rx=0 tx=0).stop 2024-03-04T08:50:04.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:50:04.397+0000 7f5f0f616700 1 -- 172.21.15.71:0/1720266234 shutdown_connections 2024-03-04T08:50:04.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:50:04.397+0000 7f5f0f616700 1 --2- 172.21.15.71:0/1720266234 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f5ef4038e80 0x7f5ef403b340 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:50:04.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:50:04.397+0000 7f5f0f616700 1 --2- 172.21.15.71:0/1720266234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f08103c80 0x7f5f0819fa20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:50:04.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:50:04.397+0000 7f5f0f616700 1 --2- 172.21.15.71:0/1720266234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f081020e0 0x7f5f08198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:50:04.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:50:04.397+0000 7f5f0f616700 1 --2- 172.21.15.71:0/1720266234 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5f081032e0 0x7f5f081989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:50:04.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:50:04.397+0000 7f5f0f616700 1 -- 172.21.15.71:0/1720266234 >> 172.21.15.71:0/1720266234 conn(0x7f5f080fd6a0 msgr2=0x7f5f081066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:50:04.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:50:04.397+0000 7f5f0f616700 1 -- 172.21.15.71:0/1720266234 shutdown_connections 2024-03-04T08:50:04.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:50:04.397+0000 7f5f0f616700 1 -- 172.21.15.71:0/1720266234 wait complete. 2024-03-04T08:50:05.028 DEBUG:teuthology.orchestra.run.smithi071:osd.0> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.0.service 2024-03-04T08:50:05.031 INFO:tasks.cephadm:Deploying osd.1 on smithi150 with /dev/vg_nvme/lv_4... 2024-03-04T08:50:05.031 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-04T08:50:05.275 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T08:50:05.396 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:05 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:05.396 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:05 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:05.397 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:05 smithi150 ceph-mon[36807]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:50:05.397 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:05 smithi150 ceph-mon[36807]: from='osd.0 [v2:172.21.15.71:6802/1173641530,v1:172.21.15.71:6803/1173641530]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-04T08:50:05.397 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:05 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:05.397 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:05 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:05.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:05 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:05.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:05 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:05.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:05 smithi071 ceph-mon[32789]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:50:05.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:05 smithi071 ceph-mon[32789]: from='osd.0 [v2:172.21.15.71:6802/1173641530,v1:172.21.15.71:6803/1173641530]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-04T08:50:05.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:05 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:05.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:05 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:05.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:05 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:05.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:05 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:05.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:05 smithi158 ceph-mon[37241]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:50:05.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:05 smithi158 ceph-mon[37241]: from='osd.0 [v2:172.21.15.71:6802/1173641530,v1:172.21.15.71:6803/1173641530]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-04T08:50:05.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:05 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:05.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:05 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:06.610 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:06 smithi150 ceph-mon[36807]: from='osd.0 [v2:172.21.15.71:6802/1173641530,v1:172.21.15.71:6803/1173641530]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-04T08:50:06.610 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:06 smithi150 ceph-mon[36807]: osdmap e6: 1 total, 0 up, 1 in 2024-03-04T08:50:06.610 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:06 smithi150 ceph-mon[36807]: from='osd.0 [v2:172.21.15.71:6802/1173641530,v1:172.21.15.71:6803/1173641530]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-04T08:50:06.610 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:06 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T08:50:06.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:06 smithi071 ceph-mon[32789]: from='osd.0 [v2:172.21.15.71:6802/1173641530,v1:172.21.15.71:6803/1173641530]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-04T08:50:06.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:06 smithi071 ceph-mon[32789]: osdmap e6: 1 total, 0 up, 1 in 2024-03-04T08:50:06.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:06 smithi071 ceph-mon[32789]: from='osd.0 [v2:172.21.15.71:6802/1173641530,v1:172.21.15.71:6803/1173641530]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-04T08:50:06.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:06 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T08:50:06.672 INFO:journalctl@ceph.osd.0.smithi071.stdout:Mar 04 08:50:06 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-0[47571]: 2024-03-04T08:50:06.364+0000 7f1eceefc700 -1 osd.0 0 waiting for initial osdmap 2024-03-04T08:50:06.672 INFO:journalctl@ceph.osd.0.smithi071.stdout:Mar 04 08:50:06 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-0[47571]: 2024-03-04T08:50:06.366+0000 7f1ecacef700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T08:50:06.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:06 smithi158 ceph-mon[37241]: from='osd.0 [v2:172.21.15.71:6802/1173641530,v1:172.21.15.71:6803/1173641530]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-04T08:50:06.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:06 smithi158 ceph-mon[37241]: osdmap e6: 1 total, 0 up, 1 in 2024-03-04T08:50:06.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:06 smithi158 ceph-mon[37241]: from='osd.0 [v2:172.21.15.71:6802/1173641530,v1:172.21.15.71:6803/1173641530]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-04T08:50:06.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:06 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T08:50:07.148 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-04T08:50:07.149 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-04T08:50:07.149 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-03-04T08:50:07.149 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-03-04T08:50:07.149 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112659 s, 931 MB/s 2024-03-04T08:50:07.149 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-03-04T08:50:07.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:07 smithi150 ceph-mon[36807]: from='osd.0 [v2:172.21.15.71:6802/1173641530,v1:172.21.15.71:6803/1173641530]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-03-04T08:50:07.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:07 smithi150 ceph-mon[36807]: osdmap e7: 1 total, 0 up, 1 in 2024-03-04T08:50:07.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:07 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T08:50:07.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:07 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T08:50:07.453 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:07 smithi150 ceph-mon[36807]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:50:07.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:07 smithi071 ceph-mon[32789]: from='osd.0 [v2:172.21.15.71:6802/1173641530,v1:172.21.15.71:6803/1173641530]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-03-04T08:50:07.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:07 smithi071 ceph-mon[32789]: osdmap e7: 1 total, 0 up, 1 in 2024-03-04T08:50:07.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:07 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T08:50:07.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:07 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T08:50:07.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:07 smithi071 ceph-mon[32789]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:50:07.764 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi150:vg_nvme/lv_4 2024-03-04T08:50:07.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:07 smithi158 ceph-mon[37241]: from='osd.0 [v2:172.21.15.71:6802/1173641530,v1:172.21.15.71:6803/1173641530]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-03-04T08:50:07.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:07 smithi158 ceph-mon[37241]: osdmap e7: 1 total, 0 up, 1 in 2024-03-04T08:50:07.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:07 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T08:50:07.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:07 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T08:50:07.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:07 smithi158 ceph-mon[37241]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T08:50:07.998 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T08:50:08.432 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:08 smithi150 ceph-mon[36807]: purged_snaps scrub starts 2024-03-04T08:50:08.432 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:08 smithi150 ceph-mon[36807]: purged_snaps scrub ok 2024-03-04T08:50:08.432 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:08 smithi150 ceph-mon[36807]: osd.0 [v2:172.21.15.71:6802/1173641530,v1:172.21.15.71:6803/1173641530] boot 2024-03-04T08:50:08.432 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:08 smithi150 ceph-mon[36807]: osdmap e8: 1 total, 1 up, 1 in 2024-03-04T08:50:08.432 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:08 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T08:50:08.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:08 smithi071 ceph-mon[32789]: purged_snaps scrub starts 2024-03-04T08:50:08.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:08 smithi071 ceph-mon[32789]: purged_snaps scrub ok 2024-03-04T08:50:08.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:08 smithi071 ceph-mon[32789]: osd.0 [v2:172.21.15.71:6802/1173641530,v1:172.21.15.71:6803/1173641530] boot 2024-03-04T08:50:08.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:08 smithi071 ceph-mon[32789]: osdmap e8: 1 total, 1 up, 1 in 2024-03-04T08:50:08.445 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:08 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T08:50:08.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:08 smithi158 ceph-mon[37241]: purged_snaps scrub starts 2024-03-04T08:50:08.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:08 smithi158 ceph-mon[37241]: purged_snaps scrub ok 2024-03-04T08:50:08.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:08 smithi158 ceph-mon[37241]: osd.0 [v2:172.21.15.71:6802/1173641530,v1:172.21.15.71:6803/1173641530] boot 2024-03-04T08:50:08.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:08 smithi158 ceph-mon[37241]: osdmap e8: 1 total, 1 up, 1 in 2024-03-04T08:50:08.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:08 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T08:50:09.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.587+0000 7f9e57059700 1 -- 172.21.15.150:0/3800295096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e501020f0 msgr2=0x7f9e50102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:50:09.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.587+0000 7f9e57059700 1 --2- 172.21.15.150:0/3800295096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e501020f0 0x7f9e50102510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0099c0 tx=0x7f9e4c009cd0 comp rx=0 tx=0).stop 2024-03-04T08:50:09.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.587+0000 7f9e57059700 1 -- 172.21.15.150:0/3800295096 shutdown_connections 2024-03-04T08:50:09.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.587+0000 7f9e57059700 1 --2- 172.21.15.150:0/3800295096 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9e50103c90 0x7f9e5010a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:50:09.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.587+0000 7f9e57059700 1 --2- 172.21.15.150:0/3800295096 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e501020f0 0x7f9e50102510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:50:09.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.587+0000 7f9e57059700 1 --2- 172.21.15.150:0/3800295096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e501032f0 0x7f9e50103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:50:09.586 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.587+0000 7f9e57059700 1 -- 172.21.15.150:0/3800295096 >> 172.21.15.150:0/3800295096 conn(0x7f9e500fd6d0 msgr2=0x7f9e500ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:50:09.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.587+0000 7f9e57059700 1 -- 172.21.15.150:0/3800295096 shutdown_connections 2024-03-04T08:50:09.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.587+0000 7f9e57059700 1 -- 172.21.15.150:0/3800295096 wait complete. 2024-03-04T08:50:09.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.588+0000 7f9e57059700 1 Processor -- start 2024-03-04T08:50:09.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.588+0000 7f9e57059700 1 -- start start 2024-03-04T08:50:09.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.588+0000 7f9e57059700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9e501020f0 0x7f9e5019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:50:09.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.588+0000 7f9e57059700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e501032f0 0x7f9e5019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:50:09.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.588+0000 7f9e57059700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e50103c90 0x7f9e501a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:50:09.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.588+0000 7f9e57059700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5010dd60 con 0x7f9e50103c90 2024-03-04T08:50:09.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.588+0000 7f9e57059700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5010dbe0 con 0x7f9e501032f0 2024-03-04T08:50:09.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.588+0000 7f9e57059700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5010dee0 con 0x7f9e501020f0 2024-03-04T08:50:09.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.588+0000 7f9e47fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e501032f0 0x7f9e5019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:50:09.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.588+0000 7f9e47fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e501032f0 0x7f9e5019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49822/0 (socket says 172.21.15.150:49822) 2024-03-04T08:50:09.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.588+0000 7f9e47fff700 1 -- 172.21.15.150:0/295549986 learned_addr learned my addr 172.21.15.150:0/295549986 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-04T08:50:09.587 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.588+0000 7f9e54df5700 1 --2- 172.21.15.150:0/295549986 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9e501020f0 0x7f9e5019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:50:09.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.588+0000 7f9e555f6700 1 --2- 172.21.15.150:0/295549986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e50103c90 0x7f9e501a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:50:09.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.588+0000 7f9e47fff700 1 -- 172.21.15.150:0/295549986 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9e501020f0 msgr2=0x7f9e5019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:50:09.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.588+0000 7f9e47fff700 1 --2- 172.21.15.150:0/295549986 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9e501020f0 0x7f9e5019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:50:09.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.588+0000 7f9e47fff700 1 -- 172.21.15.150:0/295549986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e50103c90 msgr2=0x7f9e501a1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:50:09.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.588+0000 7f9e47fff700 1 --2- 172.21.15.150:0/295549986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e50103c90 0x7f9e501a1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:50:09.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.588+0000 7f9e47fff700 1 -- 172.21.15.150:0/295549986 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e501a2240 con 0x7f9e501032f0 2024-03-04T08:50:09.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.588+0000 7f9e47fff700 1 --2- 172.21.15.150:0/295549986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e501032f0 0x7f9e5019ab20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000b530 tx=0x7f9e4000b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:50:09.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.588+0000 7f9e45ffb700 1 -- 172.21.15.150:0/295549986 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e40015020 con 0x7f9e501032f0 2024-03-04T08:50:09.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.589+0000 7f9e57059700 1 -- 172.21.15.150:0/295549986 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e501a2530 con 0x7f9e501032f0 2024-03-04T08:50:09.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.589+0000 7f9e45ffb700 1 -- 172.21.15.150:0/295549986 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e40011d10 con 0x7f9e501032f0 2024-03-04T08:50:09.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.589+0000 7f9e45ffb700 1 -- 172.21.15.150:0/295549986 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4000f2e0 con 0x7f9e501032f0 2024-03-04T08:50:09.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.589+0000 7f9e57059700 1 -- 172.21.15.150:0/295549986 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e501a2af0 con 0x7f9e501032f0 2024-03-04T08:50:09.588 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.590+0000 7f9e57059700 1 -- 172.21.15.150:0/295549986 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e50068d80 con 0x7f9e501032f0 2024-03-04T08:50:09.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.591+0000 7f9e45ffb700 1 -- 172.21.15.150:0/295549986 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 15) v1 ==== 136700+0+0 (secure 0 0 0) 0x7f9e40018020 con 0x7f9e501032f0 2024-03-04T08:50:09.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.591+0000 7f9e45ffb700 1 --2- 172.21.15.150:0/295549986 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f9e380a1d80 0x7f9e380a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:50:09.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.592+0000 7f9e45ffb700 1 -- 172.21.15.150:0/295549986 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f9e400cbb00 con 0x7f9e501032f0 2024-03-04T08:50:09.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.592+0000 7f9e54df5700 1 --2- 172.21.15.150:0/295549986 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f9e380a1d80 0x7f9e380a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:50:09.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.592+0000 7f9e54df5700 1 --2- 172.21.15.150:0/295549986 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f9e380a1d80 0x7f9e380a4240 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c003890 tx=0x7f9e4c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:50:09.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.596+0000 7f9e45ffb700 1 -- 172.21.15.150:0/295549986 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9e4001f020 con 0x7f9e501032f0 2024-03-04T08:50:09.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:09 smithi071 ceph-mon[32789]: osdmap e9: 1 total, 1 up, 1 in 2024-03-04T08:50:09.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:09 smithi071 ceph-mon[32789]: Detected new or changed devices on smithi071 2024-03-04T08:50:09.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:09 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:09.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:09 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:09.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:09 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:50:09.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:09 smithi071 ceph-mon[32789]: Adjusting osd_memory_target on smithi071 to 17088M 2024-03-04T08:50:09.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:09 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:09.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:09 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:50:09.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:09 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:50:09.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:09 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:09.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:09 smithi071 ceph-mon[32789]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:09.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:09 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:50:09.673 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:09 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:50:09.673 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:09 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:50:09.673 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:09 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:09.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:09 smithi150 ceph-mon[36807]: osdmap e9: 1 total, 1 up, 1 in 2024-03-04T08:50:09.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:09 smithi150 ceph-mon[36807]: Detected new or changed devices on smithi071 2024-03-04T08:50:09.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:09 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:09.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:09 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:09.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:09 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:50:09.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:09 smithi150 ceph-mon[36807]: Adjusting osd_memory_target on smithi071 to 17088M 2024-03-04T08:50:09.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:09 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:09.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:09 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:50:09.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:09 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:50:09.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:09 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:09.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:09 smithi150 ceph-mon[36807]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:09.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:09 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:50:09.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:09 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:50:09.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:09 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:50:09.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:09 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:09.793 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:09.797+0000 7f9e57059700 1 -- 172.21.15.150:0/295549986 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e50075a10 con 0x7f9e380a1d80 2024-03-04T08:50:09.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:09 smithi158 ceph-mon[37241]: osdmap e9: 1 total, 1 up, 1 in 2024-03-04T08:50:09.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:09 smithi158 ceph-mon[37241]: Detected new or changed devices on smithi071 2024-03-04T08:50:09.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:09 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:09.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:09 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:09.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:09 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:50:09.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:09 smithi158 ceph-mon[37241]: Adjusting osd_memory_target on smithi071 to 17088M 2024-03-04T08:50:09.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:09 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:09.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:09 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:50:09.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:09 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:50:09.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:09 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:09.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:09 smithi158 ceph-mon[37241]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:09.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:09 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:50:09.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:09 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:50:09.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:09 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:50:09.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:09 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:10.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:10 smithi150 ceph-mon[36807]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:50:10.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:10 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:50:10.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:10 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:50:10.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:10 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:50:10.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:10 smithi158 ceph-mon[37241]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:50:10.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:10 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:50:10.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:10 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:50:10.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:10 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:50:10.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:10 smithi071 ceph-mon[32789]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:50:10.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:10 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:50:10.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:10 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:50:10.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:10 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:50:11.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:11 smithi158 ceph-mon[37241]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:11.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:11 smithi071 ceph-mon[32789]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:12.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:11 smithi150 ceph-mon[36807]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:12.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:12 smithi158 ceph-mon[37241]: from='client.? 172.21.15.150:0/4008357035' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1209f214-f2e2-4209-bf69-4528a43a2123"}]: dispatch 2024-03-04T08:50:12.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:12 smithi158 ceph-mon[37241]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1209f214-f2e2-4209-bf69-4528a43a2123"}]: dispatch 2024-03-04T08:50:12.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:12 smithi158 ceph-mon[37241]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1209f214-f2e2-4209-bf69-4528a43a2123"}]': finished 2024-03-04T08:50:12.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:12 smithi158 ceph-mon[37241]: osdmap e10: 2 total, 1 up, 2 in 2024-03-04T08:50:12.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:12 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T08:50:12.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:12 smithi071 ceph-mon[32789]: from='client.? 172.21.15.150:0/4008357035' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1209f214-f2e2-4209-bf69-4528a43a2123"}]: dispatch 2024-03-04T08:50:12.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:12 smithi071 ceph-mon[32789]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1209f214-f2e2-4209-bf69-4528a43a2123"}]: dispatch 2024-03-04T08:50:12.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:12 smithi071 ceph-mon[32789]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1209f214-f2e2-4209-bf69-4528a43a2123"}]': finished 2024-03-04T08:50:12.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:12 smithi071 ceph-mon[32789]: osdmap e10: 2 total, 1 up, 2 in 2024-03-04T08:50:12.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:12 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T08:50:13.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:12 smithi150 ceph-mon[36807]: from='client.? 172.21.15.150:0/4008357035' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1209f214-f2e2-4209-bf69-4528a43a2123"}]: dispatch 2024-03-04T08:50:13.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:12 smithi150 ceph-mon[36807]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1209f214-f2e2-4209-bf69-4528a43a2123"}]: dispatch 2024-03-04T08:50:13.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:12 smithi150 ceph-mon[36807]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1209f214-f2e2-4209-bf69-4528a43a2123"}]': finished 2024-03-04T08:50:13.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:12 smithi150 ceph-mon[36807]: osdmap e10: 2 total, 1 up, 2 in 2024-03-04T08:50:13.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:12 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T08:50:13.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:13 smithi071 ceph-mon[32789]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:13.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:13 smithi071 ceph-mon[32789]: from='client.? 172.21.15.150:0/3375727557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:50:14.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:13 smithi150 ceph-mon[36807]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:14.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:13 smithi150 ceph-mon[36807]: from='client.? 172.21.15.150:0/3375727557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:50:14.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:13 smithi158 ceph-mon[37241]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:14.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:13 smithi158 ceph-mon[37241]: from='client.? 172.21.15.150:0/3375727557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:50:15.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:15 smithi071 ceph-mon[32789]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:15.993 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:15 smithi150 ceph-mon[36807]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:16.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:15 smithi158 ceph-mon[37241]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:17.708 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:17 smithi150 ceph-mon[36807]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:17.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:17 smithi071 ceph-mon[32789]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:18.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:17 smithi158 ceph-mon[37241]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:19.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:19 smithi071 ceph-mon[32789]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:19.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:19 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-04T08:50:19.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:19 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:50:19.965 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:19 smithi150 ceph-mon[36807]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:19.965 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:19 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-04T08:50:19.965 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:19 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:50:20.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:19 smithi158 ceph-mon[37241]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:20.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:19 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-04T08:50:20.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:19 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:50:20.979 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:20 smithi150 ceph-mon[36807]: Deploying daemon osd.1 on smithi150 2024-03-04T08:50:21.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:20 smithi158 ceph-mon[37241]: Deploying daemon osd.1 on smithi150 2024-03-04T08:50:21.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:20 smithi071 ceph-mon[32789]: Deploying daemon osd.1 on smithi150 2024-03-04T08:50:21.978 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:21 smithi150 ceph-mon[36807]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:22.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:21 smithi158 ceph-mon[37241]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:22.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:21 smithi071 ceph-mon[32789]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:23.936 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:23 smithi150 ceph-mon[36807]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:24.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:23 smithi158 ceph-mon[37241]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:24.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:23 smithi071 ceph-mon[32789]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:25.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:25 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:25.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:25 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:25.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:25 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:50:25.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:25 smithi150 ceph-mon[36807]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:25.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:25 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:25.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:25 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:25.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:25 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:50:25.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:25 smithi158 ceph-mon[37241]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:25.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:25 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:25.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:25 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:25.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:25 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:50:25.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:25 smithi071 ceph-mon[32789]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:27.756 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:27 smithi150 ceph-mon[36807]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:27.756 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:27.759+0000 7f9e45ffb700 1 -- 172.21.15.150:0/295549986 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9e50075a10 con 0x7f9e380a1d80 2024-03-04T08:50:27.757 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 1 on host 'smithi150' 2024-03-04T08:50:27.759 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:27.763+0000 7f9e57059700 1 -- 172.21.15.150:0/295549986 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f9e380a1d80 msgr2=0x7f9e380a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:50:27.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:27.763+0000 7f9e57059700 1 --2- 172.21.15.150:0/295549986 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f9e380a1d80 0x7f9e380a4240 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c003890 tx=0x7f9e4c005fb0 comp rx=0 tx=0).stop 2024-03-04T08:50:27.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:27.763+0000 7f9e57059700 1 -- 172.21.15.150:0/295549986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e501032f0 msgr2=0x7f9e5019ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:50:27.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:27.763+0000 7f9e57059700 1 --2- 172.21.15.150:0/295549986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e501032f0 0x7f9e5019ab20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000b530 tx=0x7f9e4000b840 comp rx=0 tx=0).stop 2024-03-04T08:50:27.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:27.763+0000 7f9e57059700 1 -- 172.21.15.150:0/295549986 shutdown_connections 2024-03-04T08:50:27.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:27.763+0000 7f9e57059700 1 --2- 172.21.15.150:0/295549986 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f9e380a1d80 0x7f9e380a4240 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:50:27.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:27.763+0000 7f9e57059700 1 --2- 172.21.15.150:0/295549986 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9e501020f0 0x7f9e5019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:50:27.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:27.763+0000 7f9e57059700 1 --2- 172.21.15.150:0/295549986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e50103c90 0x7f9e501a1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:50:27.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:27.763+0000 7f9e57059700 1 --2- 172.21.15.150:0/295549986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e501032f0 0x7f9e5019ab20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:50:27.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:27.763+0000 7f9e57059700 1 -- 172.21.15.150:0/295549986 >> 172.21.15.150:0/295549986 conn(0x7f9e500fd6d0 msgr2=0x7f9e500742f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:50:27.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:27.763+0000 7f9e57059700 1 -- 172.21.15.150:0/295549986 shutdown_connections 2024-03-04T08:50:27.760 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:50:27.763+0000 7f9e57059700 1 -- 172.21.15.150:0/295549986 wait complete. 2024-03-04T08:50:27.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:27 smithi071 ceph-mon[32789]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:28.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:27 smithi158 ceph-mon[37241]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:28.385 DEBUG:teuthology.orchestra.run.smithi150:osd.1> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.1.service 2024-03-04T08:50:28.387 INFO:tasks.cephadm:Deploying osd.2 on smithi158 with /dev/vg_nvme/lv_4... 2024-03-04T08:50:28.387 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-04T08:50:28.633 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T08:50:29.024 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:28 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:29.024 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:28 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:29.024 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:28 smithi158 ceph-mon[37241]: from='osd.1 [v2:172.21.15.150:6800/3383948110,v1:172.21.15.150:6801/3383948110]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-04T08:50:29.024 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:28 smithi158 ceph-mon[37241]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-04T08:50:29.024 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:28 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:29.024 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:28 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:29.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:28 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:29.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:28 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:29.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:28 smithi150 ceph-mon[36807]: from='osd.1 [v2:172.21.15.150:6800/3383948110,v1:172.21.15.150:6801/3383948110]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-04T08:50:29.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:28 smithi150 ceph-mon[36807]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-04T08:50:29.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:28 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:29.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:28 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:29.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:28 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:29.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:28 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:29.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:28 smithi071 ceph-mon[32789]: from='osd.1 [v2:172.21.15.150:6800/3383948110,v1:172.21.15.150:6801/3383948110]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-04T08:50:29.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:28 smithi071 ceph-mon[32789]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-04T08:50:29.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:28 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:29.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:28 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:30.021 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:29 smithi158 ceph-mon[37241]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:30.021 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:29 smithi158 ceph-mon[37241]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-04T08:50:30.021 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:29 smithi158 ceph-mon[37241]: from='osd.1 [v2:172.21.15.150:6800/3383948110,v1:172.21.15.150:6801/3383948110]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-04T08:50:30.021 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:29 smithi158 ceph-mon[37241]: osdmap e11: 2 total, 1 up, 2 in 2024-03-04T08:50:30.021 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:29 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T08:50:30.021 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:29 smithi158 ceph-mon[37241]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-04T08:50:30.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:29 smithi150 ceph-mon[36807]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:30.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:29 smithi150 ceph-mon[36807]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-04T08:50:30.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:29 smithi150 ceph-mon[36807]: from='osd.1 [v2:172.21.15.150:6800/3383948110,v1:172.21.15.150:6801/3383948110]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-04T08:50:30.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:29 smithi150 ceph-mon[36807]: osdmap e11: 2 total, 1 up, 2 in 2024-03-04T08:50:30.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:29 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T08:50:30.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:29 smithi150 ceph-mon[36807]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-04T08:50:30.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:29 smithi071 ceph-mon[32789]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:30.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:29 smithi071 ceph-mon[32789]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-04T08:50:30.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:29 smithi071 ceph-mon[32789]: from='osd.1 [v2:172.21.15.150:6800/3383948110,v1:172.21.15.150:6801/3383948110]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-04T08:50:30.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:29 smithi071 ceph-mon[32789]: osdmap e11: 2 total, 1 up, 2 in 2024-03-04T08:50:30.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:29 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T08:50:30.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:29 smithi071 ceph-mon[32789]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-04T08:50:30.542 INFO:teuthology.orchestra.run.smithi158.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-04T08:50:30.542 INFO:teuthology.orchestra.run.smithi158.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-04T08:50:30.542 INFO:teuthology.orchestra.run.smithi158.stderr: stderr: 10+0 records in 2024-03-04T08:50:30.542 INFO:teuthology.orchestra.run.smithi158.stderr:10+0 records out 2024-03-04T08:50:30.542 INFO:teuthology.orchestra.run.smithi158.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00864455 s, 1.2 GB/s 2024-03-04T08:50:30.543 INFO:teuthology.orchestra.run.smithi158.stderr:--> Zapping successful for: 2024-03-04T08:50:30.726 INFO:journalctl@ceph.osd.1.smithi150.stdout:Mar 04 08:50:30 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-1[41405]: 2024-03-04T08:50:30.473+0000 7f070babe700 -1 osd.1 0 waiting for initial osdmap 2024-03-04T08:50:30.726 INFO:journalctl@ceph.osd.1.smithi150.stdout:Mar 04 08:50:30 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-1[41405]: 2024-03-04T08:50:30.478+0000 7f07038a9700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T08:50:30.923 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:30 smithi158 ceph-mon[37241]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-03-04T08:50:30.923 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:30 smithi158 ceph-mon[37241]: osdmap e12: 2 total, 1 up, 2 in 2024-03-04T08:50:30.923 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:30 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T08:50:30.923 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:30 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T08:50:30.923 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:30 smithi158 ceph-mon[37241]: from='osd.1 ' entity='osd.1' 2024-03-04T08:50:31.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:30 smithi150 ceph-mon[36807]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-03-04T08:50:31.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:30 smithi150 ceph-mon[36807]: osdmap e12: 2 total, 1 up, 2 in 2024-03-04T08:50:31.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:30 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T08:50:31.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:30 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T08:50:31.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:30 smithi150 ceph-mon[36807]: from='osd.1 ' entity='osd.1' 2024-03-04T08:50:31.112 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi158:vg_nvme/lv_4 2024-03-04T08:50:31.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:30 smithi071 ceph-mon[32789]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-03-04T08:50:31.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:30 smithi071 ceph-mon[32789]: osdmap e12: 2 total, 1 up, 2 in 2024-03-04T08:50:31.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:30 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T08:50:31.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:30 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T08:50:31.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:30 smithi071 ceph-mon[32789]: from='osd.1 ' entity='osd.1' 2024-03-04T08:50:31.348 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T08:50:31.836 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:31 smithi150 ceph-mon[36807]: purged_snaps scrub starts 2024-03-04T08:50:31.836 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:31 smithi150 ceph-mon[36807]: purged_snaps scrub ok 2024-03-04T08:50:31.836 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:31 smithi150 ceph-mon[36807]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:31.836 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:31 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T08:50:31.836 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:31 smithi150 ceph-mon[36807]: osd.1 [v2:172.21.15.150:6800/3383948110,v1:172.21.15.150:6801/3383948110] boot 2024-03-04T08:50:31.836 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:31 smithi150 ceph-mon[36807]: osdmap e13: 2 total, 2 up, 2 in 2024-03-04T08:50:31.836 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:31 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T08:50:32.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:31 smithi158 ceph-mon[37241]: purged_snaps scrub starts 2024-03-04T08:50:32.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:31 smithi158 ceph-mon[37241]: purged_snaps scrub ok 2024-03-04T08:50:32.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:31 smithi158 ceph-mon[37241]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:32.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:31 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T08:50:32.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:31 smithi158 ceph-mon[37241]: osd.1 [v2:172.21.15.150:6800/3383948110,v1:172.21.15.150:6801/3383948110] boot 2024-03-04T08:50:32.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:31 smithi158 ceph-mon[37241]: osdmap e13: 2 total, 2 up, 2 in 2024-03-04T08:50:32.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:31 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T08:50:32.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:31 smithi071 ceph-mon[32789]: purged_snaps scrub starts 2024-03-04T08:50:32.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:31 smithi071 ceph-mon[32789]: purged_snaps scrub ok 2024-03-04T08:50:32.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:31 smithi071 ceph-mon[32789]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T08:50:32.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:31 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T08:50:32.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:31 smithi071 ceph-mon[32789]: osd.1 [v2:172.21.15.150:6800/3383948110,v1:172.21.15.150:6801/3383948110] boot 2024-03-04T08:50:32.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:31 smithi071 ceph-mon[32789]: osdmap e13: 2 total, 2 up, 2 in 2024-03-04T08:50:32.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:31 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T08:50:32.850 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.853+0000 7fa328ea4700 1 -- 172.21.15.158:0/1846645767 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa324100ff0 msgr2=0x7fa324101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:50:32.850 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.853+0000 7fa328ea4700 1 --2- 172.21.15.158:0/1846645767 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa324100ff0 0x7fa324101410 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa314009990 tx=0x7fa314009ca0 comp rx=0 tx=0).stop 2024-03-04T08:50:32.850 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.853+0000 7fa328ea4700 1 -- 172.21.15.158:0/1846645767 shutdown_connections 2024-03-04T08:50:32.850 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.853+0000 7fa328ea4700 1 --2- 172.21.15.158:0/1846645767 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa324100ff0 0x7fa324101410 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:50:32.850 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.853+0000 7fa328ea4700 1 --2- 172.21.15.158:0/1846645767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa324102b90 0x7fa324109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:50:32.850 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.853+0000 7fa328ea4700 1 --2- 172.21.15.158:0/1846645767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3241021f0 0x7fa324102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:50:32.850 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.853+0000 7fa328ea4700 1 -- 172.21.15.158:0/1846645767 >> 172.21.15.158:0/1846645767 conn(0x7fa3240fc5b0 msgr2=0x7fa3240fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:50:32.850 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.853+0000 7fa328ea4700 1 -- 172.21.15.158:0/1846645767 shutdown_connections 2024-03-04T08:50:32.850 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.853+0000 7fa328ea4700 1 -- 172.21.15.158:0/1846645767 wait complete. 2024-03-04T08:50:32.855 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.854+0000 7fa328ea4700 1 Processor -- start 2024-03-04T08:50:32.855 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.854+0000 7fa328ea4700 1 -- start start 2024-03-04T08:50:32.856 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.854+0000 7fa328ea4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa324100ff0 0x7fa32406b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:50:32.856 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.854+0000 7fa328ea4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3241021f0 0x7fa32419d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:50:32.856 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.854+0000 7fa328ea4700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa324102b90 0x7fa32419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:50:32.856 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.854+0000 7fa328ea4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32410cbc0 con 0x7fa324100ff0 2024-03-04T08:50:32.856 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.854+0000 7fa328ea4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32410ca40 con 0x7fa3241021f0 2024-03-04T08:50:32.856 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.854+0000 7fa328ea4700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32410cd40 con 0x7fa324102b90 2024-03-04T08:50:32.856 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.855+0000 7fa322d9d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa324102b90 0x7fa32419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:50:32.856 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.855+0000 7fa322d9d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa324102b90 0x7fa32419f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.158:57580/0 (socket says 172.21.15.158:57580) 2024-03-04T08:50:32.856 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.855+0000 7fa322d9d700 1 -- 172.21.15.158:0/4086718764 learned_addr learned my addr 172.21.15.158:0/4086718764 (peer_addr_for_me v2:172.21.15.158:0/0) 2024-03-04T08:50:32.856 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.855+0000 7fa32259c700 1 --2- 172.21.15.158:0/4086718764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa324100ff0 0x7fa32406b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:50:32.856 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.855+0000 7fa321d9b700 1 --2- 172.21.15.158:0/4086718764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3241021f0 0x7fa32419d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:50:32.857 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.855+0000 7fa322d9d700 1 -- 172.21.15.158:0/4086718764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3241021f0 msgr2=0x7fa32419d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:50:32.857 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.855+0000 7fa322d9d700 1 --2- 172.21.15.158:0/4086718764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3241021f0 0x7fa32419d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:50:32.857 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.855+0000 7fa322d9d700 1 -- 172.21.15.158:0/4086718764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa324100ff0 msgr2=0x7fa32406b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:50:32.857 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.855+0000 7fa322d9d700 1 --2- 172.21.15.158:0/4086718764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa324100ff0 0x7fa32406b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:50:32.857 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.855+0000 7fa322d9d700 1 -- 172.21.15.158:0/4086718764 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32419ffa0 con 0x7fa324102b90 2024-03-04T08:50:32.857 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.855+0000 7fa322d9d700 1 --2- 172.21.15.158:0/4086718764 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa324102b90 0x7fa32419f6d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa31c007ee0 tx=0x7fa31c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:50:32.857 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.855+0000 7fa3137fe700 1 -- 172.21.15.158:0/4086718764 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31c004bb0 con 0x7fa324102b90 2024-03-04T08:50:32.857 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.855+0000 7fa328ea4700 1 -- 172.21.15.158:0/4086718764 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3241a0290 con 0x7fa324102b90 2024-03-04T08:50:32.857 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.855+0000 7fa3137fe700 1 -- 172.21.15.158:0/4086718764 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa31c004d50 con 0x7fa324102b90 2024-03-04T08:50:32.857 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.855+0000 7fa3137fe700 1 -- 172.21.15.158:0/4086718764 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31c005960 con 0x7fa324102b90 2024-03-04T08:50:32.857 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.855+0000 7fa328ea4700 1 -- 172.21.15.158:0/4086718764 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3241a0780 con 0x7fa324102b90 2024-03-04T08:50:32.857 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.856+0000 7fa328ea4700 1 -- 172.21.15.158:0/4086718764 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa324068d80 con 0x7fa324102b90 2024-03-04T08:50:32.858 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.857+0000 7fa3137fe700 1 -- 172.21.15.158:0/4086718764 <== mon.1 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 15) v1 ==== 136700+0+0 (secure 0 0 0) 0x7fa31c01a020 con 0x7fa324102b90 2024-03-04T08:50:32.858 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.858+0000 7fa3137fe700 1 --2- 172.21.15.158:0/4086718764 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa30c0a1d80 0x7fa30c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:50:32.858 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.858+0000 7fa3137fe700 1 -- 172.21.15.158:0/4086718764 <== mon.1 v2:172.21.15.158:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7fa31c0cc270 con 0x7fa324102b90 2024-03-04T08:50:32.858 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.858+0000 7fa32259c700 1 --2- 172.21.15.158:0/4086718764 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa30c0a1d80 0x7fa30c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:50:32.858 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.859+0000 7fa32259c700 1 --2- 172.21.15.158:0/4086718764 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa30c0a1d80 0x7fa30c0a4240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa314003890 tx=0x7fa314005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:50:32.859 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:32.862+0000 7fa3137fe700 1 -- 172.21.15.158:0/4086718764 <== mon.1 v2:172.21.15.158:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa31c010070 con 0x7fa324102b90 2024-03-04T08:50:33.061 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:50:33.065+0000 7fa328ea4700 1 -- 172.21.15.158:0/4086718764 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa324062500 con 0x7fa30c0a1d80 2024-03-04T08:50:33.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:32 smithi071 ceph-mon[32789]: Detected new or changed devices on smithi150 2024-03-04T08:50:33.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:33.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:33.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:50:33.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:32 smithi071 ceph-mon[32789]: Adjusting osd_memory_target on smithi150 to 17080M 2024-03-04T08:50:33.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:33.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:50:33.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:50:33.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:33.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:50:33.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:32 smithi071 ceph-mon[32789]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:33.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:50:33.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:50:33.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:32 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:33.181 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:32 smithi150 ceph-mon[36807]: Detected new or changed devices on smithi150 2024-03-04T08:50:33.181 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:33.181 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:33.181 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:50:33.181 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:32 smithi150 ceph-mon[36807]: Adjusting osd_memory_target on smithi150 to 17080M 2024-03-04T08:50:33.181 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:33.182 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:50:33.182 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:50:33.182 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:33.182 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:50:33.182 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:32 smithi150 ceph-mon[36807]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:33.182 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:50:33.182 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:50:33.182 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:32 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:33.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:32 smithi158 ceph-mon[37241]: Detected new or changed devices on smithi150 2024-03-04T08:50:33.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:33.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:33.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:50:33.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:32 smithi158 ceph-mon[37241]: Adjusting osd_memory_target on smithi150 to 17080M 2024-03-04T08:50:33.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:33.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:50:33.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:50:33.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:33.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:50:33.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:32 smithi158 ceph-mon[37241]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:33.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:50:33.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:50:33.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:32 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:50:34.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:33 smithi071 ceph-mon[32789]: osdmap e14: 2 total, 2 up, 2 in 2024-03-04T08:50:34.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:33 smithi071 ceph-mon[32789]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:50:34.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:33 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:50:34.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:33 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:50:34.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:33 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:50:34.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:33 smithi150 ceph-mon[36807]: osdmap e14: 2 total, 2 up, 2 in 2024-03-04T08:50:34.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:33 smithi150 ceph-mon[36807]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:50:34.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:33 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:50:34.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:33 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:50:34.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:33 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:50:34.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:33 smithi158 ceph-mon[37241]: osdmap e14: 2 total, 2 up, 2 in 2024-03-04T08:50:34.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:33 smithi158 ceph-mon[37241]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:50:34.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:33 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:50:34.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:33 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:50:34.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:33 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:50:35.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:34 smithi071 ceph-mon[32789]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:35.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:34 smithi150 ceph-mon[36807]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:35.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:34 smithi158 ceph-mon[37241]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:37.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:37 smithi071 ceph-mon[32789]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:38.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:37 smithi150 ceph-mon[36807]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:38.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:37 smithi158 ceph-mon[37241]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:39.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:39 smithi071 ceph-mon[32789]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:40.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:39 smithi150 ceph-mon[36807]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:40.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:39 smithi158 ceph-mon[37241]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:41.714 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:41 smithi158 ceph-mon[37241]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:41.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:41 smithi150 ceph-mon[36807]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:41.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:41 smithi071 ceph-mon[32789]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:43.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:43 smithi071 ceph-mon[32789]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:43.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:43 smithi071 ceph-mon[32789]: from='client.? 172.21.15.158:0/566880143' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b88c3082-93a9-4c9b-9221-a7105a44c91a"}]: dispatch 2024-03-04T08:50:43.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:43 smithi071 ceph-mon[32789]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b88c3082-93a9-4c9b-9221-a7105a44c91a"}]: dispatch 2024-03-04T08:50:43.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:43 smithi071 ceph-mon[32789]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b88c3082-93a9-4c9b-9221-a7105a44c91a"}]': finished 2024-03-04T08:50:43.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:43 smithi071 ceph-mon[32789]: osdmap e15: 3 total, 2 up, 3 in 2024-03-04T08:50:43.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:43 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T08:50:43.946 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:43 smithi158 ceph-mon[37241]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:43.946 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:43 smithi158 ceph-mon[37241]: from='client.? 172.21.15.158:0/566880143' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b88c3082-93a9-4c9b-9221-a7105a44c91a"}]: dispatch 2024-03-04T08:50:43.946 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:43 smithi158 ceph-mon[37241]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b88c3082-93a9-4c9b-9221-a7105a44c91a"}]: dispatch 2024-03-04T08:50:43.946 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:43 smithi158 ceph-mon[37241]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b88c3082-93a9-4c9b-9221-a7105a44c91a"}]': finished 2024-03-04T08:50:43.946 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:43 smithi158 ceph-mon[37241]: osdmap e15: 3 total, 2 up, 3 in 2024-03-04T08:50:43.946 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:43 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T08:50:44.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:43 smithi150 ceph-mon[36807]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:44.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:43 smithi150 ceph-mon[36807]: from='client.? 172.21.15.158:0/566880143' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b88c3082-93a9-4c9b-9221-a7105a44c91a"}]: dispatch 2024-03-04T08:50:44.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:43 smithi150 ceph-mon[36807]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b88c3082-93a9-4c9b-9221-a7105a44c91a"}]: dispatch 2024-03-04T08:50:44.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:43 smithi150 ceph-mon[36807]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b88c3082-93a9-4c9b-9221-a7105a44c91a"}]': finished 2024-03-04T08:50:44.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:43 smithi150 ceph-mon[36807]: osdmap e15: 3 total, 2 up, 3 in 2024-03-04T08:50:44.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:43 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T08:50:44.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:44 smithi150 ceph-mon[36807]: from='client.? 172.21.15.158:0/1760386595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:50:45.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:44 smithi158 ceph-mon[37241]: from='client.? 172.21.15.158:0/1760386595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:50:45.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:44 smithi071 ceph-mon[32789]: from='client.? 172.21.15.158:0/1760386595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:50:46.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:45 smithi150 ceph-mon[36807]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:46.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:45 smithi158 ceph-mon[37241]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:46.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:45 smithi071 ceph-mon[32789]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:47.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:47 smithi158 ceph-mon[37241]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:48.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:47 smithi150 ceph-mon[36807]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:48.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:47 smithi071 ceph-mon[32789]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:50.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:49 smithi150 ceph-mon[36807]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:50.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:49 smithi158 ceph-mon[37241]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:50.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:49 smithi071 ceph-mon[32789]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:51.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:50 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-04T08:50:51.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:50 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:50:51.041 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:50 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-04T08:50:51.041 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:50 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:50:51.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:50 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-04T08:50:51.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:50 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:50:51.976 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:51 smithi158 ceph-mon[37241]: Deploying daemon osd.2 on smithi158 2024-03-04T08:50:51.976 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:51 smithi158 ceph-mon[37241]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:52.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:51 smithi150 ceph-mon[36807]: Deploying daemon osd.2 on smithi158 2024-03-04T08:50:52.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:51 smithi150 ceph-mon[36807]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:52.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:51 smithi071 ceph-mon[32789]: Deploying daemon osd.2 on smithi158 2024-03-04T08:50:52.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:51 smithi071 ceph-mon[32789]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:54.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:53 smithi150 ceph-mon[36807]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:54.042 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:53 smithi158 ceph-mon[37241]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:54.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:53 smithi071 ceph-mon[32789]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:56.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:55 smithi150 ceph-mon[36807]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:56.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:55 smithi158 ceph-mon[37241]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:56.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:55 smithi071 ceph-mon[32789]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:58.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:57 smithi150 ceph-mon[36807]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:58.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:57 smithi158 ceph-mon[37241]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:50:58.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:57 smithi071 ceph-mon[32789]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:00.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:50:59 smithi150 ceph-mon[36807]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:00.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:50:59 smithi158 ceph-mon[37241]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:00.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:50:59 smithi071 ceph-mon[32789]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:02.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:01 smithi150 ceph-mon[36807]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:02.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:01 smithi158 ceph-mon[37241]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:02.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:01 smithi071 ceph-mon[32789]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:03.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:03 smithi150 ceph-mon[36807]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:03.535 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:03 smithi158 ceph-mon[37241]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:03.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:03 smithi071 ceph-mon[32789]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:05.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:04 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:05.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:04 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:05.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:04 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:51:05.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:04 smithi150 ceph-mon[36807]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:05.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:04 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:05.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:04 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:05.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:04 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:51:05.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:04 smithi158 ceph-mon[37241]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:05.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:04 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:05.449 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:04 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:05.449 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:04 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:51:05.449 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:04 smithi071 ceph-mon[32789]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:08.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:07 smithi150 ceph-mon[36807]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:08.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:07 smithi158 ceph-mon[37241]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:08.125 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:07 smithi071 ceph-mon[32789]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:09.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:08 smithi150 ceph-mon[36807]: from='osd.2 [v2:172.21.15.158:6800/586944144,v1:172.21.15.158:6801/586944144]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-04T08:51:09.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:08 smithi150 ceph-mon[36807]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-04T08:51:09.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:08 smithi158 ceph-mon[37241]: from='osd.2 [v2:172.21.15.158:6800/586944144,v1:172.21.15.158:6801/586944144]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-04T08:51:09.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:08 smithi158 ceph-mon[37241]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-04T08:51:09.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:08 smithi071 ceph-mon[32789]: from='osd.2 [v2:172.21.15.158:6800/586944144,v1:172.21.15.158:6801/586944144]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-04T08:51:09.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:08 smithi071 ceph-mon[32789]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-04T08:51:10.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:09 smithi150 ceph-mon[36807]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:10.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:09 smithi150 ceph-mon[36807]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-04T08:51:10.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:09 smithi150 ceph-mon[36807]: from='osd.2 [v2:172.21.15.158:6800/586944144,v1:172.21.15.158:6801/586944144]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-03-04T08:51:10.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:09 smithi150 ceph-mon[36807]: osdmap e16: 3 total, 2 up, 3 in 2024-03-04T08:51:10.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:09 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T08:51:10.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:09 smithi150 ceph-mon[36807]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-03-04T08:51:10.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:09 smithi158 ceph-mon[37241]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:10.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:09 smithi071 ceph-mon[32789]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:10.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:09 smithi071 ceph-mon[32789]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-04T08:51:10.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:09 smithi071 ceph-mon[32789]: from='osd.2 [v2:172.21.15.158:6800/586944144,v1:172.21.15.158:6801/586944144]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-03-04T08:51:10.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:09 smithi071 ceph-mon[32789]: osdmap e16: 3 total, 2 up, 3 in 2024-03-04T08:51:10.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:09 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T08:51:10.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:09 smithi071 ceph-mon[32789]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-03-04T08:51:12.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:12 smithi150 ceph-mon[36807]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]': finished 2024-03-04T08:51:12.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:12 smithi150 ceph-mon[36807]: osdmap e17: 3 total, 2 up, 3 in 2024-03-04T08:51:12.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:12 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T08:51:12.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:12 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T08:51:12.283 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:11 smithi158 ceph-mon[37241]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-04T08:51:12.284 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:11 smithi158 ceph-mon[37241]: from='osd.2 [v2:172.21.15.158:6800/586944144,v1:172.21.15.158:6801/586944144]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-03-04T08:51:12.284 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:11 smithi158 ceph-mon[37241]: osdmap e16: 3 total, 2 up, 3 in 2024-03-04T08:51:12.284 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:11 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T08:51:12.284 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:11 smithi158 ceph-mon[37241]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-03-04T08:51:12.284 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:12 smithi158 ceph-mon[37241]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]': finished 2024-03-04T08:51:12.284 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:12 smithi158 ceph-mon[37241]: osdmap e17: 3 total, 2 up, 3 in 2024-03-04T08:51:12.284 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:12 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T08:51:12.284 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:12 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T08:51:12.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:12 smithi071 ceph-mon[32789]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]': finished 2024-03-04T08:51:12.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:12 smithi071 ceph-mon[32789]: osdmap e17: 3 total, 2 up, 3 in 2024-03-04T08:51:12.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:12 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T08:51:12.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:12 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T08:51:13.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:13 smithi150 ceph-mon[36807]: purged_snaps scrub starts 2024-03-04T08:51:13.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:13 smithi150 ceph-mon[36807]: purged_snaps scrub ok 2024-03-04T08:51:13.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:13 smithi150 ceph-mon[36807]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:13.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:13 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T08:51:13.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:13 smithi150 ceph-mon[36807]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:13.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:13 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T08:51:13.287 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:13 smithi158 ceph-mon[37241]: purged_snaps scrub starts 2024-03-04T08:51:13.287 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:13 smithi158 ceph-mon[37241]: purged_snaps scrub ok 2024-03-04T08:51:13.287 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:13 smithi158 ceph-mon[37241]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:13.287 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:13 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T08:51:13.287 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:13 smithi158 ceph-mon[37241]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:13.287 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:13 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T08:51:13.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:13 smithi071 ceph-mon[32789]: purged_snaps scrub starts 2024-03-04T08:51:13.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:13 smithi071 ceph-mon[32789]: purged_snaps scrub ok 2024-03-04T08:51:13.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:13 smithi071 ceph-mon[32789]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:13.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:13 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T08:51:13.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:13 smithi071 ceph-mon[32789]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-03-04T08:51:13.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:13 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T08:51:14.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:14 smithi150 ceph-mon[36807]: osd.2 [v2:172.21.15.158:6800/586944144,v1:172.21.15.158:6801/586944144] boot 2024-03-04T08:51:14.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:14 smithi150 ceph-mon[36807]: osdmap e18: 3 total, 3 up, 3 in 2024-03-04T08:51:14.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:14 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T08:51:14.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:14 smithi158 ceph-mon[37241]: osd.2 [v2:172.21.15.158:6800/586944144,v1:172.21.15.158:6801/586944144] boot 2024-03-04T08:51:14.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:14 smithi158 ceph-mon[37241]: osdmap e18: 3 total, 3 up, 3 in 2024-03-04T08:51:14.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:14 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T08:51:14.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:14 smithi071 ceph-mon[32789]: osd.2 [v2:172.21.15.158:6800/586944144,v1:172.21.15.158:6801/586944144] boot 2024-03-04T08:51:14.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:14 smithi071 ceph-mon[32789]: osdmap e18: 3 total, 3 up, 3 in 2024-03-04T08:51:14.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:14 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T08:51:15.176 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:15 smithi158 ceph-mon[37241]: osdmap e19: 3 total, 3 up, 3 in 2024-03-04T08:51:15.177 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:15.177 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:15.177 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:15 smithi158 ceph-mon[37241]: pgmap v95: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:15.177 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-04T08:51:15.177 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:51:15.177 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:51:15.177 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:15.232 INFO:teuthology.orchestra.run.smithi158.stdout:Created osd(s) 2 on host 'smithi158' 2024-03-04T08:51:15.233 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:51:15.236+0000 7fa3137fe700 1 -- 172.21.15.158:0/4086718764 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa324062500 con 0x7fa30c0a1d80 2024-03-04T08:51:15.234 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:51:15.238+0000 7fa328ea4700 1 -- 172.21.15.158:0/4086718764 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa30c0a1d80 msgr2=0x7fa30c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:51:15.235 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:51:15.238+0000 7fa328ea4700 1 --2- 172.21.15.158:0/4086718764 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa30c0a1d80 0x7fa30c0a4240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa314003890 tx=0x7fa314005fd0 comp rx=0 tx=0).stop 2024-03-04T08:51:15.235 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:51:15.238+0000 7fa328ea4700 1 -- 172.21.15.158:0/4086718764 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa324102b90 msgr2=0x7fa32419f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:51:15.235 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:51:15.238+0000 7fa328ea4700 1 --2- 172.21.15.158:0/4086718764 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa324102b90 0x7fa32419f6d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa31c007ee0 tx=0x7fa31c00d3b0 comp rx=0 tx=0).stop 2024-03-04T08:51:15.235 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:51:15.239+0000 7fa328ea4700 1 -- 172.21.15.158:0/4086718764 shutdown_connections 2024-03-04T08:51:15.235 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:51:15.239+0000 7fa328ea4700 1 --2- 172.21.15.158:0/4086718764 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa30c0a1d80 0x7fa30c0a4240 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:51:15.235 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:51:15.239+0000 7fa328ea4700 1 --2- 172.21.15.158:0/4086718764 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa324102b90 0x7fa32419f6d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:51:15.235 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:51:15.239+0000 7fa328ea4700 1 --2- 172.21.15.158:0/4086718764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa324100ff0 0x7fa32406b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:51:15.235 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:51:15.239+0000 7fa328ea4700 1 --2- 172.21.15.158:0/4086718764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3241021f0 0x7fa32419d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:51:15.235 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:51:15.239+0000 7fa328ea4700 1 -- 172.21.15.158:0/4086718764 >> 172.21.15.158:0/4086718764 conn(0x7fa3240fc5b0 msgr2=0x7fa3241055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:51:15.235 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:51:15.239+0000 7fa328ea4700 1 -- 172.21.15.158:0/4086718764 shutdown_connections 2024-03-04T08:51:15.235 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:51:15.239+0000 7fa328ea4700 1 -- 172.21.15.158:0/4086718764 wait complete. 2024-03-04T08:51:15.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:15 smithi150 ceph-mon[36807]: osdmap e19: 3 total, 3 up, 3 in 2024-03-04T08:51:15.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:15 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:15.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:15 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:15.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:15 smithi150 ceph-mon[36807]: pgmap v95: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:15.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:15 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-04T08:51:15.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:15 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:51:15.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:15 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:51:15.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:15 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:15.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:15 smithi071 ceph-mon[32789]: osdmap e19: 3 total, 3 up, 3 in 2024-03-04T08:51:15.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:15 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:15.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:15 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:15.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:15 smithi071 ceph-mon[32789]: pgmap v95: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:15.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:15 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-04T08:51:15.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:15 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:51:15.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:15 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:51:15.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:15 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:15.818 DEBUG:teuthology.orchestra.run.smithi158:osd.2> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.2.service 2024-03-04T08:51:15.820 INFO:tasks.cephadm:Deploying osd.3 on smithi071 with /dev/vg_nvme/lv_3... 2024-03-04T08:51:15.820 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-04T08:51:16.057 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:51:16.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:16 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:16.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:16 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:16.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:16 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:51:16.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:16 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-04T08:51:16.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:16 smithi158 ceph-mon[37241]: osdmap e20: 3 total, 3 up, 3 in 2024-03-04T08:51:16.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:16 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-04T08:51:16.431 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:16 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:16.431 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:16 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:16.431 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:16 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:51:16.431 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:16 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-04T08:51:16.431 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:16 smithi071 ceph-mon[32789]: osdmap e20: 3 total, 3 up, 3 in 2024-03-04T08:51:16.431 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:16 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-04T08:51:16.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:16 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:16.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:16 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:16.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:16 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:51:16.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:16 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-04T08:51:16.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:16 smithi150 ceph-mon[36807]: osdmap e20: 3 total, 3 up, 3 in 2024-03-04T08:51:16.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:16 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-04T08:51:17.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:17 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-04T08:51:17.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:17 smithi158 ceph-mon[37241]: osdmap e21: 3 total, 3 up, 3 in 2024-03-04T08:51:17.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:17 smithi158 ceph-mon[37241]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:17.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:17 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-04T08:51:17.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:17 smithi071 ceph-mon[32789]: osdmap e21: 3 total, 3 up, 3 in 2024-03-04T08:51:17.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:17 smithi071 ceph-mon[32789]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:17.672 INFO:journalctl@ceph.osd.0.smithi071.stdout:Mar 04 08:51:17 smithi071 sudo[51552]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-04T08:51:17.672 INFO:journalctl@ceph.osd.0.smithi071.stdout:Mar 04 08:51:17 smithi071 sudo[51552]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T08:51:17.673 INFO:journalctl@ceph.osd.0.smithi071.stdout:Mar 04 08:51:17 smithi071 sudo[51552]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T08:51:17.673 INFO:journalctl@ceph.osd.0.smithi071.stdout:Mar 04 08:51:17 smithi071 sudo[51552]: pam_unix(sudo:session): session closed for user root 2024-03-04T08:51:17.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:17 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-04T08:51:17.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:17 smithi150 ceph-mon[36807]: osdmap e21: 3 total, 3 up, 3 in 2024-03-04T08:51:17.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:17 smithi150 ceph-mon[36807]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:17.964 INFO:journalctl@ceph.osd.0.smithi071.stdout:Mar 04 08:51:17 smithi071 sudo[51560]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-04T08:51:17.965 INFO:journalctl@ceph.osd.0.smithi071.stdout:Mar 04 08:51:17 smithi071 sudo[51560]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T08:51:17.965 INFO:journalctl@ceph.osd.0.smithi071.stdout:Mar 04 08:51:17 smithi071 sudo[51560]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T08:51:17.965 INFO:journalctl@ceph.osd.0.smithi071.stdout:Mar 04 08:51:17 smithi071 sudo[51560]: pam_unix(sudo:session): session closed for user root 2024-03-04T08:51:17.965 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-04T08:51:17.965 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-04T08:51:17.965 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-03-04T08:51:17.965 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-03-04T08:51:17.965 INFO:teuthology.orchestra.run.smithi071.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0116402 s, 901 MB/s 2024-03-04T08:51:17.965 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-03-04T08:51:18.195 INFO:journalctl@ceph.osd.1.smithi150.stdout:Mar 04 08:51:17 smithi150 sudo[45371]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-04T08:51:18.195 INFO:journalctl@ceph.osd.1.smithi150.stdout:Mar 04 08:51:17 smithi150 sudo[45371]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T08:51:18.195 INFO:journalctl@ceph.osd.1.smithi150.stdout:Mar 04 08:51:17 smithi150 sudo[45371]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T08:51:18.195 INFO:journalctl@ceph.osd.1.smithi150.stdout:Mar 04 08:51:17 smithi150 sudo[45371]: pam_unix(sudo:session): session closed for user root 2024-03-04T08:51:18.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:18 smithi150 ceph-mon[36807]: osdmap e22: 3 total, 3 up, 3 in 2024-03-04T08:51:18.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:18 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:18.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:18 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:18.530 INFO:journalctl@ceph.osd.1.smithi150.stdout:Mar 04 08:51:18 smithi150 sudo[45374]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-04T08:51:18.530 INFO:journalctl@ceph.osd.1.smithi150.stdout:Mar 04 08:51:18 smithi150 sudo[45374]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T08:51:18.530 INFO:journalctl@ceph.osd.1.smithi150.stdout:Mar 04 08:51:18 smithi150 sudo[45374]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T08:51:18.530 INFO:journalctl@ceph.osd.1.smithi150.stdout:Mar 04 08:51:18 smithi150 sudo[45374]: pam_unix(sudo:session): session closed for user root 2024-03-04T08:51:18.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:18 smithi158 ceph-mon[37241]: osdmap e22: 3 total, 3 up, 3 in 2024-03-04T08:51:18.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:18 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:18.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:18 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:18.553 INFO:journalctl@ceph.osd.2.smithi158.stdout:Mar 04 08:51:18 smithi158 sudo[46263]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-04T08:51:18.553 INFO:journalctl@ceph.osd.2.smithi158.stdout:Mar 04 08:51:18 smithi158 sudo[46263]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T08:51:18.553 INFO:journalctl@ceph.osd.2.smithi158.stdout:Mar 04 08:51:18 smithi158 sudo[46263]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T08:51:18.553 INFO:journalctl@ceph.osd.2.smithi158.stdout:Mar 04 08:51:18 smithi158 sudo[46263]: pam_unix(sudo:session): session closed for user root 2024-03-04T08:51:18.596 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:18 smithi071 ceph-mon[32789]: osdmap e22: 3 total, 3 up, 3 in 2024-03-04T08:51:18.596 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:18 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:18.596 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:18 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:18.624 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi071:vg_nvme/lv_3 2024-03-04T08:51:18.891 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:51:19.052 INFO:journalctl@ceph.osd.2.smithi158.stdout:Mar 04 08:51:18 smithi158 sudo[46266]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-04T08:51:19.052 INFO:journalctl@ceph.osd.2.smithi158.stdout:Mar 04 08:51:18 smithi158 sudo[46266]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T08:51:19.052 INFO:journalctl@ceph.osd.2.smithi158.stdout:Mar 04 08:51:18 smithi158 sudo[46266]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T08:51:19.052 INFO:journalctl@ceph.osd.2.smithi158.stdout:Mar 04 08:51:18 smithi158 sudo[46266]: pam_unix(sudo:session): session closed for user root 2024-03-04T08:51:19.283 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 sudo[51797]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-04T08:51:19.283 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 sudo[51797]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T08:51:19.283 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 sudo[51797]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T08:51:19.283 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 sudo[51797]: pam_unix(sudo:session): session closed for user root 2024-03-04T08:51:19.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 sudo[51822]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-04T08:51:19.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 sudo[51822]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T08:51:19.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 sudo[51822]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T08:51:19.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 sudo[51822]: pam_unix(sudo:session): session closed for user root 2024-03-04T08:51:19.776 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 sudo[46301]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-04T08:51:19.776 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 sudo[46301]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T08:51:19.777 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 sudo[46301]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T08:51:19.777 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 sudo[46301]: pam_unix(sudo:session): session closed for user root 2024-03-04T08:51:19.777 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 sudo[46304]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-04T08:51:19.777 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 sudo[46304]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T08:51:19.777 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 sudo[46304]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T08:51:19.962 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 ceph-mon[32789]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-04T08:51:19.963 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 ceph-mon[32789]: pgmap v100: 1 pgs: 1 creating+activating; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:19.963 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 ceph-mon[32789]: Detected new or changed devices on smithi158 2024-03-04T08:51:19.963 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:19.963 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:19.963 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:51:19.963 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 ceph-mon[32789]: Adjusting osd_memory_target on smithi158 to 17080M 2024-03-04T08:51:19.963 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:19.963 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:51:19.963 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:51:19.963 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:19.963 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:19.963 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 ceph-mon[32789]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T08:51:19.964 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 ceph-mon[32789]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T08:51:19.964 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T08:51:19.964 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:51:19.964 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:51:19.964 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T08:51:19.964 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:51:19.964 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:19 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:51:20.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 sudo[46304]: pam_unix(sudo:session): session closed for user root 2024-03-04T08:51:20.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 ceph-mon[37241]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-04T08:51:20.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 ceph-mon[37241]: pgmap v100: 1 pgs: 1 creating+activating; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:20.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 ceph-mon[37241]: Detected new or changed devices on smithi158 2024-03-04T08:51:20.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:20.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:20.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:51:20.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 ceph-mon[37241]: Adjusting osd_memory_target on smithi158 to 17080M 2024-03-04T08:51:20.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:20.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:51:20.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:51:20.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:20.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:20.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 ceph-mon[37241]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T08:51:20.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 ceph-mon[37241]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T08:51:20.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T08:51:20.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:51:20.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:51:20.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T08:51:20.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:51:20.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:19 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:51:20.274 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 sudo[45409]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-04T08:51:20.275 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 sudo[45409]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T08:51:20.275 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 sudo[45409]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T08:51:20.275 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 sudo[45409]: pam_unix(sudo:session): session closed for user root 2024-03-04T08:51:20.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.474+0000 7f595dcae700 1 -- 172.21.15.71:0/3855889832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59581020e0 msgr2=0x7f5958102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:51:20.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.474+0000 7f595dcae700 1 --2- 172.21.15.71:0/3855889832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59581020e0 0x7f5958102500 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f59480099c0 tx=0x7f5948009cd0 comp rx=0 tx=0).stop 2024-03-04T08:51:20.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.474+0000 7f595dcae700 1 -- 172.21.15.71:0/3855889832 shutdown_connections 2024-03-04T08:51:20.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.474+0000 7f595dcae700 1 --2- 172.21.15.71:0/3855889832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5958103c80 0x7f595810a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:51:20.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.474+0000 7f595dcae700 1 --2- 172.21.15.71:0/3855889832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59581020e0 0x7f5958102500 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:51:20.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.474+0000 7f595dcae700 1 --2- 172.21.15.71:0/3855889832 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59581032e0 0x7f5958103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:51:20.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.474+0000 7f595dcae700 1 -- 172.21.15.71:0/3855889832 >> 172.21.15.71:0/3855889832 conn(0x7f59580fd6a0 msgr2=0x7f59580ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:51:20.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.474+0000 7f595dcae700 1 -- 172.21.15.71:0/3855889832 shutdown_connections 2024-03-04T08:51:20.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.474+0000 7f595dcae700 1 -- 172.21.15.71:0/3855889832 wait complete. 2024-03-04T08:51:20.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.475+0000 7f595dcae700 1 Processor -- start 2024-03-04T08:51:20.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.475+0000 7f595dcae700 1 -- start start 2024-03-04T08:51:20.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.475+0000 7f595dcae700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59581020e0 0x7f5958198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:51:20.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.475+0000 7f595dcae700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59581032e0 0x7f59581989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:51:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.475+0000 7f595dcae700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5958103c80 0x7f595819fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:51:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.475+0000 7f595dcae700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595810dda0 con 0x7f5958103c80 2024-03-04T08:51:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.475+0000 7f595dcae700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595810dc20 con 0x7f59581020e0 2024-03-04T08:51:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.475+0000 7f595dcae700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595810df20 con 0x7f59581032e0 2024-03-04T08:51:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.475+0000 7f5957fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5958103c80 0x7f595819fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:51:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.475+0000 7f5957fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5958103c80 0x7f595819fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41494/0 (socket says 172.21.15.71:41494) 2024-03-04T08:51:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.475+0000 7f5957fff700 1 -- 172.21.15.71:0/2065904144 learned_addr learned my addr 172.21.15.71:0/2065904144 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:51:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.475+0000 7f5956ffd700 1 --2- 172.21.15.71:0/2065904144 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59581032e0 0x7f59581989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:51:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.475+0000 7f59577fe700 1 --2- 172.21.15.71:0/2065904144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59581020e0 0x7f5958198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:51:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.476+0000 7f5957fff700 1 -- 172.21.15.71:0/2065904144 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59581032e0 msgr2=0x7f59581989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:51:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.476+0000 7f5957fff700 1 --2- 172.21.15.71:0/2065904144 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59581032e0 0x7f59581989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:51:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.476+0000 7f5957fff700 1 -- 172.21.15.71:0/2065904144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59581020e0 msgr2=0x7f5958198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:51:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.476+0000 7f5957fff700 1 --2- 172.21.15.71:0/2065904144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59581020e0 0x7f5958198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:51:20.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.476+0000 7f5957fff700 1 -- 172.21.15.71:0/2065904144 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59581a00c0 con 0x7f5958103c80 2024-03-04T08:51:20.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.476+0000 7f5957fff700 1 --2- 172.21.15.71:0/2065904144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5958103c80 0x7f595819fa20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f595000b9a0 tx=0x7f595000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:51:20.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.476+0000 7f5954ff9700 1 -- 172.21.15.71:0/2065904144 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5950019070 con 0x7f5958103c80 2024-03-04T08:51:20.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.476+0000 7f595dcae700 1 -- 172.21.15.71:0/2065904144 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59581a03b0 con 0x7f5958103c80 2024-03-04T08:51:20.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.476+0000 7f5954ff9700 1 -- 172.21.15.71:0/2065904144 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5950014440 con 0x7f5958103c80 2024-03-04T08:51:20.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.476+0000 7f5954ff9700 1 -- 172.21.15.71:0/2065904144 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59500075d0 con 0x7f5958103c80 2024-03-04T08:51:20.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.476+0000 7f595dcae700 1 -- 172.21.15.71:0/2065904144 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59581a0970 con 0x7f5958103c80 2024-03-04T08:51:20.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.477+0000 7f595dcae700 1 -- 172.21.15.71:0/2065904144 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5958068d80 con 0x7f5958103c80 2024-03-04T08:51:20.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.479+0000 7f5954ff9700 1 -- 172.21.15.71:0/2065904144 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f595001c070 con 0x7f5958103c80 2024-03-04T08:51:20.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.479+0000 7f5954ff9700 1 --2- 172.21.15.71:0/2065904144 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f59400a1e90 0x7f59400a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:51:20.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.479+0000 7f59577fe700 1 --2- 172.21.15.71:0/2065904144 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f59400a1e90 0x7f59400a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:51:20.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.480+0000 7f5954ff9700 1 -- 172.21.15.71:0/2065904144 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 3237+0+0 (secure 0 0 0) 0x7f59500cc340 con 0x7f5958103c80 2024-03-04T08:51:20.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.480+0000 7f59577fe700 1 --2- 172.21.15.71:0/2065904144 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f59400a1e90 0x7f59400a4350 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f594800b580 tx=0x7f5948005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:51:20.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.483+0000 7f5954ff9700 1 -- 172.21.15.71:0/2065904144 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f595009ab30 con 0x7f5958103c80 2024-03-04T08:51:20.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 sudo[45412]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-04T08:51:20.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 sudo[45412]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T08:51:20.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 sudo[45412]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T08:51:20.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 sudo[45412]: pam_unix(sudo:session): session closed for user root 2024-03-04T08:51:20.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-04T08:51:20.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: pgmap v100: 1 pgs: 1 creating+activating; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:20.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: Detected new or changed devices on smithi158 2024-03-04T08:51:20.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:20.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:20.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:51:20.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: Adjusting osd_memory_target on smithi158 to 17080M 2024-03-04T08:51:20.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:20.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:51:20.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:51:20.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:20.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:20.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T08:51:20.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T08:51:20.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T08:51:20.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:51:20.532 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:51:20.532 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T08:51:20.532 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:51:20.532 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:51:20.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:20.686+0000 7f595dcae700 1 -- 172.21.15.71:0/2065904144 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f5958062500 con 0x7f59400a1e90 2024-03-04T08:51:21.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:20 smithi158 ceph-mon[37241]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T08:51:21.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:20 smithi158 ceph-mon[37241]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T08:51:21.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:20 smithi158 ceph-mon[37241]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T08:51:21.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:20 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T08:51:21.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:20 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:51:21.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:20 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:51:21.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:20 smithi158 ceph-mon[37241]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T08:51:21.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:20 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:51:21.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:20 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:51:21.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:20 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:51:21.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:20 smithi071 ceph-mon[32789]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T08:51:21.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:20 smithi071 ceph-mon[32789]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T08:51:21.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:20 smithi071 ceph-mon[32789]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T08:51:21.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:20 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T08:51:21.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:20 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:51:21.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:20 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:51:21.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:20 smithi071 ceph-mon[32789]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T08:51:21.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:20 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:51:21.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:20 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:51:21.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:20 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:51:21.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T08:51:21.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T08:51:21.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T08:51:21.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T08:51:21.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:51:21.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:51:21.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T08:51:21.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:51:21.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:51:21.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:20 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:51:21.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:21 smithi071 ceph-mon[32789]: pgmap v101: 1 pgs: 1 creating+activating; 0 B data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:21.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:21 smithi071 ceph-mon[32789]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:51:22.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:21 smithi158 ceph-mon[37241]: pgmap v101: 1 pgs: 1 creating+activating; 0 B data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:22.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:21 smithi158 ceph-mon[37241]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:51:22.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:21 smithi150 ceph-mon[36807]: pgmap v101: 1 pgs: 1 creating+activating; 0 B data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:22.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:21 smithi150 ceph-mon[36807]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:51:24.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:23 smithi158 ceph-mon[37241]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:24.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:23 smithi071 ceph-mon[32789]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:24.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:23 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/4194921521' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4fc7723-fd73-40b2-b9d0-7f9bda282ece"}]: dispatch 2024-03-04T08:51:24.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:23 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/4194921521' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4fc7723-fd73-40b2-b9d0-7f9bda282ece"}]': finished 2024-03-04T08:51:24.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:23 smithi071 ceph-mon[32789]: osdmap e23: 4 total, 3 up, 4 in 2024-03-04T08:51:24.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:23 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T08:51:24.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:23 smithi150 ceph-mon[36807]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:24.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:23 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/4194921521' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4fc7723-fd73-40b2-b9d0-7f9bda282ece"}]: dispatch 2024-03-04T08:51:24.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:23 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/4194921521' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4fc7723-fd73-40b2-b9d0-7f9bda282ece"}]': finished 2024-03-04T08:51:24.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:23 smithi150 ceph-mon[36807]: osdmap e23: 4 total, 3 up, 4 in 2024-03-04T08:51:24.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:23 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T08:51:27.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:27 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3324112966' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:51:27.801 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:27 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3324112966' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:51:27.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:27 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/4194921521' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4fc7723-fd73-40b2-b9d0-7f9bda282ece"}]: dispatch 2024-03-04T08:51:27.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:27 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/4194921521' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4fc7723-fd73-40b2-b9d0-7f9bda282ece"}]': finished 2024-03-04T08:51:27.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:27 smithi158 ceph-mon[37241]: osdmap e23: 4 total, 3 up, 4 in 2024-03-04T08:51:27.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:27 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T08:51:27.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:27 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3324112966' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:51:28.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:28 smithi071 ceph-mon[32789]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:28.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:28 smithi071 ceph-mon[32789]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:28.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:28 smithi150 ceph-mon[36807]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:28.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:28 smithi150 ceph-mon[36807]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:28.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:28 smithi158 ceph-mon[37241]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:28.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:28 smithi158 ceph-mon[37241]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:29.597 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:29 smithi071 ceph-mon[32789]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:29.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:29 smithi150 ceph-mon[36807]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:29.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:29 smithi158 ceph-mon[37241]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:31.720 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:31 smithi071 ceph-mon[32789]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:31.720 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:31 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-04T08:51:31.720 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:31 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:51:31.720 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:31 smithi071 ceph-mon[32789]: Deploying daemon osd.3 on smithi071 2024-03-04T08:51:32.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:31 smithi150 ceph-mon[36807]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:32.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:31 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-04T08:51:32.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:31 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:51:32.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:31 smithi150 ceph-mon[36807]: Deploying daemon osd.3 on smithi071 2024-03-04T08:51:32.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:31 smithi158 ceph-mon[37241]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:32.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:31 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-04T08:51:32.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:31 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:51:32.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:31 smithi158 ceph-mon[37241]: Deploying daemon osd.3 on smithi071 2024-03-04T08:51:33.856 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:33 smithi071 ceph-mon[32789]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:34.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:33 smithi150 ceph-mon[36807]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:34.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:33 smithi158 ceph-mon[37241]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:35.933 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:35 smithi071 ceph-mon[32789]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:36.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:35 smithi150 ceph-mon[36807]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:36.046 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:35 smithi158 ceph-mon[37241]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:37.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:37 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:37.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:37 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:37.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:37 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:51:37.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:37 smithi071 ceph-mon[32789]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:37.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:37 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:37.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:37 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:37.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:37 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:51:37.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:37 smithi150 ceph-mon[36807]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:37.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:37 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:37.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:37 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:37.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:37 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:51:37.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:37 smithi158 ceph-mon[37241]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:39.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:39 smithi071 ceph-mon[32789]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:39.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:39 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:39.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:39 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:40.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:39 smithi150 ceph-mon[36807]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:40.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:39 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:40.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:39 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:40.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:40.040+0000 7f5954ff9700 1 -- 172.21.15.71:0/2065904144 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5958062500 con 0x7f59400a1e90 2024-03-04T08:51:40.037 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 3 on host 'smithi071' 2024-03-04T08:51:40.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:40.042+0000 7f595dcae700 1 -- 172.21.15.71:0/2065904144 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f59400a1e90 msgr2=0x7f59400a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:51:40.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:40.042+0000 7f595dcae700 1 --2- 172.21.15.71:0/2065904144 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f59400a1e90 0x7f59400a4350 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f594800b580 tx=0x7f5948005fb0 comp rx=0 tx=0).stop 2024-03-04T08:51:40.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:40.042+0000 7f595dcae700 1 -- 172.21.15.71:0/2065904144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5958103c80 msgr2=0x7f595819fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:51:40.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:40.042+0000 7f595dcae700 1 --2- 172.21.15.71:0/2065904144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5958103c80 0x7f595819fa20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f595000b9a0 tx=0x7f595000bd60 comp rx=0 tx=0).stop 2024-03-04T08:51:40.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:40.042+0000 7f595dcae700 1 -- 172.21.15.71:0/2065904144 shutdown_connections 2024-03-04T08:51:40.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:40.042+0000 7f595dcae700 1 --2- 172.21.15.71:0/2065904144 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f59400a1e90 0x7f59400a4350 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:51:40.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:40.042+0000 7f595dcae700 1 --2- 172.21.15.71:0/2065904144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5958103c80 0x7f595819fa20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:51:40.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:40.042+0000 7f595dcae700 1 --2- 172.21.15.71:0/2065904144 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59581020e0 0x7f5958198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:51:40.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:40.042+0000 7f595dcae700 1 --2- 172.21.15.71:0/2065904144 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59581032e0 0x7f59581989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:51:40.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:40.042+0000 7f595dcae700 1 -- 172.21.15.71:0/2065904144 >> 172.21.15.71:0/2065904144 conn(0x7f59580fd6a0 msgr2=0x7f59581066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:51:40.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:40.043+0000 7f595dcae700 1 -- 172.21.15.71:0/2065904144 shutdown_connections 2024-03-04T08:51:40.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:51:40.043+0000 7f595dcae700 1 -- 172.21.15.71:0/2065904144 wait complete. 2024-03-04T08:51:40.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:39 smithi158 ceph-mon[37241]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:40.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:39 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:40.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:39 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:40.722 DEBUG:teuthology.orchestra.run.smithi071:osd.3> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.3.service 2024-03-04T08:51:40.725 INFO:tasks.cephadm:Deploying osd.4 on smithi150 with /dev/vg_nvme/lv_3... 2024-03-04T08:51:40.725 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-04T08:51:40.968 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T08:51:41.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:41.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:41.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:41 smithi150 ceph-mon[36807]: from='osd.3 [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-04T08:51:41.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:41 smithi150 ceph-mon[36807]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:41.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:41.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:41.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:41 smithi158 ceph-mon[37241]: from='osd.3 [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-04T08:51:41.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:41 smithi158 ceph-mon[37241]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:41.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:41.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:41.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:41 smithi071 ceph-mon[32789]: from='osd.3 [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-04T08:51:41.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:41 smithi071 ceph-mon[32789]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:42.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:42 smithi071 ceph-mon[32789]: from='osd.3 [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-04T08:51:42.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:42 smithi071 ceph-mon[32789]: osdmap e24: 4 total, 3 up, 4 in 2024-03-04T08:51:42.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:42 smithi071 ceph-mon[32789]: from='osd.3 [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-04T08:51:42.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:42 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T08:51:42.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:42 smithi150 ceph-mon[36807]: from='osd.3 [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-04T08:51:42.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:42 smithi150 ceph-mon[36807]: osdmap e24: 4 total, 3 up, 4 in 2024-03-04T08:51:42.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:42 smithi150 ceph-mon[36807]: from='osd.3 [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-04T08:51:42.409 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:42 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T08:51:42.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:42 smithi158 ceph-mon[37241]: from='osd.3 [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-04T08:51:42.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:42 smithi158 ceph-mon[37241]: osdmap e24: 4 total, 3 up, 4 in 2024-03-04T08:51:42.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:42 smithi158 ceph-mon[37241]: from='osd.3 [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-04T08:51:42.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:42 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T08:51:42.976 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-04T08:51:42.977 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-04T08:51:42.977 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-03-04T08:51:42.977 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-03-04T08:51:42.977 INFO:teuthology.orchestra.run.smithi150.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114141 s, 919 MB/s 2024-03-04T08:51:42.977 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 2024-03-04T08:51:42.977 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-03-04T08:51:43.086 INFO:journalctl@ceph.osd.3.smithi071.stdout:Mar 04 08:51:42 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-3[54095]: 2024-03-04T08:51:42.764+0000 7fac96898700 -1 osd.3 0 waiting for initial osdmap 2024-03-04T08:51:43.086 INFO:journalctl@ceph.osd.3.smithi071.stdout:Mar 04 08:51:42 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-3[54095]: 2024-03-04T08:51:42.769+0000 7fac8e683700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T08:51:43.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:43 smithi071 ceph-mon[32789]: from='osd.3 [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-03-04T08:51:43.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:43 smithi071 ceph-mon[32789]: osdmap e25: 4 total, 3 up, 4 in 2024-03-04T08:51:43.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:43 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T08:51:43.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:43 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T08:51:43.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:43 smithi071 ceph-mon[32789]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:43.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:43 smithi071 ceph-mon[32789]: from='osd.3 [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262]' entity='osd.3' 2024-03-04T08:51:43.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:43 smithi071 ceph-mon[32789]: osd.3 [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262] boot 2024-03-04T08:51:43.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:43 smithi071 ceph-mon[32789]: osdmap e26: 4 total, 4 up, 4 in 2024-03-04T08:51:43.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:43 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T08:51:43.444 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:43 smithi150 ceph-mon[36807]: from='osd.3 [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-03-04T08:51:43.444 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:43 smithi150 ceph-mon[36807]: osdmap e25: 4 total, 3 up, 4 in 2024-03-04T08:51:43.444 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:43 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T08:51:43.444 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:43 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T08:51:43.445 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:43 smithi150 ceph-mon[36807]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:43.445 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:43 smithi150 ceph-mon[36807]: from='osd.3 [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262]' entity='osd.3' 2024-03-04T08:51:43.445 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:43 smithi150 ceph-mon[36807]: osd.3 [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262] boot 2024-03-04T08:51:43.445 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:43 smithi150 ceph-mon[36807]: osdmap e26: 4 total, 4 up, 4 in 2024-03-04T08:51:43.445 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:43 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T08:51:43.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:43 smithi158 ceph-mon[37241]: from='osd.3 [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-03-04T08:51:43.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:43 smithi158 ceph-mon[37241]: osdmap e25: 4 total, 3 up, 4 in 2024-03-04T08:51:43.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:43 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T08:51:43.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:43 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T08:51:43.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:43 smithi158 ceph-mon[37241]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T08:51:43.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:43 smithi158 ceph-mon[37241]: from='osd.3 [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262]' entity='osd.3' 2024-03-04T08:51:43.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:43 smithi158 ceph-mon[37241]: osd.3 [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262] boot 2024-03-04T08:51:43.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:43 smithi158 ceph-mon[37241]: osdmap e26: 4 total, 4 up, 4 in 2024-03-04T08:51:43.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:43 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T08:51:43.637 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi150:vg_nvme/lv_3 2024-03-04T08:51:43.876 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T08:51:44.252 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:44 smithi150 ceph-mon[36807]: purged_snaps scrub starts 2024-03-04T08:51:44.252 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:44 smithi150 ceph-mon[36807]: purged_snaps scrub ok 2024-03-04T08:51:44.252 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:44 smithi150 ceph-mon[36807]: osdmap e27: 4 total, 4 up, 4 in 2024-03-04T08:51:44.256 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:44 smithi071 ceph-mon[32789]: purged_snaps scrub starts 2024-03-04T08:51:44.256 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:44 smithi071 ceph-mon[32789]: purged_snaps scrub ok 2024-03-04T08:51:44.256 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:44 smithi071 ceph-mon[32789]: osdmap e27: 4 total, 4 up, 4 in 2024-03-04T08:51:44.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:44 smithi158 ceph-mon[37241]: purged_snaps scrub starts 2024-03-04T08:51:44.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:44 smithi158 ceph-mon[37241]: purged_snaps scrub ok 2024-03-04T08:51:44.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:44 smithi158 ceph-mon[37241]: osdmap e27: 4 total, 4 up, 4 in 2024-03-04T08:51:45.444 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.442+0000 7fdd9aadc700 1 -- 172.21.15.150:0/1531325190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd941032f0 msgr2=0x7fdd94103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:51:45.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.442+0000 7fdd9aadc700 1 --2- 172.21.15.150:0/1531325190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd941032f0 0x7fdd94103750 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdd800099c0 tx=0x7fdd80009cd0 comp rx=0 tx=0).stop 2024-03-04T08:51:45.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.442+0000 7fdd9aadc700 1 -- 172.21.15.150:0/1531325190 shutdown_connections 2024-03-04T08:51:45.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.442+0000 7fdd9aadc700 1 --2- 172.21.15.150:0/1531325190 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdd941020f0 0x7fdd94102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:51:45.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.442+0000 7fdd9aadc700 1 --2- 172.21.15.150:0/1531325190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd94103c90 0x7fdd9410a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:51:45.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.442+0000 7fdd9aadc700 1 --2- 172.21.15.150:0/1531325190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd941032f0 0x7fdd94103750 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:51:45.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.442+0000 7fdd9aadc700 1 -- 172.21.15.150:0/1531325190 >> 172.21.15.150:0/1531325190 conn(0x7fdd940fd6d0 msgr2=0x7fdd940ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:51:45.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.442+0000 7fdd9aadc700 1 -- 172.21.15.150:0/1531325190 shutdown_connections 2024-03-04T08:51:45.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.442+0000 7fdd9aadc700 1 -- 172.21.15.150:0/1531325190 wait complete. 2024-03-04T08:51:45.445 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.443+0000 7fdd9aadc700 1 Processor -- start 2024-03-04T08:51:45.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.443+0000 7fdd9aadc700 1 -- start start 2024-03-04T08:51:45.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.443+0000 7fdd9aadc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd941020f0 0x7fdd94075e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:51:45.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.443+0000 7fdd9aadc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd941032f0 0x7fdd940744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:51:45.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.443+0000 7fdd9aadc700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdd94103c90 0x7fdd940749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:51:45.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.443+0000 7fdd9aadc700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9410dcb0 con 0x7fdd941032f0 2024-03-04T08:51:45.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.443+0000 7fdd9aadc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9410db30 con 0x7fdd941020f0 2024-03-04T08:51:45.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.443+0000 7fdd9aadc700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9410de30 con 0x7fdd94103c90 2024-03-04T08:51:45.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.443+0000 7fdd98878700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd941020f0 0x7fdd94075e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:51:45.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.443+0000 7fdd98878700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd941020f0 0x7fdd94075e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:46976/0 (socket says 172.21.15.150:46976) 2024-03-04T08:51:45.446 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.443+0000 7fdd98878700 1 -- 172.21.15.150:0/1563996808 learned_addr learned my addr 172.21.15.150:0/1563996808 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-04T08:51:45.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.443+0000 7fdd93fff700 1 --2- 172.21.15.150:0/1563996808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd941032f0 0x7fdd940744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:51:45.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.443+0000 7fdd99079700 1 --2- 172.21.15.150:0/1563996808 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdd94103c90 0x7fdd940749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:51:45.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.444+0000 7fdd98878700 1 -- 172.21.15.150:0/1563996808 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdd94103c90 msgr2=0x7fdd940749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:51:45.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.444+0000 7fdd98878700 1 --2- 172.21.15.150:0/1563996808 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdd94103c90 0x7fdd940749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:51:45.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.444+0000 7fdd98878700 1 -- 172.21.15.150:0/1563996808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd941032f0 msgr2=0x7fdd940744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:51:45.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.444+0000 7fdd98878700 1 --2- 172.21.15.150:0/1563996808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd941032f0 0x7fdd940744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:51:45.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.444+0000 7fdd98878700 1 -- 172.21.15.150:0/1563996808 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd940752a0 con 0x7fdd941020f0 2024-03-04T08:51:45.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.444+0000 7fdd99079700 1 --2- 172.21.15.150:0/1563996808 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdd94103c90 0x7fdd940749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:51:45.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.444+0000 7fdd93fff700 1 --2- 172.21.15.150:0/1563996808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd941032f0 0x7fdd940744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:51:45.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.444+0000 7fdd98878700 1 --2- 172.21.15.150:0/1563996808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd941020f0 0x7fdd94075e60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800c970 tx=0x7fdd8800cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:51:45.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.444+0000 7fdd91ffb700 1 -- 172.21.15.150:0/1563996808 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd88004220 con 0x7fdd941020f0 2024-03-04T08:51:45.447 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.444+0000 7fdd9aadc700 1 -- 172.21.15.150:0/1563996808 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd941a2480 con 0x7fdd941020f0 2024-03-04T08:51:45.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.444+0000 7fdd91ffb700 1 -- 172.21.15.150:0/1563996808 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd88007d80 con 0x7fdd941020f0 2024-03-04T08:51:45.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.444+0000 7fdd91ffb700 1 -- 172.21.15.150:0/1563996808 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd88003e50 con 0x7fdd941020f0 2024-03-04T08:51:45.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.444+0000 7fdd9aadc700 1 -- 172.21.15.150:0/1563996808 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd941a2970 con 0x7fdd941020f0 2024-03-04T08:51:45.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.445+0000 7fdd9aadc700 1 -- 172.21.15.150:0/1563996808 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd94068d80 con 0x7fdd941020f0 2024-03-04T08:51:45.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.446+0000 7fdd91ffb700 1 -- 172.21.15.150:0/1563996808 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fdd880078a0 con 0x7fdd941020f0 2024-03-04T08:51:45.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.446+0000 7fdd91ffb700 1 --2- 172.21.15.150:0/1563996808 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fdd840a1fb0 0x7fdd840a4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:51:45.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.447+0000 7fdd91ffb700 1 -- 172.21.15.150:0/1563996808 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 3698+0+0 (secure 0 0 0) 0x7fdd880cc840 con 0x7fdd941020f0 2024-03-04T08:51:45.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.447+0000 7fdd93fff700 1 --2- 172.21.15.150:0/1563996808 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fdd840a1fb0 0x7fdd840a4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:51:45.448 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.447+0000 7fdd93fff700 1 --2- 172.21.15.150:0/1563996808 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fdd840a1fb0 0x7fdd840a4470 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdd80003870 tx=0x7fdd80003940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:51:45.449 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.451+0000 7fdd91ffb700 1 -- 172.21.15.150:0/1563996808 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdd88014030 con 0x7fdd941020f0 2024-03-04T08:51:45.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:45 smithi150 ceph-mon[36807]: Detected new or changed devices on smithi071 2024-03-04T08:51:45.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:45 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:45.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:45 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:45.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:45 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:51:45.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:45 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:51:45.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:45 smithi150 ceph-mon[36807]: Adjusting osd_memory_target on smithi071 to 8544M 2024-03-04T08:51:45.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:45 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:45.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:45 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:51:45.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:45 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:51:45.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:45 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:45.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:45 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:51:45.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:45 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:51:45.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:45 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:51:45.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:45 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:45.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:45 smithi150 ceph-mon[36807]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 357 GiB / 358 GiB avail 2024-03-04T08:51:45.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:45 smithi150 ceph-mon[36807]: osdmap e28: 4 total, 4 up, 4 in 2024-03-04T08:51:45.653 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:51:45.657+0000 7fdd9aadc700 1 -- 172.21.15.150:0/1563996808 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd94062500 con 0x7fdd840a1fb0 2024-03-04T08:51:45.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:45 smithi071 ceph-mon[32789]: Detected new or changed devices on smithi071 2024-03-04T08:51:45.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:45 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:45.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:45 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:45.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:45 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:51:45.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:45 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:51:45.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:45 smithi071 ceph-mon[32789]: Adjusting osd_memory_target on smithi071 to 8544M 2024-03-04T08:51:45.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:45 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:45.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:45 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:51:45.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:45 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:51:45.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:45 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:45.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:45 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:51:45.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:45 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:51:45.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:45 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:51:45.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:45 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:45.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:45 smithi071 ceph-mon[32789]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 357 GiB / 358 GiB avail 2024-03-04T08:51:45.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:45 smithi071 ceph-mon[32789]: osdmap e28: 4 total, 4 up, 4 in 2024-03-04T08:51:45.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:45 smithi158 ceph-mon[37241]: Detected new or changed devices on smithi071 2024-03-04T08:51:45.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:45 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:45.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:45 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:45.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:45 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:51:45.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:45 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:51:45.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:45 smithi158 ceph-mon[37241]: Adjusting osd_memory_target on smithi071 to 8544M 2024-03-04T08:51:45.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:45 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:45.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:45 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:51:45.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:45 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:51:45.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:45 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:45.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:45 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:51:45.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:45 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:51:45.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:45 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:51:45.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:45 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:51:45.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:45 smithi158 ceph-mon[37241]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 357 GiB / 358 GiB avail 2024-03-04T08:51:45.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:45 smithi158 ceph-mon[37241]: osdmap e28: 4 total, 4 up, 4 in 2024-03-04T08:51:46.505 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:46 smithi150 ceph-mon[36807]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:51:46.505 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:46 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:51:46.505 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:46 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:51:46.505 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:46 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:51:46.505 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:46 smithi150 ceph-mon[36807]: osdmap e29: 4 total, 4 up, 4 in 2024-03-04T08:51:46.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:46 smithi158 ceph-mon[37241]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:51:46.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:46 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:51:46.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:46 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:51:46.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:46 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:51:46.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:46 smithi158 ceph-mon[37241]: osdmap e29: 4 total, 4 up, 4 in 2024-03-04T08:51:46.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:46 smithi071 ceph-mon[32789]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:51:46.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:46 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:51:46.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:46 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:51:46.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:46 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:51:46.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:46 smithi071 ceph-mon[32789]: osdmap e29: 4 total, 4 up, 4 in 2024-03-04T08:51:47.599 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:47 smithi150 ceph-mon[36807]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 357 GiB / 358 GiB avail 2024-03-04T08:51:47.599 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:47 smithi150 ceph-mon[36807]: osdmap e30: 4 total, 4 up, 4 in 2024-03-04T08:51:47.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:47 smithi158 ceph-mon[37241]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 357 GiB / 358 GiB avail 2024-03-04T08:51:47.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:47 smithi158 ceph-mon[37241]: osdmap e30: 4 total, 4 up, 4 in 2024-03-04T08:51:47.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:47 smithi071 ceph-mon[32789]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 357 GiB / 358 GiB avail 2024-03-04T08:51:47.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:47 smithi071 ceph-mon[32789]: osdmap e30: 4 total, 4 up, 4 in 2024-03-04T08:51:48.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:48 smithi150 ceph-mon[36807]: from='client.? 172.21.15.150:0/1294692543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7eb72ac-f066-4ccd-b2bf-22a6a26daf41"}]: dispatch 2024-03-04T08:51:48.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:48 smithi150 ceph-mon[36807]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7eb72ac-f066-4ccd-b2bf-22a6a26daf41"}]: dispatch 2024-03-04T08:51:48.782 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:48 smithi150 ceph-mon[36807]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7eb72ac-f066-4ccd-b2bf-22a6a26daf41"}]': finished 2024-03-04T08:51:48.782 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:48 smithi150 ceph-mon[36807]: osdmap e31: 5 total, 4 up, 5 in 2024-03-04T08:51:48.782 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:48 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T08:51:48.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:48 smithi158 ceph-mon[37241]: from='client.? 172.21.15.150:0/1294692543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7eb72ac-f066-4ccd-b2bf-22a6a26daf41"}]: dispatch 2024-03-04T08:51:48.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:48 smithi158 ceph-mon[37241]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7eb72ac-f066-4ccd-b2bf-22a6a26daf41"}]: dispatch 2024-03-04T08:51:48.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:48 smithi158 ceph-mon[37241]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7eb72ac-f066-4ccd-b2bf-22a6a26daf41"}]': finished 2024-03-04T08:51:48.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:48 smithi158 ceph-mon[37241]: osdmap e31: 5 total, 4 up, 5 in 2024-03-04T08:51:48.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:48 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T08:51:48.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:48 smithi071 ceph-mon[32789]: from='client.? 172.21.15.150:0/1294692543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7eb72ac-f066-4ccd-b2bf-22a6a26daf41"}]: dispatch 2024-03-04T08:51:48.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:48 smithi071 ceph-mon[32789]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7eb72ac-f066-4ccd-b2bf-22a6a26daf41"}]: dispatch 2024-03-04T08:51:48.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:48 smithi071 ceph-mon[32789]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7eb72ac-f066-4ccd-b2bf-22a6a26daf41"}]': finished 2024-03-04T08:51:48.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:48 smithi071 ceph-mon[32789]: osdmap e31: 5 total, 4 up, 5 in 2024-03-04T08:51:48.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:48 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T08:51:49.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:49 smithi150 ceph-mon[36807]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-04T08:51:49.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:49 smithi150 ceph-mon[36807]: from='client.? 172.21.15.150:0/1940538947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:51:49.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:49 smithi158 ceph-mon[37241]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-04T08:51:49.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:49 smithi158 ceph-mon[37241]: from='client.? 172.21.15.150:0/1940538947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:51:49.877 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:49 smithi071 ceph-mon[32789]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-04T08:51:49.877 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:49 smithi071 ceph-mon[32789]: from='client.? 172.21.15.150:0/1940538947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:51:51.939 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:51 smithi150 ceph-mon[36807]: pgmap v125: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-04T08:51:52.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:51 smithi158 ceph-mon[37241]: pgmap v125: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-04T08:51:52.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:51 smithi071 ceph-mon[32789]: pgmap v125: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-04T08:51:53.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:53 smithi150 ceph-mon[36807]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-04T08:51:54.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:53 smithi158 ceph-mon[37241]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-04T08:51:54.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:53 smithi071 ceph-mon[32789]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-04T08:51:56.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:55 smithi150 ceph-mon[36807]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-04T08:51:56.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:55 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-04T08:51:56.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:55 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:51:56.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:55 smithi158 ceph-mon[37241]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-04T08:51:56.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:55 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-04T08:51:56.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:55 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:51:56.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:55 smithi071 ceph-mon[32789]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-04T08:51:56.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:55 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-04T08:51:56.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:55 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:51:56.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:56 smithi150 ceph-mon[36807]: Deploying daemon osd.4 on smithi150 2024-03-04T08:51:57.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:56 smithi158 ceph-mon[37241]: Deploying daemon osd.4 on smithi150 2024-03-04T08:51:57.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:56 smithi071 ceph-mon[32789]: Deploying daemon osd.4 on smithi150 2024-03-04T08:51:57.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:57 smithi150 ceph-mon[36807]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T08:51:58.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:57 smithi158 ceph-mon[37241]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T08:51:58.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:57 smithi071 ceph-mon[32789]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T08:51:59.960 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:51:59 smithi150 ceph-mon[36807]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T08:52:00.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:51:59 smithi158 ceph-mon[37241]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T08:52:00.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:51:59 smithi071 ceph-mon[32789]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T08:52:01.802 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:01 smithi150 ceph-mon[36807]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T08:52:01.802 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:01 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:01.802 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:01 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:01.802 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:01 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:52:02.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:01 smithi158 ceph-mon[37241]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T08:52:02.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:01 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:02.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:01 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:02.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:01 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:52:02.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:01 smithi071 ceph-mon[32789]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T08:52:02.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:01 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:02.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:01 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:02.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:01 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:52:03.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:03 smithi150 ceph-mon[36807]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T08:52:04.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:03 smithi158 ceph-mon[37241]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T08:52:04.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:03 smithi071 ceph-mon[32789]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T08:52:05.018 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 4 on host 'smithi150' 2024-03-04T08:52:05.019 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:52:05.022+0000 7fdd91ffb700 1 -- 172.21.15.150:0/1563996808 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdd94062500 con 0x7fdd840a1fb0 2024-03-04T08:52:05.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:52:05.024+0000 7fdd9aadc700 1 -- 172.21.15.150:0/1563996808 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fdd840a1fb0 msgr2=0x7fdd840a4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:52:05.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:52:05.024+0000 7fdd9aadc700 1 --2- 172.21.15.150:0/1563996808 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fdd840a1fb0 0x7fdd840a4470 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdd80003870 tx=0x7fdd80003940 comp rx=0 tx=0).stop 2024-03-04T08:52:05.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:52:05.025+0000 7fdd9aadc700 1 -- 172.21.15.150:0/1563996808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd941020f0 msgr2=0x7fdd94075e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:52:05.023 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:52:05.025+0000 7fdd9aadc700 1 --2- 172.21.15.150:0/1563996808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd941020f0 0x7fdd94075e60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800c970 tx=0x7fdd8800cd30 comp rx=0 tx=0).stop 2024-03-04T08:52:05.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:52:05.025+0000 7fdd9aadc700 1 -- 172.21.15.150:0/1563996808 shutdown_connections 2024-03-04T08:52:05.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:52:05.025+0000 7fdd9aadc700 1 --2- 172.21.15.150:0/1563996808 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fdd840a1fb0 0x7fdd840a4470 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:52:05.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:52:05.025+0000 7fdd9aadc700 1 --2- 172.21.15.150:0/1563996808 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdd94103c90 0x7fdd940749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:52:05.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:52:05.025+0000 7fdd9aadc700 1 --2- 172.21.15.150:0/1563996808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd941020f0 0x7fdd94075e60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:52:05.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:52:05.025+0000 7fdd9aadc700 1 --2- 172.21.15.150:0/1563996808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd941032f0 0x7fdd940744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:52:05.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:52:05.025+0000 7fdd9aadc700 1 -- 172.21.15.150:0/1563996808 >> 172.21.15.150:0/1563996808 conn(0x7fdd940fd6d0 msgr2=0x7fdd941066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:52:05.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:52:05.025+0000 7fdd9aadc700 1 -- 172.21.15.150:0/1563996808 shutdown_connections 2024-03-04T08:52:05.024 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:52:05.025+0000 7fdd9aadc700 1 -- 172.21.15.150:0/1563996808 wait complete. 2024-03-04T08:52:05.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:05 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:05.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:05 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:05.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:05 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:52:05.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:05 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:52:05.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:05 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:05.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:05 smithi150 ceph-mon[36807]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T08:52:05.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:05 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:05.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:05 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:05.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:05 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:52:05.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:05 smithi150 ceph-mon[36807]: from='osd.4 [v2:172.21.15.150:6808/2300837573,v1:172.21.15.150:6809/2300837573]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-04T08:52:05.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:05 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:05.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:05 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:05.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:05 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:52:05.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:05 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:52:05.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:05 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:05.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:05 smithi071 ceph-mon[32789]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T08:52:05.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:05 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:05.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:05 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:05.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:05 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:52:05.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:05 smithi071 ceph-mon[32789]: from='osd.4 [v2:172.21.15.150:6808/2300837573,v1:172.21.15.150:6809/2300837573]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-04T08:52:05.450 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:05 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:05.450 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:05 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:05.450 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:05 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:52:05.450 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:05 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:52:05.450 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:05 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:05.450 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:05 smithi158 ceph-mon[37241]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T08:52:05.450 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:05 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:05.451 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:05 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:05.451 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:05 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:52:05.451 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:05 smithi158 ceph-mon[37241]: from='osd.4 [v2:172.21.15.150:6808/2300837573,v1:172.21.15.150:6809/2300837573]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-04T08:52:05.725 DEBUG:teuthology.orchestra.run.smithi150:osd.4> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.4.service 2024-03-04T08:52:05.728 INFO:tasks.cephadm:Deploying osd.5 on smithi158 with /dev/vg_nvme/lv_3... 2024-03-04T08:52:05.728 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-04T08:52:05.965 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T08:52:07.259 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:06 smithi158 ceph-mon[37241]: from='osd.4 [v2:172.21.15.150:6808/2300837573,v1:172.21.15.150:6809/2300837573]' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-04T08:52:07.259 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:06 smithi158 ceph-mon[37241]: osdmap e32: 5 total, 4 up, 5 in 2024-03-04T08:52:07.259 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:06 smithi158 ceph-mon[37241]: from='osd.4 [v2:172.21.15.150:6808/2300837573,v1:172.21.15.150:6809/2300837573]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-04T08:52:07.259 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:06 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T08:52:07.259 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:06 smithi158 ceph-mon[37241]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T08:52:07.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:06 smithi150 ceph-mon[36807]: from='osd.4 [v2:172.21.15.150:6808/2300837573,v1:172.21.15.150:6809/2300837573]' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-04T08:52:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:06 smithi150 ceph-mon[36807]: osdmap e32: 5 total, 4 up, 5 in 2024-03-04T08:52:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:06 smithi150 ceph-mon[36807]: from='osd.4 [v2:172.21.15.150:6808/2300837573,v1:172.21.15.150:6809/2300837573]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-04T08:52:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:06 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T08:52:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:06 smithi150 ceph-mon[36807]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T08:52:07.280 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 04 08:52:07 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-4[48124]: 2024-03-04T08:52:07.006+0000 7f5096f4d700 -1 osd.4 0 waiting for initial osdmap 2024-03-04T08:52:07.280 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 04 08:52:07 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-4[48124]: 2024-03-04T08:52:07.011+0000 7f5092d40700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T08:52:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:06 smithi071 ceph-mon[32789]: from='osd.4 [v2:172.21.15.150:6808/2300837573,v1:172.21.15.150:6809/2300837573]' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-04T08:52:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:06 smithi071 ceph-mon[32789]: osdmap e32: 5 total, 4 up, 5 in 2024-03-04T08:52:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:06 smithi071 ceph-mon[32789]: from='osd.4 [v2:172.21.15.150:6808/2300837573,v1:172.21.15.150:6809/2300837573]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-04T08:52:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:06 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T08:52:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:06 smithi071 ceph-mon[32789]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T08:52:07.809 INFO:teuthology.orchestra.run.smithi158.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-04T08:52:07.809 INFO:teuthology.orchestra.run.smithi158.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-04T08:52:07.809 INFO:teuthology.orchestra.run.smithi158.stderr: stderr: 10+0 records in 2024-03-04T08:52:07.809 INFO:teuthology.orchestra.run.smithi158.stderr:10+0 records out 2024-03-04T08:52:07.809 INFO:teuthology.orchestra.run.smithi158.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00840644 s, 1.2 GB/s 2024-03-04T08:52:07.809 INFO:teuthology.orchestra.run.smithi158.stderr:--> Zapping successful for: 2024-03-04T08:52:08.026 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:08 smithi150 ceph-mon[36807]: from='osd.4 [v2:172.21.15.150:6808/2300837573,v1:172.21.15.150:6809/2300837573]' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-03-04T08:52:08.026 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:08 smithi150 ceph-mon[36807]: osdmap e33: 5 total, 4 up, 5 in 2024-03-04T08:52:08.026 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:08 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T08:52:08.027 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:08 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T08:52:08.082 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:08 smithi158 ceph-mon[37241]: from='osd.4 [v2:172.21.15.150:6808/2300837573,v1:172.21.15.150:6809/2300837573]' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-03-04T08:52:08.082 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:08 smithi158 ceph-mon[37241]: osdmap e33: 5 total, 4 up, 5 in 2024-03-04T08:52:08.082 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:08 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T08:52:08.083 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:08 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T08:52:08.323 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:08 smithi071 ceph-mon[32789]: from='osd.4 [v2:172.21.15.150:6808/2300837573,v1:172.21.15.150:6809/2300837573]' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-03-04T08:52:08.323 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:08 smithi071 ceph-mon[32789]: osdmap e33: 5 total, 4 up, 5 in 2024-03-04T08:52:08.323 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:08 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T08:52:08.323 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:08 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T08:52:08.376 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi158:vg_nvme/lv_3 2024-03-04T08:52:08.610 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T08:52:09.247 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:09 smithi150 ceph-mon[36807]: purged_snaps scrub starts 2024-03-04T08:52:09.247 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:09 smithi150 ceph-mon[36807]: purged_snaps scrub ok 2024-03-04T08:52:09.247 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:09 smithi150 ceph-mon[36807]: osd.4 [v2:172.21.15.150:6808/2300837573,v1:172.21.15.150:6809/2300837573] boot 2024-03-04T08:52:09.247 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:09 smithi150 ceph-mon[36807]: osdmap e34: 5 total, 5 up, 5 in 2024-03-04T08:52:09.247 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:09 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T08:52:09.247 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:09 smithi150 ceph-mon[36807]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:09.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:09 smithi158 ceph-mon[37241]: purged_snaps scrub starts 2024-03-04T08:52:09.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:09 smithi158 ceph-mon[37241]: purged_snaps scrub ok 2024-03-04T08:52:09.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:09 smithi158 ceph-mon[37241]: osd.4 [v2:172.21.15.150:6808/2300837573,v1:172.21.15.150:6809/2300837573] boot 2024-03-04T08:52:09.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:09 smithi158 ceph-mon[37241]: osdmap e34: 5 total, 5 up, 5 in 2024-03-04T08:52:09.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:09 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T08:52:09.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:09 smithi158 ceph-mon[37241]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:09.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:09 smithi071 ceph-mon[32789]: purged_snaps scrub starts 2024-03-04T08:52:09.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:09 smithi071 ceph-mon[32789]: purged_snaps scrub ok 2024-03-04T08:52:09.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:09 smithi071 ceph-mon[32789]: osd.4 [v2:172.21.15.150:6808/2300837573,v1:172.21.15.150:6809/2300837573] boot 2024-03-04T08:52:09.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:09 smithi071 ceph-mon[32789]: osdmap e34: 5 total, 5 up, 5 in 2024-03-04T08:52:09.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:09 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T08:52:09.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:09 smithi071 ceph-mon[32789]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:10.230 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.234+0000 7f84ff3c5700 1 -- 172.21.15.158:0/3986455570 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f84f8103df0 msgr2=0x7f84f810a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:52:10.230 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.234+0000 7f84ff3c5700 1 --2- 172.21.15.158:0/3986455570 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f84f8103df0 0x7f84f810a6a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f84f000b0a0 tx=0x7f84f000b3b0 comp rx=0 tx=0).stop 2024-03-04T08:52:10.231 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.234+0000 7f84ff3c5700 1 -- 172.21.15.158:0/3986455570 shutdown_connections 2024-03-04T08:52:10.231 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.234+0000 7f84ff3c5700 1 --2- 172.21.15.158:0/3986455570 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f84f8103df0 0x7f84f810a6a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:52:10.231 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.234+0000 7f84ff3c5700 1 --2- 172.21.15.158:0/3986455570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84f81003d0 0x7f84f81007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:52:10.231 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.234+0000 7f84ff3c5700 1 --2- 172.21.15.158:0/3986455570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84f8103450 0x7f84f81038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:52:10.231 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.234+0000 7f84ff3c5700 1 -- 172.21.15.158:0/3986455570 >> 172.21.15.158:0/3986455570 conn(0x7f84f80fbd30 msgr2=0x7f84f80fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:52:10.231 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.234+0000 7f84ff3c5700 1 -- 172.21.15.158:0/3986455570 shutdown_connections 2024-03-04T08:52:10.231 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.234+0000 7f84ff3c5700 1 -- 172.21.15.158:0/3986455570 wait complete. 2024-03-04T08:52:10.233 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.235+0000 7f84ff3c5700 1 Processor -- start 2024-03-04T08:52:10.233 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.235+0000 7f84ff3c5700 1 -- start start 2024-03-04T08:52:10.233 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.235+0000 7f84ff3c5700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f84f81003d0 0x7f84f8198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:52:10.233 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.235+0000 7f84ff3c5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84f8103450 0x7f84f81987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:52:10.234 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.235+0000 7f84ff3c5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84f8103df0 0x7f84f819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:52:10.234 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.235+0000 7f84ff3c5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f810dd60 con 0x7f84f8103df0 2024-03-04T08:52:10.234 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.235+0000 7f84ff3c5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f810dbe0 con 0x7f84f8103450 2024-03-04T08:52:10.234 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.235+0000 7f84ff3c5700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f810dee0 con 0x7f84f81003d0 2024-03-04T08:52:10.234 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.235+0000 7f84fd161700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f84f81003d0 0x7f84f8198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:52:10.234 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.235+0000 7f84fd161700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f84f81003d0 0x7f84f8198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.158:34576/0 (socket says 172.21.15.158:34576) 2024-03-04T08:52:10.234 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.235+0000 7f84fd161700 1 -- 172.21.15.158:0/2783889753 learned_addr learned my addr 172.21.15.158:0/2783889753 (peer_addr_for_me v2:172.21.15.158:0/0) 2024-03-04T08:52:10.234 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.236+0000 7f84fd962700 1 --2- 172.21.15.158:0/2783889753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84f8103df0 0x7f84f819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:52:10.234 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.236+0000 7f84fc960700 1 --2- 172.21.15.158:0/2783889753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84f8103450 0x7f84f81987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:52:10.234 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.236+0000 7f84fd161700 1 -- 172.21.15.158:0/2783889753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84f8103450 msgr2=0x7f84f81987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:52:10.234 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.236+0000 7f84fd161700 1 --2- 172.21.15.158:0/2783889753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84f8103450 0x7f84f81987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:52:10.234 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.236+0000 7f84fd161700 1 -- 172.21.15.158:0/2783889753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84f8103df0 msgr2=0x7f84f819f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:52:10.234 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.236+0000 7f84fd161700 1 --2- 172.21.15.158:0/2783889753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84f8103df0 0x7f84f819f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:52:10.235 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.236+0000 7f84fd161700 1 -- 172.21.15.158:0/2783889753 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f819fec0 con 0x7f84f81003d0 2024-03-04T08:52:10.235 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.236+0000 7f84fd161700 1 --2- 172.21.15.158:0/2783889753 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f84f81003d0 0x7f84f8198260 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f84f400b530 tx=0x7f84f400b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:52:10.235 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.236+0000 7f84ea7fc700 1 -- 172.21.15.158:0/2783889753 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f4015020 con 0x7f84f81003d0 2024-03-04T08:52:10.235 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.236+0000 7f84ea7fc700 1 -- 172.21.15.158:0/2783889753 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84f4011d10 con 0x7f84f81003d0 2024-03-04T08:52:10.235 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.236+0000 7f84ff3c5700 1 -- 172.21.15.158:0/2783889753 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f81a01b0 con 0x7f84f81003d0 2024-03-04T08:52:10.235 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.236+0000 7f84ea7fc700 1 -- 172.21.15.158:0/2783889753 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f400f2e0 con 0x7f84f81003d0 2024-03-04T08:52:10.235 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.236+0000 7f84ff3c5700 1 -- 172.21.15.158:0/2783889753 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f81a0770 con 0x7f84f81003d0 2024-03-04T08:52:10.235 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.237+0000 7f84ff3c5700 1 -- 172.21.15.158:0/2783889753 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f8068d80 con 0x7f84f81003d0 2024-03-04T08:52:10.236 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.238+0000 7f84ea7fc700 1 -- 172.21.15.158:0/2783889753 <== mon.1 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f84f4018020 con 0x7f84f81003d0 2024-03-04T08:52:10.236 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.239+0000 7f84ea7fc700 1 --2- 172.21.15.158:0/2783889753 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f84e40a1e10 0x7f84e40a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:52:10.236 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.239+0000 7f84ea7fc700 1 -- 172.21.15.158:0/2783889753 <== mon.1 v2:172.21.15.158:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f84f40cc580 con 0x7f84f81003d0 2024-03-04T08:52:10.236 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.239+0000 7f84fc960700 1 --2- 172.21.15.158:0/2783889753 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f84e40a1e10 0x7f84e40a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:52:10.236 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.239+0000 7f84fc960700 1 --2- 172.21.15.158:0/2783889753 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f84e40a1e10 0x7f84e40a42d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00ba60 tx=0x7f84ec005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:52:10.239 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.243+0000 7f84ea7fc700 1 -- 172.21.15.158:0/2783889753 <== mon.1 v2:172.21.15.158:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f84f401f020 con 0x7f84f81003d0 2024-03-04T08:52:10.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:10 smithi150 ceph-mon[36807]: osdmap e35: 5 total, 5 up, 5 in 2024-03-04T08:52:10.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:10 smithi150 ceph-mon[36807]: Detected new or changed devices on smithi150 2024-03-04T08:52:10.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:10 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:10.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:10 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:10.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:10 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:52:10.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:10 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:52:10.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:10 smithi150 ceph-mon[36807]: Adjusting osd_memory_target on smithi150 to 8540M 2024-03-04T08:52:10.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:10 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:10.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:10 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:52:10.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:10 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:52:10.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:10 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:10.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:10 smithi158 ceph-mon[37241]: osdmap e35: 5 total, 5 up, 5 in 2024-03-04T08:52:10.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:10 smithi158 ceph-mon[37241]: Detected new or changed devices on smithi150 2024-03-04T08:52:10.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:10 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:10.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:10 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:10.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:10 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:52:10.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:10 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:52:10.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:10 smithi158 ceph-mon[37241]: Adjusting osd_memory_target on smithi150 to 8540M 2024-03-04T08:52:10.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:10 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:10.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:10 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:52:10.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:10 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:52:10.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:10 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:10.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:10 smithi071 ceph-mon[32789]: osdmap e35: 5 total, 5 up, 5 in 2024-03-04T08:52:10.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:10 smithi071 ceph-mon[32789]: Detected new or changed devices on smithi150 2024-03-04T08:52:10.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:10 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:10.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:10 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:10.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:10 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:52:10.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:10 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:52:10.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:10 smithi071 ceph-mon[32789]: Adjusting osd_memory_target on smithi150 to 8540M 2024-03-04T08:52:10.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:10 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:10.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:10 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:52:10.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:10 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:52:10.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:10 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:10.445 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:10.449+0000 7f84ff3c5700 1 -- 172.21.15.158:0/2783889753 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f84f8062500 con 0x7f84e40a1e10 2024-03-04T08:52:11.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:11 smithi150 ceph-mon[36807]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:52:11.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:11 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:52:11.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:11 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:52:11.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:11 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:52:11.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:11 smithi150 ceph-mon[36807]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:11.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:11 smithi158 ceph-mon[37241]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:52:11.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:11 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:52:11.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:11 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:52:11.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:11 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:52:11.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:11 smithi158 ceph-mon[37241]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:11.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:11 smithi071 ceph-mon[32789]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:52:11.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:11 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:52:11.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:11 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:52:11.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:11 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:52:11.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:11 smithi071 ceph-mon[32789]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:14.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:13 smithi150 ceph-mon[36807]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:14.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:13 smithi150 ceph-mon[36807]: from='client.? 172.21.15.158:0/586037005' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b295840-e104-4a66-9cc4-7020627c1765"}]: dispatch 2024-03-04T08:52:14.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:13 smithi150 ceph-mon[36807]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b295840-e104-4a66-9cc4-7020627c1765"}]: dispatch 2024-03-04T08:52:14.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:13 smithi150 ceph-mon[36807]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b295840-e104-4a66-9cc4-7020627c1765"}]': finished 2024-03-04T08:52:14.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:13 smithi150 ceph-mon[36807]: osdmap e36: 6 total, 5 up, 6 in 2024-03-04T08:52:14.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:13 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T08:52:14.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:13 smithi150 ceph-mon[36807]: from='client.? 172.21.15.158:0/2983261014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:52:14.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:13 smithi158 ceph-mon[37241]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:14.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:13 smithi158 ceph-mon[37241]: from='client.? 172.21.15.158:0/586037005' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b295840-e104-4a66-9cc4-7020627c1765"}]: dispatch 2024-03-04T08:52:14.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:13 smithi158 ceph-mon[37241]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b295840-e104-4a66-9cc4-7020627c1765"}]: dispatch 2024-03-04T08:52:14.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:13 smithi158 ceph-mon[37241]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b295840-e104-4a66-9cc4-7020627c1765"}]': finished 2024-03-04T08:52:14.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:13 smithi158 ceph-mon[37241]: osdmap e36: 6 total, 5 up, 6 in 2024-03-04T08:52:14.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:13 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T08:52:14.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:13 smithi158 ceph-mon[37241]: from='client.? 172.21.15.158:0/2983261014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:52:14.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:13 smithi071 ceph-mon[32789]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:14.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:13 smithi071 ceph-mon[32789]: from='client.? 172.21.15.158:0/586037005' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b295840-e104-4a66-9cc4-7020627c1765"}]: dispatch 2024-03-04T08:52:14.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:13 smithi071 ceph-mon[32789]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b295840-e104-4a66-9cc4-7020627c1765"}]: dispatch 2024-03-04T08:52:14.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:13 smithi071 ceph-mon[32789]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b295840-e104-4a66-9cc4-7020627c1765"}]': finished 2024-03-04T08:52:14.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:13 smithi071 ceph-mon[32789]: osdmap e36: 6 total, 5 up, 6 in 2024-03-04T08:52:14.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:13 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T08:52:14.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:13 smithi071 ceph-mon[32789]: from='client.? 172.21.15.158:0/2983261014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:52:16.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:15 smithi150 ceph-mon[36807]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:16.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:15 smithi158 ceph-mon[37241]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:16.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:15 smithi071 ceph-mon[32789]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:18.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:17 smithi150 ceph-mon[36807]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:18.030 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:17 smithi158 ceph-mon[37241]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:18.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:17 smithi071 ceph-mon[32789]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:19.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:19 smithi071 ceph-mon[32789]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:20.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:19 smithi150 ceph-mon[36807]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:20.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:19 smithi158 ceph-mon[37241]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:21.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:20 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-04T08:52:21.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:20 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:52:21.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:20 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-04T08:52:21.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:20 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:52:21.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:20 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-04T08:52:21.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:20 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:52:22.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:21 smithi150 ceph-mon[36807]: Deploying daemon osd.5 on smithi158 2024-03-04T08:52:22.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:21 smithi150 ceph-mon[36807]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:22.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:21 smithi158 ceph-mon[37241]: Deploying daemon osd.5 on smithi158 2024-03-04T08:52:22.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:21 smithi158 ceph-mon[37241]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:22.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:21 smithi071 ceph-mon[32789]: Deploying daemon osd.5 on smithi158 2024-03-04T08:52:22.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:21 smithi071 ceph-mon[32789]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:24.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:23 smithi150 ceph-mon[36807]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:24.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:23 smithi158 ceph-mon[37241]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:24.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:23 smithi071 ceph-mon[32789]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:26.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:25 smithi150 ceph-mon[36807]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:26.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:25 smithi158 ceph-mon[37241]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:26.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:25 smithi071 ceph-mon[32789]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:28.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:28 smithi071 ceph-mon[32789]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:28.951 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:28 smithi158 ceph-mon[37241]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:29.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:28 smithi150 ceph-mon[36807]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:29.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:29 smithi071 ceph-mon[32789]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:29.955 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:29 smithi158 ceph-mon[37241]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:30.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:29 smithi150 ceph-mon[36807]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:31.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:31 smithi071 ceph-mon[32789]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:32.000 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:31 smithi158 ceph-mon[37241]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:32.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:31 smithi150 ceph-mon[36807]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:34.011 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:33 smithi158 ceph-mon[37241]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:34.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:33 smithi150 ceph-mon[36807]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:34.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:33 smithi071 ceph-mon[32789]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:35.106 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:34 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:35.106 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:34 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:35.106 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:34 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:52:35.106 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:34 smithi150 ceph-mon[36807]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:35.238 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:34 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:35.238 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:34 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:35.238 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:34 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:52:35.238 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:34 smithi158 ceph-mon[37241]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:35.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:34 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:35.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:34 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:35.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:34 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:52:35.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:34 smithi071 ceph-mon[32789]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:38.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:37 smithi150 ceph-mon[36807]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:38.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:37 smithi158 ceph-mon[37241]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:38.125 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:37 smithi071 ceph-mon[32789]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:39.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:38 smithi150 ceph-mon[36807]: from='osd.5 [v2:172.21.15.158:6808/1922694045,v1:172.21.15.158:6809/1922694045]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T08:52:39.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:38 smithi150 ceph-mon[36807]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T08:52:39.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:38 smithi158 ceph-mon[37241]: from='osd.5 [v2:172.21.15.158:6808/1922694045,v1:172.21.15.158:6809/1922694045]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T08:52:39.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:38 smithi158 ceph-mon[37241]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T08:52:39.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:38 smithi071 ceph-mon[32789]: from='osd.5 [v2:172.21.15.158:6808/1922694045,v1:172.21.15.158:6809/1922694045]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T08:52:39.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:38 smithi071 ceph-mon[32789]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T08:52:40.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:39 smithi150 ceph-mon[36807]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:40.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:39 smithi150 ceph-mon[36807]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-04T08:52:40.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:39 smithi150 ceph-mon[36807]: from='osd.5 [v2:172.21.15.158:6808/1922694045,v1:172.21.15.158:6809/1922694045]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-03-04T08:52:40.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:39 smithi150 ceph-mon[36807]: osdmap e37: 6 total, 5 up, 6 in 2024-03-04T08:52:40.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:39 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T08:52:40.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:39 smithi150 ceph-mon[36807]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-03-04T08:52:40.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:39 smithi158 ceph-mon[37241]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:40.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:39 smithi071 ceph-mon[32789]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:40.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:39 smithi071 ceph-mon[32789]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-04T08:52:40.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:39 smithi071 ceph-mon[32789]: from='osd.5 [v2:172.21.15.158:6808/1922694045,v1:172.21.15.158:6809/1922694045]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-03-04T08:52:40.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:39 smithi071 ceph-mon[32789]: osdmap e37: 6 total, 5 up, 6 in 2024-03-04T08:52:40.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:39 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T08:52:40.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:39 smithi071 ceph-mon[32789]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-03-04T08:52:42.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:42 smithi071 ceph-mon[32789]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]': finished 2024-03-04T08:52:42.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:42 smithi071 ceph-mon[32789]: osdmap e38: 6 total, 5 up, 6 in 2024-03-04T08:52:42.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:42 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T08:52:42.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:42 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T08:52:42.456 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:42 smithi158 ceph-mon[37241]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-04T08:52:42.456 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:42 smithi158 ceph-mon[37241]: from='osd.5 [v2:172.21.15.158:6808/1922694045,v1:172.21.15.158:6809/1922694045]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-03-04T08:52:42.456 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:42 smithi158 ceph-mon[37241]: osdmap e37: 6 total, 5 up, 6 in 2024-03-04T08:52:42.456 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:42 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T08:52:42.457 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:42 smithi158 ceph-mon[37241]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-03-04T08:52:42.457 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:42 smithi158 ceph-mon[37241]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]': finished 2024-03-04T08:52:42.457 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:42 smithi158 ceph-mon[37241]: osdmap e38: 6 total, 5 up, 6 in 2024-03-04T08:52:42.457 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:42 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T08:52:42.457 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:42 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T08:52:42.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:42 smithi150 ceph-mon[36807]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]': finished 2024-03-04T08:52:42.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:42 smithi150 ceph-mon[36807]: osdmap e38: 6 total, 5 up, 6 in 2024-03-04T08:52:42.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:42 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T08:52:42.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:42 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T08:52:43.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:43 smithi071 ceph-mon[32789]: purged_snaps scrub starts 2024-03-04T08:52:43.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:43 smithi071 ceph-mon[32789]: purged_snaps scrub ok 2024-03-04T08:52:43.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:43 smithi071 ceph-mon[32789]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:43.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:43 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T08:52:43.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:43 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T08:52:43.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:43 smithi071 ceph-mon[32789]: osd.5 [v2:172.21.15.158:6808/1922694045,v1:172.21.15.158:6809/1922694045] boot 2024-03-04T08:52:43.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:43 smithi071 ceph-mon[32789]: osdmap e39: 6 total, 6 up, 6 in 2024-03-04T08:52:43.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:43 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T08:52:43.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:43 smithi071 ceph-mon[32789]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:43.529 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:43 smithi158 ceph-mon[37241]: purged_snaps scrub starts 2024-03-04T08:52:43.529 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:43 smithi158 ceph-mon[37241]: purged_snaps scrub ok 2024-03-04T08:52:43.529 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:43 smithi158 ceph-mon[37241]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:43.529 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:43 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T08:52:43.529 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:43 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T08:52:43.529 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:43 smithi158 ceph-mon[37241]: osd.5 [v2:172.21.15.158:6808/1922694045,v1:172.21.15.158:6809/1922694045] boot 2024-03-04T08:52:43.529 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:43 smithi158 ceph-mon[37241]: osdmap e39: 6 total, 6 up, 6 in 2024-03-04T08:52:43.529 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:43 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T08:52:43.530 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:43 smithi158 ceph-mon[37241]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:43.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:43 smithi150 ceph-mon[36807]: purged_snaps scrub starts 2024-03-04T08:52:43.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:43 smithi150 ceph-mon[36807]: purged_snaps scrub ok 2024-03-04T08:52:43.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:43 smithi150 ceph-mon[36807]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:43.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:43 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T08:52:43.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:43 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T08:52:43.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:43 smithi150 ceph-mon[36807]: osd.5 [v2:172.21.15.158:6808/1922694045,v1:172.21.15.158:6809/1922694045] boot 2024-03-04T08:52:43.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:43 smithi150 ceph-mon[36807]: osdmap e39: 6 total, 6 up, 6 in 2024-03-04T08:52:43.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:43 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T08:52:43.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:43 smithi150 ceph-mon[36807]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-03-04T08:52:44.456 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:44 smithi158 ceph-mon[37241]: osdmap e40: 6 total, 6 up, 6 in 2024-03-04T08:52:44.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:44 smithi150 ceph-mon[36807]: osdmap e40: 6 total, 6 up, 6 in 2024-03-04T08:52:44.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:44 smithi071 ceph-mon[32789]: osdmap e40: 6 total, 6 up, 6 in 2024-03-04T08:52:45.515 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:45 smithi158 ceph-mon[37241]: osdmap e41: 6 total, 6 up, 6 in 2024-03-04T08:52:45.515 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:45 smithi158 ceph-mon[37241]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:52:45.515 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:45 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:45.515 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:45 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:45.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:45 smithi150 ceph-mon[36807]: osdmap e41: 6 total, 6 up, 6 in 2024-03-04T08:52:45.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:45 smithi150 ceph-mon[36807]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:52:45.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:45 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:45.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:45 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:45.598 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:45.602+0000 7f84ea7fc700 1 -- 172.21.15.158:0/2783889753 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f84f8062500 con 0x7f84e40a1e10 2024-03-04T08:52:45.598 INFO:teuthology.orchestra.run.smithi158.stdout:Created osd(s) 5 on host 'smithi158' 2024-03-04T08:52:45.600 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:45.604+0000 7f84ff3c5700 1 -- 172.21.15.158:0/2783889753 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f84e40a1e10 msgr2=0x7f84e40a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:52:45.600 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:45.604+0000 7f84ff3c5700 1 --2- 172.21.15.158:0/2783889753 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f84e40a1e10 0x7f84e40a42d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00ba60 tx=0x7f84ec005d50 comp rx=0 tx=0).stop 2024-03-04T08:52:45.601 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:45.604+0000 7f84ff3c5700 1 -- 172.21.15.158:0/2783889753 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f84f81003d0 msgr2=0x7f84f8198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:52:45.601 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:45.604+0000 7f84ff3c5700 1 --2- 172.21.15.158:0/2783889753 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f84f81003d0 0x7f84f8198260 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f84f400b530 tx=0x7f84f400b8f0 comp rx=0 tx=0).stop 2024-03-04T08:52:45.601 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:45.604+0000 7f84ff3c5700 1 -- 172.21.15.158:0/2783889753 shutdown_connections 2024-03-04T08:52:45.601 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:45.604+0000 7f84ff3c5700 1 --2- 172.21.15.158:0/2783889753 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f84e40a1e10 0x7f84e40a42d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:52:45.601 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:45.604+0000 7f84ff3c5700 1 --2- 172.21.15.158:0/2783889753 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f84f81003d0 0x7f84f8198260 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:52:45.601 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:45.604+0000 7f84ff3c5700 1 --2- 172.21.15.158:0/2783889753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84f8103df0 0x7f84f819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:52:45.601 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:45.604+0000 7f84ff3c5700 1 --2- 172.21.15.158:0/2783889753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84f8103450 0x7f84f81987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:52:45.601 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:45.604+0000 7f84ff3c5700 1 -- 172.21.15.158:0/2783889753 >> 172.21.15.158:0/2783889753 conn(0x7f84f80fbd30 msgr2=0x7f84f8106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:52:45.601 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:45.604+0000 7f84ff3c5700 1 -- 172.21.15.158:0/2783889753 shutdown_connections 2024-03-04T08:52:45.601 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:52:45.604+0000 7f84ff3c5700 1 -- 172.21.15.158:0/2783889753 wait complete. 2024-03-04T08:52:45.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:45 smithi071 ceph-mon[32789]: osdmap e41: 6 total, 6 up, 6 in 2024-03-04T08:52:45.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:45 smithi071 ceph-mon[32789]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:52:45.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:45 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:45.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:45 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:46.257 DEBUG:teuthology.orchestra.run.smithi158:osd.5> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.5.service 2024-03-04T08:52:46.260 INFO:tasks.cephadm:Deploying osd.6 on smithi071 with /dev/vg_nvme/lv_2... 2024-03-04T08:52:46.260 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-04T08:52:46.509 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:52:46.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:46 smithi150 ceph-mon[36807]: osdmap e42: 6 total, 6 up, 6 in 2024-03-04T08:52:46.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:46 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:46.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:46 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:46.530 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:46 smithi158 ceph-mon[37241]: osdmap e42: 6 total, 6 up, 6 in 2024-03-04T08:52:46.530 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:46 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:46.530 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:46 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:46.631 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:46 smithi071 ceph-mon[32789]: osdmap e42: 6 total, 6 up, 6 in 2024-03-04T08:52:46.631 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:46 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:46.631 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:46 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:47.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:47 smithi150 ceph-mon[36807]: osdmap e43: 6 total, 6 up, 6 in 2024-03-04T08:52:47.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:47 smithi150 ceph-mon[36807]: pgmap v165: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:52:47.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:47 smithi158 ceph-mon[37241]: osdmap e43: 6 total, 6 up, 6 in 2024-03-04T08:52:47.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:47 smithi158 ceph-mon[37241]: pgmap v165: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:52:47.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:47 smithi071 ceph-mon[32789]: osdmap e43: 6 total, 6 up, 6 in 2024-03-04T08:52:47.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:47 smithi071 ceph-mon[32789]: pgmap v165: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:52:48.709 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-04T08:52:48.709 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-04T08:52:48.709 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-03-04T08:52:48.709 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-03-04T08:52:48.709 INFO:teuthology.orchestra.run.smithi071.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.014098 s, 744 MB/s 2024-03-04T08:52:48.709 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-03-04T08:52:49.513 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi071:vg_nvme/lv_2 2024-03-04T08:52:49.759 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:52:49.880 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:49 smithi071 ceph-mon[32789]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-03-04T08:52:49.880 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:49 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:49.880 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:49 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:49.881 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:49 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:52:49.881 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:49 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:52:49.881 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:49 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:49.881 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:49 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:52:49.881 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:49 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:52:49.881 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:49 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:49.881 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:49 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:52:50.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:49 smithi150 ceph-mon[36807]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-03-04T08:52:50.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:49 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:50.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:49 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:50.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:49 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:52:50.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:49 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:52:50.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:49 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:50.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:49 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:52:50.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:49 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:52:50.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:49 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:50.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:49 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:52:50.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:49 smithi158 ceph-mon[37241]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-03-04T08:52:50.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:49 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:50.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:49 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:50.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:49 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:52:50.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:49 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:52:50.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:49 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:50.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:49 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:52:50.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:49 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:52:50.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:49 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:50.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:49 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:52:51.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:50 smithi150 ceph-mon[36807]: Detected new or changed devices on smithi158 2024-03-04T08:52:51.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:50 smithi150 ceph-mon[36807]: Adjusting osd_memory_target on smithi158 to 8540M 2024-03-04T08:52:51.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:50 smithi158 ceph-mon[37241]: Detected new or changed devices on smithi158 2024-03-04T08:52:51.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:50 smithi158 ceph-mon[37241]: Adjusting osd_memory_target on smithi158 to 8540M 2024-03-04T08:52:51.055 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:50 smithi071 ceph-mon[32789]: Detected new or changed devices on smithi158 2024-03-04T08:52:51.056 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:50 smithi071 ceph-mon[32789]: Adjusting osd_memory_target on smithi158 to 8540M 2024-03-04T08:52:51.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.637+0000 7f44ada9c700 1 -- 172.21.15.71:0/2229935259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a8103cb0 msgr2=0x7f44a810a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:52:51.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.637+0000 7f44ada9c700 1 --2- 172.21.15.71:0/2229935259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a8103cb0 0x7f44a810a560 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f44a000b0d0 tx=0x7f44a000b3e0 comp rx=0 tx=0).stop 2024-03-04T08:52:51.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.637+0000 7f44ada9c700 1 -- 172.21.15.71:0/2229935259 shutdown_connections 2024-03-04T08:52:51.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.637+0000 7f44ada9c700 1 --2- 172.21.15.71:0/2229935259 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f44a8102110 0x7f44a8102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:52:51.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.637+0000 7f44ada9c700 1 --2- 172.21.15.71:0/2229935259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a8103cb0 0x7f44a810a560 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:52:51.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.637+0000 7f44ada9c700 1 --2- 172.21.15.71:0/2229935259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a8103310 0x7f44a8103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:52:51.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.637+0000 7f44ada9c700 1 -- 172.21.15.71:0/2229935259 >> 172.21.15.71:0/2229935259 conn(0x7f44a80fd6d0 msgr2=0x7f44a80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:52:51.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.637+0000 7f44ada9c700 1 -- 172.21.15.71:0/2229935259 shutdown_connections 2024-03-04T08:52:51.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.637+0000 7f44ada9c700 1 -- 172.21.15.71:0/2229935259 wait complete. 2024-03-04T08:52:51.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.638+0000 7f44ada9c700 1 Processor -- start 2024-03-04T08:52:51.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.638+0000 7f44ada9c700 1 -- start start 2024-03-04T08:52:51.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.638+0000 7f44ada9c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a8102110 0x7f44a8198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:52:51.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.638+0000 7f44ada9c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f44a8103310 0x7f44a8198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:52:51.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.638+0000 7f44ada9c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a8103cb0 0x7f44a819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:52:51.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.638+0000 7f44ada9c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a810dce0 con 0x7f44a8103cb0 2024-03-04T08:52:51.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.638+0000 7f44ada9c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a810db60 con 0x7f44a8102110 2024-03-04T08:52:51.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.638+0000 7f44ada9c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a810de60 con 0x7f44a8103310 2024-03-04T08:52:51.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.639+0000 7f44a7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a8103cb0 0x7f44a819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:52:51.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.639+0000 7f44a7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a8103cb0 0x7f44a819f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52012/0 (socket says 172.21.15.71:52012) 2024-03-04T08:52:51.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.639+0000 7f44a7fff700 1 -- 172.21.15.71:0/467609602 learned_addr learned my addr 172.21.15.71:0/467609602 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:52:51.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.639+0000 7f44a6ffd700 1 --2- 172.21.15.71:0/467609602 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f44a8103310 0x7f44a8198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:52:51.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.639+0000 7f44a77fe700 1 --2- 172.21.15.71:0/467609602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a8102110 0x7f44a8198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:52:51.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.639+0000 7f44a7fff700 1 -- 172.21.15.71:0/467609602 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f44a8103310 msgr2=0x7f44a8198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:52:51.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.639+0000 7f44a7fff700 1 --2- 172.21.15.71:0/467609602 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f44a8103310 0x7f44a8198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:52:51.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.639+0000 7f44a7fff700 1 -- 172.21.15.71:0/467609602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a8102110 msgr2=0x7f44a8198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:52:51.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.639+0000 7f44a7fff700 1 --2- 172.21.15.71:0/467609602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a8102110 0x7f44a8198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:52:51.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.639+0000 7f44a7fff700 1 -- 172.21.15.71:0/467609602 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a819fe90 con 0x7f44a8103cb0 2024-03-04T08:52:51.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.639+0000 7f44a7fff700 1 --2- 172.21.15.71:0/467609602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a8103cb0 0x7f44a819f7f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f44a000b770 tx=0x7f44a0007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:52:51.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.639+0000 7f44a4ff9700 1 -- 172.21.15.71:0/467609602 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a0003e10 con 0x7f44a8103cb0 2024-03-04T08:52:51.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.639+0000 7f44a4ff9700 1 -- 172.21.15.71:0/467609602 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44a0007cc0 con 0x7f44a8103cb0 2024-03-04T08:52:51.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.639+0000 7f44a4ff9700 1 -- 172.21.15.71:0/467609602 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a001ba90 con 0x7f44a8103cb0 2024-03-04T08:52:51.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.639+0000 7f44ada9c700 1 -- 172.21.15.71:0/467609602 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a81a0120 con 0x7f44a8103cb0 2024-03-04T08:52:51.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.639+0000 7f44ada9c700 1 -- 172.21.15.71:0/467609602 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a81a0570 con 0x7f44a8103cb0 2024-03-04T08:52:51.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.640+0000 7f44ada9c700 1 -- 172.21.15.71:0/467609602 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a8068d80 con 0x7f44a8103cb0 2024-03-04T08:52:51.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.641+0000 7f44a4ff9700 1 -- 172.21.15.71:0/467609602 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f44a0005020 con 0x7f44a8103cb0 2024-03-04T08:52:51.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.642+0000 7f44a4ff9700 1 --2- 172.21.15.71:0/467609602 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f44900a1e10 0x7f44900a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:52:51.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.642+0000 7f44a77fe700 1 --2- 172.21.15.71:0/467609602 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f44900a1e10 0x7f44900a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:52:51.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.642+0000 7f44a4ff9700 1 -- 172.21.15.71:0/467609602 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4533+0+0 (secure 0 0 0) 0x7f44a000f020 con 0x7f44a8103cb0 2024-03-04T08:52:51.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.642+0000 7f44a77fe700 1 --2- 172.21.15.71:0/467609602 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f44900a1e10 0x7f44900a42d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f44a8103170 tx=0x7f449800b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:52:51.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.646+0000 7f44a4ff9700 1 -- 172.21.15.71:0/467609602 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f44a009c9e0 con 0x7f44a8103cb0 2024-03-04T08:52:51.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:52:51.851+0000 7f44ada9c700 1 -- 172.21.15.71:0/467609602 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f44a80625c0 con 0x7f44900a1e10 2024-03-04T08:52:51.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:51 smithi071 ceph-mon[32789]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-03-04T08:52:51.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:51 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:52:51.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:51 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:52:51.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:51 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:52.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:51 smithi150 ceph-mon[36807]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-03-04T08:52:52.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:51 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:52:52.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:51 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:52:52.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:51 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:52.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:51 smithi158 ceph-mon[37241]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-03-04T08:52:52.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:51 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:52:52.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:51 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:52:52.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:51 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:52:53.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:52 smithi150 ceph-mon[36807]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:52:53.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:52 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:52:53.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:52 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:52:53.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:52 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:52:53.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:52 smithi158 ceph-mon[37241]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:52:53.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:52 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:52:53.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:52 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:52:53.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:52 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:52:53.128 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:52 smithi071 ceph-mon[32789]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:52:53.129 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:52 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:52:53.129 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:52 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:52:53.129 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:52 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:52:54.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:53 smithi150 ceph-mon[36807]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-04T08:52:54.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:53 smithi158 ceph-mon[37241]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-04T08:52:54.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:53 smithi071 ceph-mon[32789]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-04T08:52:55.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:54 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2696274961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2"}]: dispatch 2024-03-04T08:52:55.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:54 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2696274961' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2"}]': finished 2024-03-04T08:52:55.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:54 smithi150 ceph-mon[36807]: osdmap e44: 7 total, 6 up, 7 in 2024-03-04T08:52:55.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:54 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T08:52:55.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:54 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2696274961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2"}]: dispatch 2024-03-04T08:52:55.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:54 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2696274961' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2"}]': finished 2024-03-04T08:52:55.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:54 smithi158 ceph-mon[37241]: osdmap e44: 7 total, 6 up, 7 in 2024-03-04T08:52:55.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:54 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T08:52:55.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:54 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2696274961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2"}]: dispatch 2024-03-04T08:52:55.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:54 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2696274961' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2"}]': finished 2024-03-04T08:52:55.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:54 smithi071 ceph-mon[32789]: osdmap e44: 7 total, 6 up, 7 in 2024-03-04T08:52:55.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:54 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T08:52:56.035 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:55 smithi150 ceph-mon[36807]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-04T08:52:56.035 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:55 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2654264204' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:52:56.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:55 smithi158 ceph-mon[37241]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-04T08:52:56.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:55 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2654264204' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:52:56.174 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:55 smithi071 ceph-mon[32789]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-04T08:52:56.175 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:55 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2654264204' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:52:58.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:57 smithi150 ceph-mon[36807]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-04T08:52:58.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:57 smithi158 ceph-mon[37241]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-04T08:52:58.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:57 smithi071 ceph-mon[32789]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-04T08:52:59.804 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:52:59 smithi071 ceph-mon[32789]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:00.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:52:59 smithi150 ceph-mon[36807]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:00.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:52:59 smithi158 ceph-mon[37241]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:02.010 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:01 smithi071 ceph-mon[32789]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:02.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:01 smithi150 ceph-mon[36807]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:02.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:01 smithi158 ceph-mon[37241]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:03.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:02 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-04T08:53:03.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:02 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:03.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:02 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-04T08:53:03.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:02 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:03.062 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:02 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-04T08:53:03.062 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:02 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:04.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:03 smithi150 ceph-mon[36807]: Deploying daemon osd.6 on smithi071 2024-03-04T08:53:04.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:03 smithi150 ceph-mon[36807]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:04.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:03 smithi158 ceph-mon[37241]: Deploying daemon osd.6 on smithi071 2024-03-04T08:53:04.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:03 smithi158 ceph-mon[37241]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:04.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:03 smithi071 ceph-mon[32789]: Deploying daemon osd.6 on smithi071 2024-03-04T08:53:04.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:03 smithi071 ceph-mon[32789]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:06.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:05 smithi150 ceph-mon[36807]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:06.052 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:05 smithi071 ceph-mon[32789]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:06.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:05 smithi158 ceph-mon[37241]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:07.865 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:07 smithi071 ceph-mon[32789]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:08.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:07 smithi150 ceph-mon[36807]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:08.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:07 smithi158 ceph-mon[37241]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:09.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:09 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:09.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:09 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:09.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:09 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:53:09.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:09 smithi071 ceph-mon[32789]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:09.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:09 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:09.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:09 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:09.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:09 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:53:09.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:09 smithi150 ceph-mon[36807]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:09.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:09 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:09.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:09 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:09.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:09 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:53:09.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:09 smithi158 ceph-mon[37241]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:11.761 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 6 on host 'smithi071' 2024-03-04T08:53:11.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:53:11.764+0000 7f44a4ff9700 1 -- 172.21.15.71:0/467609602 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f44a80625c0 con 0x7f44900a1e10 2024-03-04T08:53:11.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:53:11.767+0000 7f44ada9c700 1 -- 172.21.15.71:0/467609602 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f44900a1e10 msgr2=0x7f44900a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:53:11.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:53:11.767+0000 7f44ada9c700 1 --2- 172.21.15.71:0/467609602 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f44900a1e10 0x7f44900a42d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f44a8103170 tx=0x7f449800b5b0 comp rx=0 tx=0).stop 2024-03-04T08:53:11.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:53:11.767+0000 7f44ada9c700 1 -- 172.21.15.71:0/467609602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a8103cb0 msgr2=0x7f44a819f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:53:11.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:53:11.767+0000 7f44ada9c700 1 --2- 172.21.15.71:0/467609602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a8103cb0 0x7f44a819f7f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f44a000b770 tx=0x7f44a0007a20 comp rx=0 tx=0).stop 2024-03-04T08:53:11.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:53:11.767+0000 7f44ada9c700 1 -- 172.21.15.71:0/467609602 shutdown_connections 2024-03-04T08:53:11.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:53:11.767+0000 7f44ada9c700 1 --2- 172.21.15.71:0/467609602 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f44900a1e10 0x7f44900a42d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:53:11.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:53:11.767+0000 7f44ada9c700 1 --2- 172.21.15.71:0/467609602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44a8103cb0 0x7f44a819f7f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:53:11.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:53:11.767+0000 7f44ada9c700 1 --2- 172.21.15.71:0/467609602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a8102110 0x7f44a8198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:53:11.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:53:11.767+0000 7f44ada9c700 1 --2- 172.21.15.71:0/467609602 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f44a8103310 0x7f44a8198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:53:11.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:53:11.767+0000 7f44ada9c700 1 -- 172.21.15.71:0/467609602 >> 172.21.15.71:0/467609602 conn(0x7f44a80fd6d0 msgr2=0x7f44a81065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:53:11.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:53:11.767+0000 7f44ada9c700 1 -- 172.21.15.71:0/467609602 shutdown_connections 2024-03-04T08:53:11.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:53:11.767+0000 7f44ada9c700 1 -- 172.21.15.71:0/467609602 wait complete. 2024-03-04T08:53:12.020 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:11 smithi071 ceph-mon[32789]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:12.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:11 smithi150 ceph-mon[36807]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:12.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:11 smithi158 ceph-mon[37241]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:12.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:12 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:12.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:12 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:12.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:12 smithi071 ceph-mon[32789]: from='osd.6 [v2:172.21.15.71:6818/3848842707,v1:172.21.15.71:6819/3848842707]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-04T08:53:12.972 DEBUG:teuthology.orchestra.run.smithi071:osd.6> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.6.service 2024-03-04T08:53:12.974 INFO:tasks.cephadm:Deploying osd.7 on smithi150 with /dev/vg_nvme/lv_2... 2024-03-04T08:53:12.974 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-04T08:53:13.005 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:12 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:13.005 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:12 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:13.005 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:12 smithi150 ceph-mon[36807]: from='osd.6 [v2:172.21.15.71:6818/3848842707,v1:172.21.15.71:6819/3848842707]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-04T08:53:13.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:12 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:13.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:12 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:13.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:12 smithi158 ceph-mon[37241]: from='osd.6 [v2:172.21.15.71:6818/3848842707,v1:172.21.15.71:6819/3848842707]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-04T08:53:13.221 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T08:53:13.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:13 smithi071 ceph-mon[32789]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:13.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:13 smithi071 ceph-mon[32789]: from='osd.6 [v2:172.21.15.71:6818/3848842707,v1:172.21.15.71:6819/3848842707]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-04T08:53:13.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:13 smithi071 ceph-mon[32789]: osdmap e45: 7 total, 6 up, 7 in 2024-03-04T08:53:13.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:13 smithi071 ceph-mon[32789]: from='osd.6 [v2:172.21.15.71:6818/3848842707,v1:172.21.15.71:6819/3848842707]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-04T08:53:13.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:13 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T08:53:13.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:13 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:13.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:13 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:13.923 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 08:53:13 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T08:53:13.762+0000 7f1064356700 -1 osd.6 0 waiting for initial osdmap 2024-03-04T08:53:13.923 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 08:53:13 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T08:53:13.771+0000 7f1060149700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T08:53:14.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:13 smithi150 ceph-mon[36807]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:14.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:13 smithi150 ceph-mon[36807]: from='osd.6 [v2:172.21.15.71:6818/3848842707,v1:172.21.15.71:6819/3848842707]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-04T08:53:14.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:13 smithi150 ceph-mon[36807]: osdmap e45: 7 total, 6 up, 7 in 2024-03-04T08:53:14.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:13 smithi150 ceph-mon[36807]: from='osd.6 [v2:172.21.15.71:6818/3848842707,v1:172.21.15.71:6819/3848842707]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-04T08:53:14.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:13 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T08:53:14.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:13 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:14.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:13 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:14.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:13 smithi158 ceph-mon[37241]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:14.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:13 smithi158 ceph-mon[37241]: from='osd.6 [v2:172.21.15.71:6818/3848842707,v1:172.21.15.71:6819/3848842707]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-04T08:53:14.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:13 smithi158 ceph-mon[37241]: osdmap e45: 7 total, 6 up, 7 in 2024-03-04T08:53:14.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:13 smithi158 ceph-mon[37241]: from='osd.6 [v2:172.21.15.71:6818/3848842707,v1:172.21.15.71:6819/3848842707]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-04T08:53:14.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:13 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T08:53:14.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:13 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:14.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:13 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:14.999 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:14 smithi071 ceph-mon[32789]: from='osd.6 [v2:172.21.15.71:6818/3848842707,v1:172.21.15.71:6819/3848842707]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-03-04T08:53:14.999 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:14 smithi071 ceph-mon[32789]: osdmap e46: 7 total, 6 up, 7 in 2024-03-04T08:53:14.999 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:14 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T08:53:14.999 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:14 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T08:53:14.999 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:14 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T08:53:15.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:14 smithi150 ceph-mon[36807]: from='osd.6 [v2:172.21.15.71:6818/3848842707,v1:172.21.15.71:6819/3848842707]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-03-04T08:53:15.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:14 smithi150 ceph-mon[36807]: osdmap e46: 7 total, 6 up, 7 in 2024-03-04T08:53:15.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:14 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T08:53:15.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:14 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T08:53:15.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:14 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T08:53:15.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:14 smithi158 ceph-mon[37241]: from='osd.6 [v2:172.21.15.71:6818/3848842707,v1:172.21.15.71:6819/3848842707]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-03-04T08:53:15.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:14 smithi158 ceph-mon[37241]: osdmap e46: 7 total, 6 up, 7 in 2024-03-04T08:53:15.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:14 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T08:53:15.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:14 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T08:53:15.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:14 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T08:53:15.190 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-04T08:53:15.190 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-04T08:53:15.190 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-03-04T08:53:15.190 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-03-04T08:53:15.190 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129793 s, 808 MB/s 2024-03-04T08:53:15.190 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-03-04T08:53:15.796 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi150:vg_nvme/lv_2 2024-03-04T08:53:16.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:15 smithi150 ceph-mon[36807]: purged_snaps scrub starts 2024-03-04T08:53:16.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:15 smithi150 ceph-mon[36807]: purged_snaps scrub ok 2024-03-04T08:53:16.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:15 smithi150 ceph-mon[36807]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:16.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:15 smithi150 ceph-mon[36807]: osd.6 [v2:172.21.15.71:6818/3848842707,v1:172.21.15.71:6819/3848842707] boot 2024-03-04T08:53:16.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:15 smithi150 ceph-mon[36807]: osdmap e47: 7 total, 7 up, 7 in 2024-03-04T08:53:16.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:15 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T08:53:16.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:15 smithi158 ceph-mon[37241]: purged_snaps scrub starts 2024-03-04T08:53:16.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:15 smithi158 ceph-mon[37241]: purged_snaps scrub ok 2024-03-04T08:53:16.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:15 smithi158 ceph-mon[37241]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:16.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:15 smithi158 ceph-mon[37241]: osd.6 [v2:172.21.15.71:6818/3848842707,v1:172.21.15.71:6819/3848842707] boot 2024-03-04T08:53:16.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:15 smithi158 ceph-mon[37241]: osdmap e47: 7 total, 7 up, 7 in 2024-03-04T08:53:16.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T08:53:16.074 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T08:53:16.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:15 smithi071 ceph-mon[32789]: purged_snaps scrub starts 2024-03-04T08:53:16.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:15 smithi071 ceph-mon[32789]: purged_snaps scrub ok 2024-03-04T08:53:16.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:15 smithi071 ceph-mon[32789]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-04T08:53:16.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:15 smithi071 ceph-mon[32789]: osd.6 [v2:172.21.15.71:6818/3848842707,v1:172.21.15.71:6819/3848842707] boot 2024-03-04T08:53:16.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:15 smithi071 ceph-mon[32789]: osdmap e47: 7 total, 7 up, 7 in 2024-03-04T08:53:16.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:15 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T08:53:17.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:16 smithi158 ceph-mon[37241]: osdmap e48: 7 total, 7 up, 7 in 2024-03-04T08:53:17.141 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:16 smithi150 ceph-mon[36807]: osdmap e48: 7 total, 7 up, 7 in 2024-03-04T08:53:17.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:16 smithi071 ceph-mon[32789]: osdmap e48: 7 total, 7 up, 7 in 2024-03-04T08:53:17.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.666+0000 7f5b62aaf700 1 -- 172.21.15.150:0/2996016952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b5c1020f0 msgr2=0x7f5b5c102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:53:17.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.666+0000 7f5b62aaf700 1 --2- 172.21.15.150:0/2996016952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b5c1020f0 0x7f5b5c102510 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5b500099c0 tx=0x7f5b50009cd0 comp rx=0 tx=0).stop 2024-03-04T08:53:17.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.666+0000 7f5b62aaf700 1 -- 172.21.15.150:0/2996016952 shutdown_connections 2024-03-04T08:53:17.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.666+0000 7f5b62aaf700 1 --2- 172.21.15.150:0/2996016952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b5c103c90 0x7f5b5c10a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:53:17.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.666+0000 7f5b62aaf700 1 --2- 172.21.15.150:0/2996016952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b5c1020f0 0x7f5b5c102510 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:53:17.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.666+0000 7f5b62aaf700 1 --2- 172.21.15.150:0/2996016952 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b5c1032f0 0x7f5b5c103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:53:17.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.667+0000 7f5b62aaf700 1 -- 172.21.15.150:0/2996016952 >> 172.21.15.150:0/2996016952 conn(0x7f5b5c0fd6d0 msgr2=0x7f5b5c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:53:17.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.667+0000 7f5b62aaf700 1 -- 172.21.15.150:0/2996016952 shutdown_connections 2024-03-04T08:53:17.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.667+0000 7f5b62aaf700 1 -- 172.21.15.150:0/2996016952 wait complete. 2024-03-04T08:53:17.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.667+0000 7f5b62aaf700 1 Processor -- start 2024-03-04T08:53:17.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.667+0000 7f5b62aaf700 1 -- start start 2024-03-04T08:53:17.665 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.667+0000 7f5b62aaf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b5c1020f0 0x7f5b5c075e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:53:17.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.667+0000 7f5b62aaf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b5c1032f0 0x7f5b5c0744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:53:17.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.667+0000 7f5b62aaf700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b5c103c90 0x7f5b5c0749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:53:17.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.667+0000 7f5b62aaf700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c10de00 con 0x7f5b5c1032f0 2024-03-04T08:53:17.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.667+0000 7f5b62aaf700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c10dc80 con 0x7f5b5c1020f0 2024-03-04T08:53:17.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.667+0000 7f5b62aaf700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c10df80 con 0x7f5b5c103c90 2024-03-04T08:53:17.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.667+0000 7f5b6084b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b5c1020f0 0x7f5b5c075e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:53:17.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.667+0000 7f5b6084b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b5c1020f0 0x7f5b5c075e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:41174/0 (socket says 172.21.15.150:41174) 2024-03-04T08:53:17.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.667+0000 7f5b6084b700 1 -- 172.21.15.150:0/4259880789 learned_addr learned my addr 172.21.15.150:0/4259880789 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-04T08:53:17.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.668+0000 7f5b6104c700 1 --2- 172.21.15.150:0/4259880789 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b5c103c90 0x7f5b5c0749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:53:17.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.668+0000 7f5b5bfff700 1 --2- 172.21.15.150:0/4259880789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b5c1032f0 0x7f5b5c0744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:53:17.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.668+0000 7f5b6084b700 1 -- 172.21.15.150:0/4259880789 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b5c103c90 msgr2=0x7f5b5c0749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:53:17.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.668+0000 7f5b6084b700 1 --2- 172.21.15.150:0/4259880789 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b5c103c90 0x7f5b5c0749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:53:17.666 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.668+0000 7f5b6084b700 1 -- 172.21.15.150:0/4259880789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b5c1032f0 msgr2=0x7f5b5c0744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:53:17.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.668+0000 7f5b6084b700 1 --2- 172.21.15.150:0/4259880789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b5c1032f0 0x7f5b5c0744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:53:17.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.668+0000 7f5b6084b700 1 -- 172.21.15.150:0/4259880789 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5c0752a0 con 0x7f5b5c1020f0 2024-03-04T08:53:17.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.668+0000 7f5b6104c700 1 --2- 172.21.15.150:0/4259880789 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b5c103c90 0x7f5b5c0749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:53:17.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.668+0000 7f5b5bfff700 1 --2- 172.21.15.150:0/4259880789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b5c1032f0 0x7f5b5c0744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:53:17.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.668+0000 7f5b6084b700 1 --2- 172.21.15.150:0/4259880789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b5c1020f0 0x7f5b5c075e60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5b500098e0 tx=0x7f5b500047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:53:17.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.668+0000 7f5b59ffb700 1 -- 172.21.15.150:0/4259880789 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b50027070 con 0x7f5b5c1020f0 2024-03-04T08:53:17.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.668+0000 7f5b62aaf700 1 -- 172.21.15.150:0/4259880789 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5c1a2620 con 0x7f5b5c1020f0 2024-03-04T08:53:17.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.668+0000 7f5b59ffb700 1 -- 172.21.15.150:0/4259880789 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b50022470 con 0x7f5b5c1020f0 2024-03-04T08:53:17.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.668+0000 7f5b59ffb700 1 -- 172.21.15.150:0/4259880789 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5000f6c0 con 0x7f5b5c1020f0 2024-03-04T08:53:17.667 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.668+0000 7f5b62aaf700 1 -- 172.21.15.150:0/4259880789 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5c1a2af0 con 0x7f5b5c1020f0 2024-03-04T08:53:17.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.669+0000 7f5b62aaf700 1 -- 172.21.15.150:0/4259880789 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b5c068d80 con 0x7f5b5c1020f0 2024-03-04T08:53:17.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.671+0000 7f5b59ffb700 1 -- 172.21.15.150:0/4259880789 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f5b5002a070 con 0x7f5b5c1020f0 2024-03-04T08:53:17.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.671+0000 7f5b59ffb700 1 --2- 172.21.15.150:0/4259880789 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f5b4c0a1e10 0x7f5b4c0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:53:17.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.672+0000 7f5b59ffb700 1 -- 172.21.15.150:0/4259880789 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f5b500ce470 con 0x7f5b5c1020f0 2024-03-04T08:53:17.668 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.672+0000 7f5b5bfff700 1 --2- 172.21.15.150:0/4259880789 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f5b4c0a1e10 0x7f5b4c0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:53:17.669 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.672+0000 7f5b5bfff700 1 --2- 172.21.15.150:0/4259880789 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f5b4c0a1e10 0x7f5b4c0a42d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5b480099d0 tx=0x7f5b48008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:53:17.671 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.675+0000 7f5b59ffb700 1 -- 172.21.15.150:0/4259880789 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5b50013070 con 0x7f5b5c1020f0 2024-03-04T08:53:17.875 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:17.879+0000 7f5b62aaf700 1 -- 172.21.15.150:0/4259880789 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b5c062500 con 0x7f5b4c0a1e10 2024-03-04T08:53:18.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:17 smithi158 ceph-mon[37241]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:18.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:17 smithi158 ceph-mon[37241]: Detected new or changed devices on smithi071 2024-03-04T08:53:18.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:17 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:18.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:17 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:18.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:17 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:53:18.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:17 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:53:18.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:17 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:53:18.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:17 smithi158 ceph-mon[37241]: Adjusting osd_memory_target on smithi071 to 5696M 2024-03-04T08:53:18.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:17 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:18.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:17 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:18.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:17 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:53:18.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:17 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:18.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:17 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:53:18.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:17 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:18.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:17 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:53:18.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:17 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:18.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:17 smithi071 ceph-mon[32789]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:18.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:17 smithi071 ceph-mon[32789]: Detected new or changed devices on smithi071 2024-03-04T08:53:18.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:17 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:18.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:17 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:18.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:17 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:53:18.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:17 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:53:18.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:17 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:53:18.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:17 smithi071 ceph-mon[32789]: Adjusting osd_memory_target on smithi071 to 5696M 2024-03-04T08:53:18.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:17 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:18.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:17 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:18.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:17 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:53:18.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:17 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:18.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:17 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:53:18.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:17 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:18.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:17 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:53:18.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:17 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:18.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:17 smithi150 ceph-mon[36807]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:18.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:17 smithi150 ceph-mon[36807]: Detected new or changed devices on smithi071 2024-03-04T08:53:18.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:17 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:18.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:17 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:18.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:17 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:53:18.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:17 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:53:18.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:17 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:53:18.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:17 smithi150 ceph-mon[36807]: Adjusting osd_memory_target on smithi071 to 5696M 2024-03-04T08:53:18.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:17 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:18.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:17 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:18.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:17 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:53:18.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:17 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:18.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:17 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:53:18.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:17 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:18.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:17 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:53:18.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:17 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:19.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:18 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:53:19.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:18 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:53:19.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:18 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:19.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:18 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:53:19.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:18 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:53:19.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:18 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:19.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:18 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:53:19.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:18 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:53:19.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:18 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:20.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:19 smithi071 ceph-mon[32789]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:53:20.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:19 smithi071 ceph-mon[32789]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:20.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:19 smithi150 ceph-mon[36807]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:53:20.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:19 smithi150 ceph-mon[36807]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:20.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:19 smithi158 ceph-mon[37241]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:53:20.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:19 smithi158 ceph-mon[37241]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:21.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:20 smithi071 ceph-mon[32789]: from='client.? 172.21.15.150:0/3946026038' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6160a40a-d30f-4569-949c-0afb1bb3d133"}]: dispatch 2024-03-04T08:53:21.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:20 smithi071 ceph-mon[32789]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6160a40a-d30f-4569-949c-0afb1bb3d133"}]: dispatch 2024-03-04T08:53:21.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:20 smithi071 ceph-mon[32789]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6160a40a-d30f-4569-949c-0afb1bb3d133"}]': finished 2024-03-04T08:53:21.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:20 smithi071 ceph-mon[32789]: osdmap e49: 8 total, 7 up, 8 in 2024-03-04T08:53:21.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:20 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T08:53:21.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:20 smithi150 ceph-mon[36807]: from='client.? 172.21.15.150:0/3946026038' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6160a40a-d30f-4569-949c-0afb1bb3d133"}]: dispatch 2024-03-04T08:53:21.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:20 smithi150 ceph-mon[36807]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6160a40a-d30f-4569-949c-0afb1bb3d133"}]: dispatch 2024-03-04T08:53:21.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:20 smithi150 ceph-mon[36807]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6160a40a-d30f-4569-949c-0afb1bb3d133"}]': finished 2024-03-04T08:53:21.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:20 smithi150 ceph-mon[36807]: osdmap e49: 8 total, 7 up, 8 in 2024-03-04T08:53:21.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:20 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T08:53:21.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:20 smithi158 ceph-mon[37241]: from='client.? 172.21.15.150:0/3946026038' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6160a40a-d30f-4569-949c-0afb1bb3d133"}]: dispatch 2024-03-04T08:53:21.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:20 smithi158 ceph-mon[37241]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6160a40a-d30f-4569-949c-0afb1bb3d133"}]: dispatch 2024-03-04T08:53:21.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:20 smithi158 ceph-mon[37241]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6160a40a-d30f-4569-949c-0afb1bb3d133"}]': finished 2024-03-04T08:53:21.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:20 smithi158 ceph-mon[37241]: osdmap e49: 8 total, 7 up, 8 in 2024-03-04T08:53:21.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:20 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T08:53:22.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:21 smithi071 ceph-mon[32789]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:22.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:21 smithi071 ceph-mon[32789]: from='client.? 172.21.15.150:0/2729317986' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:53:22.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:21 smithi150 ceph-mon[36807]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:22.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:21 smithi150 ceph-mon[36807]: from='client.? 172.21.15.150:0/2729317986' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:53:22.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:21 smithi158 ceph-mon[37241]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:22.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:21 smithi158 ceph-mon[37241]: from='client.? 172.21.15.150:0/2729317986' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:53:24.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:23 smithi071 ceph-mon[32789]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:24.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:23 smithi150 ceph-mon[36807]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:24.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:23 smithi158 ceph-mon[37241]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:25.911 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:25 smithi150 ceph-mon[36807]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:26.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:25 smithi071 ceph-mon[32789]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:26.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:25 smithi158 ceph-mon[37241]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:28.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:27 smithi071 ceph-mon[32789]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:28.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:27 smithi150 ceph-mon[36807]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:28.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:27 smithi158 ceph-mon[37241]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:29.016 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:28 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-04T08:53:29.016 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:28 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:29.174 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:28 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-04T08:53:29.174 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:28 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:29.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:28 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-04T08:53:29.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:28 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:30.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:29 smithi150 ceph-mon[36807]: Deploying daemon osd.7 on smithi150 2024-03-04T08:53:30.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:29 smithi150 ceph-mon[36807]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:30.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:29 smithi071 ceph-mon[32789]: Deploying daemon osd.7 on smithi150 2024-03-04T08:53:30.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:29 smithi071 ceph-mon[32789]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:30.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:29 smithi158 ceph-mon[37241]: Deploying daemon osd.7 on smithi150 2024-03-04T08:53:30.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:29 smithi158 ceph-mon[37241]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:32.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:31 smithi150 ceph-mon[36807]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:32.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:31 smithi071 ceph-mon[32789]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:32.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:31 smithi158 ceph-mon[37241]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:34.104 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:33 smithi158 ceph-mon[37241]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:34.104 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:33 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:34.104 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:33 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:34.105 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:33 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:53:34.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:33 smithi071 ceph-mon[32789]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:34.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:33 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:34.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:33 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:34.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:33 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:53:34.236 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:33 smithi150 ceph-mon[36807]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:34.236 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:33 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:34.236 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:33 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:34.236 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:33 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:53:36.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:35 smithi071 ceph-mon[32789]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:36.183 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:35 smithi150 ceph-mon[36807]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:36.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:35 smithi158 ceph-mon[37241]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:37.243 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:37.244+0000 7f5b59ffb700 1 -- 172.21.15.150:0/4259880789 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5b5c062500 con 0x7f5b4c0a1e10 2024-03-04T08:53:37.244 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 7 on host 'smithi150' 2024-03-04T08:53:37.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:37.246+0000 7f5b62aaf700 1 -- 172.21.15.150:0/4259880789 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f5b4c0a1e10 msgr2=0x7f5b4c0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:53:37.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:37.246+0000 7f5b62aaf700 1 --2- 172.21.15.150:0/4259880789 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f5b4c0a1e10 0x7f5b4c0a42d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5b480099d0 tx=0x7f5b48008040 comp rx=0 tx=0).stop 2024-03-04T08:53:37.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:37.246+0000 7f5b62aaf700 1 -- 172.21.15.150:0/4259880789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b5c1020f0 msgr2=0x7f5b5c075e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:53:37.244 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:37.246+0000 7f5b62aaf700 1 --2- 172.21.15.150:0/4259880789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b5c1020f0 0x7f5b5c075e60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5b500098e0 tx=0x7f5b500047b0 comp rx=0 tx=0).stop 2024-03-04T08:53:37.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:37.246+0000 7f5b62aaf700 1 -- 172.21.15.150:0/4259880789 shutdown_connections 2024-03-04T08:53:37.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:37.246+0000 7f5b62aaf700 1 --2- 172.21.15.150:0/4259880789 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f5b4c0a1e10 0x7f5b4c0a42d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:53:37.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:37.246+0000 7f5b62aaf700 1 --2- 172.21.15.150:0/4259880789 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5b5c103c90 0x7f5b5c0749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:53:37.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:37.246+0000 7f5b62aaf700 1 --2- 172.21.15.150:0/4259880789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b5c1020f0 0x7f5b5c075e60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:53:37.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:37.246+0000 7f5b62aaf700 1 --2- 172.21.15.150:0/4259880789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b5c1032f0 0x7f5b5c0744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:53:37.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:37.246+0000 7f5b62aaf700 1 -- 172.21.15.150:0/4259880789 >> 172.21.15.150:0/4259880789 conn(0x7f5b5c0fd6d0 msgr2=0x7f5b5c106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:53:37.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:37.247+0000 7f5b62aaf700 1 -- 172.21.15.150:0/4259880789 shutdown_connections 2024-03-04T08:53:37.245 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:53:37.247+0000 7f5b62aaf700 1 -- 172.21.15.150:0/4259880789 wait complete. 2024-03-04T08:53:37.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:37 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:37.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:37 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:37.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:37 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:37.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:37 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:53:37.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:37 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:37.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:37 smithi150 ceph-mon[36807]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:37.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:37 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:53:37.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:37 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:37.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:37 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:37.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:37 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:37.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:37 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:37.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:37 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:37.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:37 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:53:37.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:37 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:37.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:37 smithi158 ceph-mon[37241]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:37.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:37 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:53:37.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:37 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:37.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:37 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:37.917 DEBUG:teuthology.orchestra.run.smithi150:osd.7> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.7.service 2024-03-04T08:53:37.919 INFO:tasks.cephadm:Deploying osd.8 on smithi158 with /dev/vg_nvme/lv_2... 2024-03-04T08:53:37.919 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-04T08:53:37.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:37 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:37.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:37 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:37.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:37 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:37.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:37 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:53:37.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:37 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:37.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:37 smithi071 ceph-mon[32789]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:37.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:37 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:53:37.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:37 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:37.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:37 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:38.167 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T08:53:38.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:38 smithi158 ceph-mon[37241]: from='osd.7 [v2:172.21.15.150:6816/1328330305,v1:172.21.15.150:6817/1328330305]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-04T08:53:38.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:38 smithi158 ceph-mon[37241]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-04T08:53:38.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:38 smithi150 ceph-mon[36807]: from='osd.7 [v2:172.21.15.150:6816/1328330305,v1:172.21.15.150:6817/1328330305]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-04T08:53:38.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:38 smithi150 ceph-mon[36807]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-04T08:53:38.823 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:38 smithi071 ceph-mon[32789]: from='osd.7 [v2:172.21.15.150:6816/1328330305,v1:172.21.15.150:6817/1328330305]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-04T08:53:38.823 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:38 smithi071 ceph-mon[32789]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-04T08:53:39.533 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:39 smithi150 ceph-mon[36807]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-04T08:53:39.533 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:39 smithi150 ceph-mon[36807]: from='osd.7 [v2:172.21.15.150:6816/1328330305,v1:172.21.15.150:6817/1328330305]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-04T08:53:39.534 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:39 smithi150 ceph-mon[36807]: osdmap e50: 8 total, 7 up, 8 in 2024-03-04T08:53:39.534 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:39 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T08:53:39.534 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:39 smithi150 ceph-mon[36807]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-04T08:53:39.534 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:39 smithi150 ceph-mon[36807]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:39.624 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:39 smithi158 ceph-mon[37241]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-04T08:53:39.624 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:39 smithi158 ceph-mon[37241]: from='osd.7 [v2:172.21.15.150:6816/1328330305,v1:172.21.15.150:6817/1328330305]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-04T08:53:39.624 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:39 smithi158 ceph-mon[37241]: osdmap e50: 8 total, 7 up, 8 in 2024-03-04T08:53:39.624 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:39 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T08:53:39.624 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:39 smithi158 ceph-mon[37241]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-04T08:53:39.624 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:39 smithi158 ceph-mon[37241]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:39.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:39 smithi071 ceph-mon[32789]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-04T08:53:39.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:39 smithi071 ceph-mon[32789]: from='osd.7 [v2:172.21.15.150:6816/1328330305,v1:172.21.15.150:6817/1328330305]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-04T08:53:39.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:39 smithi071 ceph-mon[32789]: osdmap e50: 8 total, 7 up, 8 in 2024-03-04T08:53:39.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:39 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T08:53:39.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:39 smithi071 ceph-mon[32789]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-04T08:53:39.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:39 smithi071 ceph-mon[32789]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T08:53:40.030 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 08:53:39 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T08:53:39.536+0000 7f11b075c700 -1 osd.7 0 waiting for initial osdmap 2024-03-04T08:53:40.030 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 08:53:39 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T08:53:39.544+0000 7f11ac54f700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T08:53:40.149 INFO:teuthology.orchestra.run.smithi158.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-04T08:53:40.149 INFO:teuthology.orchestra.run.smithi158.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-04T08:53:40.149 INFO:teuthology.orchestra.run.smithi158.stderr: stderr: 10+0 records in 2024-03-04T08:53:40.149 INFO:teuthology.orchestra.run.smithi158.stderr:10+0 records out 2024-03-04T08:53:40.149 INFO:teuthology.orchestra.run.smithi158.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00873503 s, 1.2 GB/s 2024-03-04T08:53:40.149 INFO:teuthology.orchestra.run.smithi158.stderr:--> Zapping successful for: 2024-03-04T08:53:40.742 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:40 smithi150 ceph-mon[36807]: purged_snaps scrub starts 2024-03-04T08:53:40.742 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:40 smithi150 ceph-mon[36807]: purged_snaps scrub ok 2024-03-04T08:53:40.742 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:40 smithi150 ceph-mon[36807]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-03-04T08:53:40.742 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:40 smithi150 ceph-mon[36807]: osdmap e51: 8 total, 7 up, 8 in 2024-03-04T08:53:40.742 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:40 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T08:53:40.764 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:40 smithi158 ceph-mon[37241]: purged_snaps scrub starts 2024-03-04T08:53:40.764 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:40 smithi158 ceph-mon[37241]: purged_snaps scrub ok 2024-03-04T08:53:40.764 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:40 smithi158 ceph-mon[37241]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-03-04T08:53:40.764 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:40 smithi158 ceph-mon[37241]: osdmap e51: 8 total, 7 up, 8 in 2024-03-04T08:53:40.764 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:40 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T08:53:40.765 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi158:vg_nvme/lv_2 2024-03-04T08:53:40.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:40 smithi071 ceph-mon[32789]: purged_snaps scrub starts 2024-03-04T08:53:40.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:40 smithi071 ceph-mon[32789]: purged_snaps scrub ok 2024-03-04T08:53:40.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:40 smithi071 ceph-mon[32789]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-03-04T08:53:40.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:40 smithi071 ceph-mon[32789]: osdmap e51: 8 total, 7 up, 8 in 2024-03-04T08:53:40.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:40 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T08:53:41.003 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T08:53:41.797 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:41 smithi158 ceph-mon[37241]: osd.7 [v2:172.21.15.150:6816/1328330305,v1:172.21.15.150:6817/1328330305] boot 2024-03-04T08:53:41.797 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:41 smithi158 ceph-mon[37241]: osdmap e52: 8 total, 8 up, 8 in 2024-03-04T08:53:41.798 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T08:53:41.798 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:41 smithi158 ceph-mon[37241]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:41.798 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:41 smithi158 ceph-mon[37241]: Detected new or changed devices on smithi150 2024-03-04T08:53:41.798 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:41.798 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:41.798 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:53:41.798 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:53:41.798 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:53:41.798 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:41 smithi158 ceph-mon[37241]: Adjusting osd_memory_target on smithi150 to 5693M 2024-03-04T08:53:41.798 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:41.798 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:41.798 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:53:41.798 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:41.799 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:41 smithi158 ceph-mon[37241]: osdmap e53: 8 total, 8 up, 8 in 2024-03-04T08:53:41.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:41 smithi071 ceph-mon[32789]: osd.7 [v2:172.21.15.150:6816/1328330305,v1:172.21.15.150:6817/1328330305] boot 2024-03-04T08:53:41.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:41 smithi071 ceph-mon[32789]: osdmap e52: 8 total, 8 up, 8 in 2024-03-04T08:53:41.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T08:53:41.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:41 smithi071 ceph-mon[32789]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:41.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:41 smithi071 ceph-mon[32789]: Detected new or changed devices on smithi150 2024-03-04T08:53:41.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:41.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:41.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:53:41.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:53:41.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:53:41.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:41 smithi071 ceph-mon[32789]: Adjusting osd_memory_target on smithi150 to 5693M 2024-03-04T08:53:41.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:41.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:41.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:53:41.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:41.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:41 smithi071 ceph-mon[32789]: osdmap e53: 8 total, 8 up, 8 in 2024-03-04T08:53:42.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:41 smithi150 ceph-mon[36807]: osd.7 [v2:172.21.15.150:6816/1328330305,v1:172.21.15.150:6817/1328330305] boot 2024-03-04T08:53:42.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:41 smithi150 ceph-mon[36807]: osdmap e52: 8 total, 8 up, 8 in 2024-03-04T08:53:42.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T08:53:42.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:41 smithi150 ceph-mon[36807]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:42.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:41 smithi150 ceph-mon[36807]: Detected new or changed devices on smithi150 2024-03-04T08:53:42.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:42.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:42.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:53:42.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:53:42.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:53:42.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:41 smithi150 ceph-mon[36807]: Adjusting osd_memory_target on smithi150 to 5693M 2024-03-04T08:53:42.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:42.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:42.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:53:42.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:42.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:41 smithi150 ceph-mon[36807]: osdmap e53: 8 total, 8 up, 8 in 2024-03-04T08:53:42.663 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.667+0000 7fabd5b4b700 1 -- 172.21.15.158:0/2977776244 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fabd0100410 msgr2=0x7fabd0100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:53:42.663 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.667+0000 7fabd5b4b700 1 --2- 172.21.15.158:0/2977776244 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fabd0100410 0x7fabd0100830 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009990 tx=0x7fabc0009ca0 comp rx=0 tx=0).stop 2024-03-04T08:53:42.663 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.667+0000 7fabd5b4b700 1 -- 172.21.15.158:0/2977776244 shutdown_connections 2024-03-04T08:53:42.663 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.667+0000 7fabd5b4b700 1 --2- 172.21.15.158:0/2977776244 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fabd0100410 0x7fabd0100830 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:53:42.663 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.667+0000 7fabd5b4b700 1 --2- 172.21.15.158:0/2977776244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabd0103e30 0x7fabd010a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:53:42.663 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.667+0000 7fabd5b4b700 1 --2- 172.21.15.158:0/2977776244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd0103490 0x7fabd01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:53:42.663 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.667+0000 7fabd5b4b700 1 -- 172.21.15.158:0/2977776244 >> 172.21.15.158:0/2977776244 conn(0x7fabd00fbd50 msgr2=0x7fabd00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:53:42.664 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.667+0000 7fabd5b4b700 1 -- 172.21.15.158:0/2977776244 shutdown_connections 2024-03-04T08:53:42.664 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.667+0000 7fabd5b4b700 1 -- 172.21.15.158:0/2977776244 wait complete. 2024-03-04T08:53:42.664 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.667+0000 7fabd5b4b700 1 Processor -- start 2024-03-04T08:53:42.664 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.667+0000 7fabd5b4b700 1 -- start start 2024-03-04T08:53:42.666 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.668+0000 7fabd5b4b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd0100410 0x7fabd0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:53:42.666 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.668+0000 7fabd5b4b700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fabd0103490 0x7fabd0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:53:42.666 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.668+0000 7fabd5b4b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabd0103e30 0x7fabd019f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:53:42.666 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.668+0000 7fabd5b4b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd010dd20 con 0x7fabd0103e30 2024-03-04T08:53:42.666 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.668+0000 7fabd5b4b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd010dba0 con 0x7fabd0100410 2024-03-04T08:53:42.666 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.668+0000 7fabd5b4b700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd010dea0 con 0x7fabd0103490 2024-03-04T08:53:42.666 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.668+0000 7fabceffd700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fabd0103490 0x7fabd0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:53:42.667 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.668+0000 7fabceffd700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fabd0103490 0x7fabd0198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.158:46938/0 (socket says 172.21.15.158:46938) 2024-03-04T08:53:42.667 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.668+0000 7fabceffd700 1 -- 172.21.15.158:0/3733838292 learned_addr learned my addr 172.21.15.158:0/3733838292 (peer_addr_for_me v2:172.21.15.158:0/0) 2024-03-04T08:53:42.667 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.668+0000 7fabcf7fe700 1 --2- 172.21.15.158:0/3733838292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd0100410 0x7fabd0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:53:42.667 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.668+0000 7fabcffff700 1 --2- 172.21.15.158:0/3733838292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabd0103e30 0x7fabd019f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:53:42.667 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.668+0000 7fabceffd700 1 -- 172.21.15.158:0/3733838292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd0100410 msgr2=0x7fabd0198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:53:42.667 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.668+0000 7fabceffd700 1 --2- 172.21.15.158:0/3733838292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd0100410 0x7fabd0198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:53:42.667 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.668+0000 7fabceffd700 1 -- 172.21.15.158:0/3733838292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabd0103e30 msgr2=0x7fabd019f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:53:42.667 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.668+0000 7fabceffd700 1 --2- 172.21.15.158:0/3733838292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabd0103e30 0x7fabd019f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:53:42.667 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.668+0000 7fabceffd700 1 -- 172.21.15.158:0/3733838292 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd019ff90 con 0x7fabd0103490 2024-03-04T08:53:42.667 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.668+0000 7fabceffd700 1 --2- 172.21.15.158:0/3733838292 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fabd0103490 0x7fabd0198870 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fabc400d9b0 tx=0x7fabc400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:53:42.667 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.668+0000 7fabccff9700 1 -- 172.21.15.158:0/3733838292 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc4017070 con 0x7fabd0103490 2024-03-04T08:53:42.667 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.668+0000 7fabccff9700 1 -- 172.21.15.158:0/3733838292 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fabc4005020 con 0x7fabd0103490 2024-03-04T08:53:42.667 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.668+0000 7fabd5b4b700 1 -- 172.21.15.158:0/3733838292 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd01a0280 con 0x7fabd0103490 2024-03-04T08:53:42.667 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.668+0000 7fabccff9700 1 -- 172.21.15.158:0/3733838292 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc4009a90 con 0x7fabd0103490 2024-03-04T08:53:42.668 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.668+0000 7fabd5b4b700 1 -- 172.21.15.158:0/3733838292 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd01a0770 con 0x7fabd0103490 2024-03-04T08:53:42.668 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.669+0000 7fabd5b4b700 1 -- 172.21.15.158:0/3733838292 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd0068d80 con 0x7fabd0103490 2024-03-04T08:53:42.668 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.670+0000 7fabccff9700 1 -- 172.21.15.158:0/3733838292 <== mon.1 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fabc40051d0 con 0x7fabd0103490 2024-03-04T08:53:42.668 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.671+0000 7fabccff9700 1 --2- 172.21.15.158:0/3733838292 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fabb80a1e10 0x7fabb80a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:53:42.669 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.671+0000 7fabccff9700 1 -- 172.21.15.158:0/3733838292 <== mon.1 v2:172.21.15.158:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5426+0+0 (secure 0 0 0) 0x7fabc40cd0f0 con 0x7fabd0103490 2024-03-04T08:53:42.669 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.671+0000 7fabcf7fe700 1 --2- 172.21.15.158:0/3733838292 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fabb80a1e10 0x7fabb80a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:53:42.669 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.672+0000 7fabcf7fe700 1 --2- 172.21.15.158:0/3733838292 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fabb80a1e10 0x7fabb80a42d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fabc0003890 tx=0x7fabc0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:53:42.671 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.676+0000 7fabccff9700 1 -- 172.21.15.158:0/3733838292 <== mon.1 v2:172.21.15.158:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fabc4020020 con 0x7fabd0103490 2024-03-04T08:53:42.872 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:53:42.876+0000 7fabd5b4b700 1 -- 172.21.15.158:0/3733838292 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fabd0062500 con 0x7fabb80a1e10 2024-03-04T08:53:43.797 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:43 smithi158 ceph-mon[37241]: osdmap e54: 8 total, 8 up, 8 in 2024-03-04T08:53:43.797 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:43 smithi158 ceph-mon[37241]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:43.797 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:43 smithi158 ceph-mon[37241]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:53:43.797 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:43 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:53:43.797 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:43 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:53:43.797 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:43 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:43.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:43 smithi071 ceph-mon[32789]: osdmap e54: 8 total, 8 up, 8 in 2024-03-04T08:53:43.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:43 smithi071 ceph-mon[32789]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:43.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:43 smithi071 ceph-mon[32789]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:53:43.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:43 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:53:43.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:43 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:53:43.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:43 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:44.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:43 smithi150 ceph-mon[36807]: osdmap e54: 8 total, 8 up, 8 in 2024-03-04T08:53:44.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:43 smithi150 ceph-mon[36807]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:44.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:43 smithi150 ceph-mon[36807]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:53:44.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:43 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:53:44.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:43 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:53:44.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:43 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:44.695 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:44 smithi158 ceph-mon[37241]: osdmap e55: 8 total, 8 up, 8 in 2024-03-04T08:53:44.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:44 smithi071 ceph-mon[32789]: osdmap e55: 8 total, 8 up, 8 in 2024-03-04T08:53:45.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:44 smithi150 ceph-mon[36807]: osdmap e55: 8 total, 8 up, 8 in 2024-03-04T08:53:45.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:45 smithi158 ceph-mon[37241]: osdmap e56: 8 total, 8 up, 8 in 2024-03-04T08:53:45.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:45 smithi158 ceph-mon[37241]: pgmap v207: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:45.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:45 smithi158 ceph-mon[37241]: from='client.? 172.21.15.158:0/203301143' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95ed2341-84a4-465c-9948-ac667680ce34"}]: dispatch 2024-03-04T08:53:45.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:45 smithi158 ceph-mon[37241]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95ed2341-84a4-465c-9948-ac667680ce34"}]: dispatch 2024-03-04T08:53:45.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:45 smithi158 ceph-mon[37241]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95ed2341-84a4-465c-9948-ac667680ce34"}]': finished 2024-03-04T08:53:45.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:45 smithi158 ceph-mon[37241]: osdmap e57: 9 total, 8 up, 9 in 2024-03-04T08:53:45.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:45 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:53:45.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:45 smithi071 ceph-mon[32789]: osdmap e56: 8 total, 8 up, 8 in 2024-03-04T08:53:45.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:45 smithi071 ceph-mon[32789]: pgmap v207: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:45.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:45 smithi071 ceph-mon[32789]: from='client.? 172.21.15.158:0/203301143' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95ed2341-84a4-465c-9948-ac667680ce34"}]: dispatch 2024-03-04T08:53:45.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:45 smithi071 ceph-mon[32789]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95ed2341-84a4-465c-9948-ac667680ce34"}]: dispatch 2024-03-04T08:53:45.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:45 smithi071 ceph-mon[32789]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95ed2341-84a4-465c-9948-ac667680ce34"}]': finished 2024-03-04T08:53:45.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:45 smithi071 ceph-mon[32789]: osdmap e57: 9 total, 8 up, 9 in 2024-03-04T08:53:45.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:45 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:53:46.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:45 smithi150 ceph-mon[36807]: osdmap e56: 8 total, 8 up, 8 in 2024-03-04T08:53:46.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:45 smithi150 ceph-mon[36807]: pgmap v207: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:46.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:45 smithi150 ceph-mon[36807]: from='client.? 172.21.15.158:0/203301143' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95ed2341-84a4-465c-9948-ac667680ce34"}]: dispatch 2024-03-04T08:53:46.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:45 smithi150 ceph-mon[36807]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95ed2341-84a4-465c-9948-ac667680ce34"}]: dispatch 2024-03-04T08:53:46.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:45 smithi150 ceph-mon[36807]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95ed2341-84a4-465c-9948-ac667680ce34"}]': finished 2024-03-04T08:53:46.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:45 smithi150 ceph-mon[36807]: osdmap e57: 9 total, 8 up, 9 in 2024-03-04T08:53:46.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:45 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:53:46.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:46 smithi158 ceph-mon[37241]: from='client.? 172.21.15.158:0/1096012830' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:53:46.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:46 smithi071 ceph-mon[32789]: from='client.? 172.21.15.158:0/1096012830' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:53:47.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:46 smithi150 ceph-mon[36807]: from='client.? 172.21.15.158:0/1096012830' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:53:47.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:47 smithi071 ceph-mon[32789]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:48.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:47 smithi150 ceph-mon[36807]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:48.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:47 smithi158 ceph-mon[37241]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:49.768 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:49 smithi158 ceph-mon[37241]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:49.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:49 smithi071 ceph-mon[32789]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:50.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:49 smithi150 ceph-mon[36807]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:51.988 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:51 smithi158 ceph-mon[37241]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:52.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:51 smithi150 ceph-mon[36807]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:52.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:51 smithi071 ceph-mon[32789]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:54.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:53 smithi150 ceph-mon[36807]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:54.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:53 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-04T08:53:54.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:53 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:54.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:53 smithi150 ceph-mon[36807]: Deploying daemon osd.8 on smithi158 2024-03-04T08:53:54.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:53 smithi158 ceph-mon[37241]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:54.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:53 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-04T08:53:54.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:53 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:54.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:53 smithi158 ceph-mon[37241]: Deploying daemon osd.8 on smithi158 2024-03-04T08:53:54.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:53 smithi071 ceph-mon[32789]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:54.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:53 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-04T08:53:54.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:53 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:53:54.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:53 smithi071 ceph-mon[32789]: Deploying daemon osd.8 on smithi158 2024-03-04T08:53:56.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:55 smithi150 ceph-mon[36807]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:56.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:55 smithi158 ceph-mon[37241]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:56.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:55 smithi071 ceph-mon[32789]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:57.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:57 smithi158 ceph-mon[37241]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:58.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:57 smithi150 ceph-mon[36807]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:58.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:57 smithi071 ceph-mon[32789]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:53:59.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:59 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:59.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:59 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:53:59.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:59 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:53:59.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:53:59 smithi071 ceph-mon[32789]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:54:00.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:59 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:00.212 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:59 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:54:00.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:59 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:00.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:53:59 smithi150 ceph-mon[36807]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:54:00.214 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:59 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:00.214 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:59 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:54:00.214 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:59 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:00.214 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:53:59 smithi158 ceph-mon[37241]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:54:02.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:01 smithi150 ceph-mon[36807]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:54:02.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:01 smithi158 ceph-mon[37241]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:54:02.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:01 smithi071 ceph-mon[32789]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:54:03.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:02 smithi150 ceph-mon[36807]: from='osd.8 [v2:172.21.15.158:6816/2195304577,v1:172.21.15.158:6817/2195304577]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-04T08:54:03.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:02 smithi150 ceph-mon[36807]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-04T08:54:03.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:02 smithi158 ceph-mon[37241]: from='osd.8 [v2:172.21.15.158:6816/2195304577,v1:172.21.15.158:6817/2195304577]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-04T08:54:03.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:02 smithi158 ceph-mon[37241]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-04T08:54:03.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:02 smithi071 ceph-mon[32789]: from='osd.8 [v2:172.21.15.158:6816/2195304577,v1:172.21.15.158:6817/2195304577]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-04T08:54:03.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:02 smithi071 ceph-mon[32789]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-04T08:54:04.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:03 smithi150 ceph-mon[36807]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:54:04.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:03 smithi150 ceph-mon[36807]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-04T08:54:04.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:03 smithi150 ceph-mon[36807]: osdmap e58: 9 total, 8 up, 9 in 2024-03-04T08:54:04.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:03 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:04.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:03 smithi150 ceph-mon[36807]: from='osd.8 [v2:172.21.15.158:6816/2195304577,v1:172.21.15.158:6817/2195304577]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-03-04T08:54:04.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:03 smithi150 ceph-mon[36807]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-03-04T08:54:04.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:03 smithi158 ceph-mon[37241]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:54:04.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:03 smithi071 ceph-mon[32789]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:54:04.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:03 smithi071 ceph-mon[32789]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-04T08:54:04.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:03 smithi071 ceph-mon[32789]: osdmap e58: 9 total, 8 up, 9 in 2024-03-04T08:54:04.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:03 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:04.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:03 smithi071 ceph-mon[32789]: from='osd.8 [v2:172.21.15.158:6816/2195304577,v1:172.21.15.158:6817/2195304577]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-03-04T08:54:04.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:03 smithi071 ceph-mon[32789]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-03-04T08:54:07.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:07 smithi158 ceph-mon[37241]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-04T08:54:07.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:07 smithi158 ceph-mon[37241]: osdmap e58: 9 total, 8 up, 9 in 2024-03-04T08:54:07.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:07 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:07.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:07 smithi158 ceph-mon[37241]: from='osd.8 [v2:172.21.15.158:6816/2195304577,v1:172.21.15.158:6817/2195304577]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-03-04T08:54:07.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:07 smithi158 ceph-mon[37241]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-03-04T08:54:07.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:07 smithi158 ceph-mon[37241]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]': finished 2024-03-04T08:54:07.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:07 smithi158 ceph-mon[37241]: osdmap e59: 9 total, 8 up, 9 in 2024-03-04T08:54:07.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:07 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:07.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:07 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:07.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:07 smithi071 ceph-mon[32789]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]': finished 2024-03-04T08:54:07.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:07 smithi071 ceph-mon[32789]: osdmap e59: 9 total, 8 up, 9 in 2024-03-04T08:54:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:07 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:07 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:07.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:07 smithi150 ceph-mon[36807]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]': finished 2024-03-04T08:54:07.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:07 smithi150 ceph-mon[36807]: osdmap e59: 9 total, 8 up, 9 in 2024-03-04T08:54:07.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:07 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:07.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:07 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:08.225 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:08 smithi158 ceph-mon[37241]: purged_snaps scrub starts 2024-03-04T08:54:08.225 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:08 smithi158 ceph-mon[37241]: purged_snaps scrub ok 2024-03-04T08:54:08.225 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:08 smithi158 ceph-mon[37241]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:54:08.225 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:08 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:08.225 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:08 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:08.226 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:08 smithi158 ceph-mon[37241]: pgmap v221: 1 pgs: 1 peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:54:08.226 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:08 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:08.226 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:08 smithi158 ceph-mon[37241]: osdmap e60: 9 total, 8 up, 9 in 2024-03-04T08:54:08.226 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:08 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:08.226 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:08 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:08.324 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:08 smithi071 ceph-mon[32789]: purged_snaps scrub starts 2024-03-04T08:54:08.324 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:08 smithi071 ceph-mon[32789]: purged_snaps scrub ok 2024-03-04T08:54:08.324 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:08 smithi071 ceph-mon[32789]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:54:08.324 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:08 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:08.324 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:08 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:08.324 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:08 smithi071 ceph-mon[32789]: pgmap v221: 1 pgs: 1 peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:54:08.324 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:08 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:08.324 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:08 smithi071 ceph-mon[32789]: osdmap e60: 9 total, 8 up, 9 in 2024-03-04T08:54:08.324 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:08 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:08.324 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:08 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:08.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:08 smithi150 ceph-mon[36807]: purged_snaps scrub starts 2024-03-04T08:54:08.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:08 smithi150 ceph-mon[36807]: purged_snaps scrub ok 2024-03-04T08:54:08.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:08 smithi150 ceph-mon[36807]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:54:08.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:08 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:08.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:08 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:08.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:08 smithi150 ceph-mon[36807]: pgmap v221: 1 pgs: 1 peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:54:08.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:08 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:08.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:08 smithi150 ceph-mon[36807]: osdmap e60: 9 total, 8 up, 9 in 2024-03-04T08:54:08.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:08 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:08.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:08 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:09.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:09 smithi158 ceph-mon[37241]: osd.8 [v2:172.21.15.158:6816/2195304577,v1:172.21.15.158:6817/2195304577] boot 2024-03-04T08:54:09.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:09 smithi158 ceph-mon[37241]: osdmap e61: 9 total, 9 up, 9 in 2024-03-04T08:54:09.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:09 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:09.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:09 smithi158 ceph-mon[37241]: pgmap v224: 1 pgs: 1 peering; 577 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:54:09.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:09 smithi071 ceph-mon[32789]: osd.8 [v2:172.21.15.158:6816/2195304577,v1:172.21.15.158:6817/2195304577] boot 2024-03-04T08:54:09.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:09 smithi071 ceph-mon[32789]: osdmap e61: 9 total, 9 up, 9 in 2024-03-04T08:54:09.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:09 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:09.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:09 smithi071 ceph-mon[32789]: pgmap v224: 1 pgs: 1 peering; 577 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:54:09.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:09 smithi150 ceph-mon[36807]: osd.8 [v2:172.21.15.158:6816/2195304577,v1:172.21.15.158:6817/2195304577] boot 2024-03-04T08:54:09.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:09 smithi150 ceph-mon[36807]: osdmap e61: 9 total, 9 up, 9 in 2024-03-04T08:54:09.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:09 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:54:09.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:09 smithi150 ceph-mon[36807]: pgmap v224: 1 pgs: 1 peering; 577 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-03-04T08:54:10.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:10 smithi071 ceph-mon[32789]: osdmap e62: 9 total, 9 up, 9 in 2024-03-04T08:54:10.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:10 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:10.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:10 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:10.463 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:10 smithi158 ceph-mon[37241]: osdmap e62: 9 total, 9 up, 9 in 2024-03-04T08:54:10.464 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:10 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:10.464 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:10 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:10.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:10 smithi150 ceph-mon[36807]: osdmap e62: 9 total, 9 up, 9 in 2024-03-04T08:54:10.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:10 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:10.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:10 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:10.867 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:54:10.871+0000 7fabccff9700 1 -- 172.21.15.158:0/3733838292 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fabd0062500 con 0x7fabb80a1e10 2024-03-04T08:54:10.867 INFO:teuthology.orchestra.run.smithi158.stdout:Created osd(s) 8 on host 'smithi158' 2024-03-04T08:54:10.869 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:54:10.873+0000 7fabd5b4b700 1 -- 172.21.15.158:0/3733838292 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fabb80a1e10 msgr2=0x7fabb80a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:54:10.869 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:54:10.873+0000 7fabd5b4b700 1 --2- 172.21.15.158:0/3733838292 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fabb80a1e10 0x7fabb80a42d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fabc0003890 tx=0x7fabc0005fd0 comp rx=0 tx=0).stop 2024-03-04T08:54:10.869 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:54:10.873+0000 7fabd5b4b700 1 -- 172.21.15.158:0/3733838292 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fabd0103490 msgr2=0x7fabd0198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:54:10.869 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:54:10.873+0000 7fabd5b4b700 1 --2- 172.21.15.158:0/3733838292 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fabd0103490 0x7fabd0198870 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fabc400d9b0 tx=0x7fabc400dcc0 comp rx=0 tx=0).stop 2024-03-04T08:54:10.869 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:54:10.873+0000 7fabd5b4b700 1 -- 172.21.15.158:0/3733838292 shutdown_connections 2024-03-04T08:54:10.869 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:54:10.873+0000 7fabd5b4b700 1 --2- 172.21.15.158:0/3733838292 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fabb80a1e10 0x7fabb80a42d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:54:10.870 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:54:10.873+0000 7fabd5b4b700 1 --2- 172.21.15.158:0/3733838292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabd0103e30 0x7fabd019f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:54:10.870 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:54:10.873+0000 7fabd5b4b700 1 --2- 172.21.15.158:0/3733838292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabd0100410 0x7fabd0198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:54:10.870 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:54:10.873+0000 7fabd5b4b700 1 --2- 172.21.15.158:0/3733838292 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fabd0103490 0x7fabd0198870 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:54:10.870 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:54:10.873+0000 7fabd5b4b700 1 -- 172.21.15.158:0/3733838292 >> 172.21.15.158:0/3733838292 conn(0x7fabd00fbd50 msgr2=0x7fabd0106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:54:10.870 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:54:10.873+0000 7fabd5b4b700 1 -- 172.21.15.158:0/3733838292 shutdown_connections 2024-03-04T08:54:10.870 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:54:10.873+0000 7fabd5b4b700 1 -- 172.21.15.158:0/3733838292 wait complete. 2024-03-04T08:54:11.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:11 smithi071 ceph-mon[32789]: osdmap e63: 9 total, 9 up, 9 in 2024-03-04T08:54:11.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:11 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:54:11.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:11 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:54:11.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:11 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:11.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:11 smithi071 ceph-mon[32789]: pgmap v227: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-04T08:54:11.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:11 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:11.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:11 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:11.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:11 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:54:11.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:11 smithi150 ceph-mon[36807]: osdmap e63: 9 total, 9 up, 9 in 2024-03-04T08:54:11.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:11 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:54:11.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:11 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:54:11.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:11 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:11.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:11 smithi150 ceph-mon[36807]: pgmap v227: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-04T08:54:11.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:11 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:11.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:11 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:11.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:11 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:54:11.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:11 smithi158 ceph-mon[37241]: osdmap e63: 9 total, 9 up, 9 in 2024-03-04T08:54:11.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:11 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:54:11.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:11 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:54:11.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:11 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:11.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:11 smithi158 ceph-mon[37241]: pgmap v227: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-04T08:54:11.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:11 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:11.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:11 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:11.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:11 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:54:11.782 DEBUG:teuthology.orchestra.run.smithi158:osd.8> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.8.service 2024-03-04T08:54:11.784 INFO:tasks.cephadm:Deploying osd.9 on smithi071 with /dev/vg_nvme/lv_1... 2024-03-04T08:54:11.784 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-04T08:54:12.031 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:54:12.386 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:12 smithi071 ceph-mon[32789]: osdmap e64: 9 total, 9 up, 9 in 2024-03-04T08:54:12.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:12 smithi150 ceph-mon[36807]: osdmap e64: 9 total, 9 up, 9 in 2024-03-04T08:54:12.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:12 smithi158 ceph-mon[37241]: osdmap e64: 9 total, 9 up, 9 in 2024-03-04T08:54:13.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:13 smithi158 ceph-mon[37241]: osdmap e65: 9 total, 9 up, 9 in 2024-03-04T08:54:13.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:13 smithi158 ceph-mon[37241]: pgmap v230: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-04T08:54:13.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:13 smithi071 ceph-mon[32789]: osdmap e65: 9 total, 9 up, 9 in 2024-03-04T08:54:13.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:13 smithi071 ceph-mon[32789]: pgmap v230: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-04T08:54:13.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:13 smithi150 ceph-mon[36807]: osdmap e65: 9 total, 9 up, 9 in 2024-03-04T08:54:13.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:13 smithi150 ceph-mon[36807]: pgmap v230: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-04T08:54:14.110 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-04T08:54:14.110 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-04T08:54:14.110 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-03-04T08:54:14.110 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-03-04T08:54:14.111 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114636 s, 915 MB/s 2024-03-04T08:54:14.111 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-03-04T08:54:14.868 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi071:vg_nvme/lv_1 2024-03-04T08:54:15.116 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:54:15.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:15 smithi071 ceph-mon[32789]: pgmap v231: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-03-04T08:54:15.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:15 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:15.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:15 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:15.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:15 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:54:15.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:15 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:54:15.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:15 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:54:15.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:15 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:15.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:15 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:54:15.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:15 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:54:15.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:15 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:16.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:15 smithi150 ceph-mon[36807]: pgmap v231: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-03-04T08:54:16.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:15 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:16.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:15 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:16.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:15 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:54:16.033 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:15 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:54:16.033 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:15 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:54:16.033 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:15 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:16.033 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:15 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:54:16.033 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:15 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:54:16.033 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:15 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:16.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:15 smithi158 ceph-mon[37241]: pgmap v231: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-03-04T08:54:16.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:16.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:16.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:54:16.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:54:16.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:54:16.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:16.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:54:16.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:54:16.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:16.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.823+0000 7fb9174c1700 1 -- 172.21.15.71:0/169486478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9101021f0 msgr2=0x7fb910102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:54:16.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.823+0000 7fb9174c1700 1 --2- 172.21.15.71:0/169486478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9101021f0 0x7fb910102650 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb9000099c0 tx=0x7fb900009cd0 comp rx=0 tx=0).stop 2024-03-04T08:54:16.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.823+0000 7fb9174c1700 1 -- 172.21.15.71:0/169486478 shutdown_connections 2024-03-04T08:54:16.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.823+0000 7fb9174c1700 1 --2- 172.21.15.71:0/169486478 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb910100ff0 0x7fb910101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:54:16.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.823+0000 7fb9174c1700 1 --2- 172.21.15.71:0/169486478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb910102b90 0x7fb910109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:54:16.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.823+0000 7fb9174c1700 1 --2- 172.21.15.71:0/169486478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9101021f0 0x7fb910102650 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:54:16.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.823+0000 7fb9174c1700 1 -- 172.21.15.71:0/169486478 >> 172.21.15.71:0/169486478 conn(0x7fb9100fc590 msgr2=0x7fb9100fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:54:16.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.823+0000 7fb9174c1700 1 -- 172.21.15.71:0/169486478 shutdown_connections 2024-03-04T08:54:16.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.823+0000 7fb9174c1700 1 -- 172.21.15.71:0/169486478 wait complete. 2024-03-04T08:54:16.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.824+0000 7fb9174c1700 1 Processor -- start 2024-03-04T08:54:16.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.824+0000 7fb9174c1700 1 -- start start 2024-03-04T08:54:16.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.824+0000 7fb9174c1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb910100ff0 0x7fb9101984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:54:16.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.824+0000 7fb9174c1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9101021f0 0x7fb910198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:54:16.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.824+0000 7fb9174c1700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb910102b90 0x7fb91019faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:54:16.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.824+0000 7fb9174c1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91010ccd0 con 0x7fb9101021f0 2024-03-04T08:54:16.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.824+0000 7fb9174c1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91010cb50 con 0x7fb910100ff0 2024-03-04T08:54:16.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.824+0000 7fb9174c1700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91010ce50 con 0x7fb910102b90 2024-03-04T08:54:16.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.824+0000 7fb914a5c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9101021f0 0x7fb910198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:54:16.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.824+0000 7fb914a5c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9101021f0 0x7fb910198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57992/0 (socket says 172.21.15.71:57992) 2024-03-04T08:54:16.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.824+0000 7fb914a5c700 1 -- 172.21.15.71:0/2585649929 learned_addr learned my addr 172.21.15.71:0/2585649929 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:54:16.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.824+0000 7fb915a5e700 1 --2- 172.21.15.71:0/2585649929 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb910102b90 0x7fb91019faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:54:16.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.824+0000 7fb91525d700 1 --2- 172.21.15.71:0/2585649929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb910100ff0 0x7fb9101984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:54:16.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.825+0000 7fb914a5c700 1 -- 172.21.15.71:0/2585649929 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb910102b90 msgr2=0x7fb91019faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:54:16.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.825+0000 7fb914a5c700 1 --2- 172.21.15.71:0/2585649929 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb910102b90 0x7fb91019faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:54:16.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.825+0000 7fb914a5c700 1 -- 172.21.15.71:0/2585649929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb910100ff0 msgr2=0x7fb9101984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:54:16.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.825+0000 7fb914a5c700 1 --2- 172.21.15.71:0/2585649929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb910100ff0 0x7fb9101984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:54:16.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.825+0000 7fb914a5c700 1 -- 172.21.15.71:0/2585649929 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9101a0140 con 0x7fb9101021f0 2024-03-04T08:54:16.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.825+0000 7fb91525d700 1 --2- 172.21.15.71:0/2585649929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb910100ff0 0x7fb9101984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:54:16.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.825+0000 7fb914a5c700 1 --2- 172.21.15.71:0/2585649929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9101021f0 0x7fb910198a20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb900000c00 tx=0x7fb900004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:54:16.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.825+0000 7fb9067fc700 1 -- 172.21.15.71:0/2585649929 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb900027070 con 0x7fb9101021f0 2024-03-04T08:54:16.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.825+0000 7fb9067fc700 1 -- 172.21.15.71:0/2585649929 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb900004150 con 0x7fb9101021f0 2024-03-04T08:54:16.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.825+0000 7fb9174c1700 1 -- 172.21.15.71:0/2585649929 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9101a03d0 con 0x7fb9101021f0 2024-03-04T08:54:16.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.825+0000 7fb9067fc700 1 -- 172.21.15.71:0/2585649929 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90000f460 con 0x7fb9101021f0 2024-03-04T08:54:16.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.825+0000 7fb9174c1700 1 -- 172.21.15.71:0/2585649929 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9101a0900 con 0x7fb9101021f0 2024-03-04T08:54:16.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.826+0000 7fb9174c1700 1 -- 172.21.15.71:0/2585649929 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb910068d80 con 0x7fb9101021f0 2024-03-04T08:54:16.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.827+0000 7fb9067fc700 1 -- 172.21.15.71:0/2585649929 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fb90002a070 con 0x7fb9101021f0 2024-03-04T08:54:16.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.828+0000 7fb9067fc700 1 --2- 172.21.15.71:0/2585649929 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb8fc0a1e90 0x7fb8fc0a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:54:16.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.828+0000 7fb91525d700 1 --2- 172.21.15.71:0/2585649929 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb8fc0a1e90 0x7fb8fc0a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:54:16.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.828+0000 7fb9067fc700 1 -- 172.21.15.71:0/2585649929 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5829+0+0 (secure 0 0 0) 0x7fb9000ce750 con 0x7fb9101021f0 2024-03-04T08:54:16.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.828+0000 7fb91525d700 1 --2- 172.21.15.71:0/2585649929 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb8fc0a1e90 0x7fb8fc0a4350 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb910102050 tx=0x7fb90c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:54:16.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:16.832+0000 7fb9067fc700 1 -- 172.21.15.71:0/2585649929 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb900031070 con 0x7fb9101021f0 2024-03-04T08:54:16.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:16 smithi071 ceph-mon[32789]: Detected new or changed devices on smithi158 2024-03-04T08:54:16.925 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:16 smithi071 ceph-mon[32789]: Adjusting osd_memory_target on smithi158 to 5693M 2024-03-04T08:54:17.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:16 smithi150 ceph-mon[36807]: Detected new or changed devices on smithi158 2024-03-04T08:54:17.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:16 smithi150 ceph-mon[36807]: Adjusting osd_memory_target on smithi158 to 5693M 2024-03-04T08:54:17.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:17.040+0000 7fb9174c1700 1 -- 172.21.15.71:0/2585649929 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb91006a4d0 con 0x7fb8fc0a1e90 2024-03-04T08:54:17.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:16 smithi158 ceph-mon[37241]: Detected new or changed devices on smithi158 2024-03-04T08:54:17.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:16 smithi158 ceph-mon[37241]: Adjusting osd_memory_target on smithi158 to 5693M 2024-03-04T08:54:17.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:17 smithi071 ceph-mon[32789]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-04T08:54:17.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:17 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:54:17.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:17 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:54:17.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:17 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:54:18.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:17 smithi150 ceph-mon[36807]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-04T08:54:18.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:17 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:54:18.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:17 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:54:18.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:17 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:54:18.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:17 smithi158 ceph-mon[37241]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-04T08:54:18.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:17 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:54:18.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:17 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:54:18.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:17 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:54:18.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:18 smithi071 ceph-mon[32789]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:54:19.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:18 smithi150 ceph-mon[36807]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:54:19.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:18 smithi158 ceph-mon[37241]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:54:20.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:19 smithi150 ceph-mon[36807]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:20.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:19 smithi158 ceph-mon[37241]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:20.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:19 smithi071 ceph-mon[32789]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:21.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:20 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/4236524055' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "088f6419-63a0-4768-afae-dbb8a5f5a565"}]: dispatch 2024-03-04T08:54:21.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:20 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/4236524055' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "088f6419-63a0-4768-afae-dbb8a5f5a565"}]': finished 2024-03-04T08:54:21.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:20 smithi150 ceph-mon[36807]: osdmap e66: 10 total, 9 up, 10 in 2024-03-04T08:54:21.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:20 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T08:54:21.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:20 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3064961954' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:54:21.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:20 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/4236524055' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "088f6419-63a0-4768-afae-dbb8a5f5a565"}]: dispatch 2024-03-04T08:54:21.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:20 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/4236524055' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "088f6419-63a0-4768-afae-dbb8a5f5a565"}]': finished 2024-03-04T08:54:21.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:20 smithi158 ceph-mon[37241]: osdmap e66: 10 total, 9 up, 10 in 2024-03-04T08:54:21.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:20 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T08:54:21.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:20 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3064961954' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:54:21.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:20 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/4236524055' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "088f6419-63a0-4768-afae-dbb8a5f5a565"}]: dispatch 2024-03-04T08:54:21.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:20 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/4236524055' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "088f6419-63a0-4768-afae-dbb8a5f5a565"}]': finished 2024-03-04T08:54:21.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:20 smithi071 ceph-mon[32789]: osdmap e66: 10 total, 9 up, 10 in 2024-03-04T08:54:21.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:20 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T08:54:21.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:20 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3064961954' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:54:22.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:21 smithi150 ceph-mon[36807]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:22.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:21 smithi158 ceph-mon[37241]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:22.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:21 smithi071 ceph-mon[32789]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:23.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:23 smithi071 ceph-mon[32789]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:24.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:23 smithi150 ceph-mon[36807]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:24.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:23 smithi158 ceph-mon[37241]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:26.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:25 smithi150 ceph-mon[36807]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:26.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:25 smithi158 ceph-mon[37241]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:26.147 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:25 smithi071 ceph-mon[32789]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:28.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:27 smithi150 ceph-mon[36807]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:28.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:27 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-04T08:54:28.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:27 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:54:28.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:27 smithi158 ceph-mon[37241]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:28.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:27 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-04T08:54:28.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:27 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:54:28.074 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:27 smithi071 ceph-mon[32789]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:28.074 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:27 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-04T08:54:28.074 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:27 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:54:29.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:28 smithi150 ceph-mon[36807]: Deploying daemon osd.9 on smithi071 2024-03-04T08:54:29.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:28 smithi158 ceph-mon[37241]: Deploying daemon osd.9 on smithi071 2024-03-04T08:54:29.108 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:28 smithi071 ceph-mon[32789]: Deploying daemon osd.9 on smithi071 2024-03-04T08:54:30.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:29 smithi150 ceph-mon[36807]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:30.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:29 smithi158 ceph-mon[37241]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:30.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:29 smithi071 ceph-mon[32789]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:31.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:31 smithi071 ceph-mon[32789]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:32.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:31 smithi150 ceph-mon[36807]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:32.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:31 smithi158 ceph-mon[37241]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:34.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:33 smithi150 ceph-mon[36807]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:34.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:33 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:34.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:33 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:54:34.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:33 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:34.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:33 smithi158 ceph-mon[37241]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:34.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:33 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:34.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:33 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:54:34.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:33 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:34.104 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:33 smithi071 ceph-mon[32789]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:34.105 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:33 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:34.105 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:33 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:54:34.105 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:33 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:36.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:35 smithi150 ceph-mon[36807]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:36.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:35 smithi158 ceph-mon[37241]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:36.067 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:35 smithi071 ceph-mon[32789]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:36.842 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 9 on host 'smithi071' 2024-03-04T08:54:36.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:36.846+0000 7fb9067fc700 1 -- 172.21.15.71:0/2585649929 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb91006a4d0 con 0x7fb8fc0a1e90 2024-03-04T08:54:36.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:36.848+0000 7fb9174c1700 1 -- 172.21.15.71:0/2585649929 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb8fc0a1e90 msgr2=0x7fb8fc0a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:54:36.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:36.848+0000 7fb9174c1700 1 --2- 172.21.15.71:0/2585649929 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb8fc0a1e90 0x7fb8fc0a4350 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb910102050 tx=0x7fb90c008040 comp rx=0 tx=0).stop 2024-03-04T08:54:36.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:36.848+0000 7fb9174c1700 1 -- 172.21.15.71:0/2585649929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9101021f0 msgr2=0x7fb910198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:54:36.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:36.848+0000 7fb9174c1700 1 --2- 172.21.15.71:0/2585649929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9101021f0 0x7fb910198a20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb900000c00 tx=0x7fb900004580 comp rx=0 tx=0).stop 2024-03-04T08:54:36.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:36.848+0000 7fb9174c1700 1 -- 172.21.15.71:0/2585649929 shutdown_connections 2024-03-04T08:54:36.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:36.848+0000 7fb9174c1700 1 --2- 172.21.15.71:0/2585649929 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb8fc0a1e90 0x7fb8fc0a4350 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:54:36.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:36.848+0000 7fb9174c1700 1 --2- 172.21.15.71:0/2585649929 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb910102b90 0x7fb91019faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:54:36.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:36.848+0000 7fb9174c1700 1 --2- 172.21.15.71:0/2585649929 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb910100ff0 0x7fb9101984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:54:36.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:36.848+0000 7fb9174c1700 1 --2- 172.21.15.71:0/2585649929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9101021f0 0x7fb910198a20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:54:36.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:36.848+0000 7fb9174c1700 1 -- 172.21.15.71:0/2585649929 >> 172.21.15.71:0/2585649929 conn(0x7fb9100fc590 msgr2=0x7fb910069dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:54:36.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:36.848+0000 7fb9174c1700 1 -- 172.21.15.71:0/2585649929 shutdown_connections 2024-03-04T08:54:36.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:54:36.848+0000 7fb9174c1700 1 -- 172.21.15.71:0/2585649929 wait complete. 2024-03-04T08:54:37.899 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:37 smithi071 ceph-mon[32789]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:37.900 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:37 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:37.900 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:37 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:37.900 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:37 smithi071 ceph-mon[32789]: from='osd.9 [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-04T08:54:37.926 DEBUG:teuthology.orchestra.run.smithi071:osd.9> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.9.service 2024-03-04T08:54:37.928 INFO:tasks.cephadm:Deploying osd.10 on smithi150 with /dev/vg_nvme/lv_1... 2024-03-04T08:54:37.928 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-04T08:54:38.140 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:37 smithi158 ceph-mon[37241]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:38.140 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:37 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:38.140 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:37 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:38.141 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:37 smithi158 ceph-mon[37241]: from='osd.9 [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-04T08:54:38.177 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T08:54:38.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:37 smithi150 ceph-mon[36807]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:38.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:37 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:38.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:37 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:38.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:37 smithi150 ceph-mon[36807]: from='osd.9 [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-04T08:54:39.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:38 smithi150 ceph-mon[36807]: from='osd.9 [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-04T08:54:39.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:38 smithi150 ceph-mon[36807]: osdmap e67: 10 total, 9 up, 10 in 2024-03-04T08:54:39.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:38 smithi150 ceph-mon[36807]: from='osd.9 [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-04T08:54:39.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:38 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T08:54:39.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:38 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:39.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:38 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:39.111 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:38 smithi071 ceph-mon[32789]: from='osd.9 [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-04T08:54:39.112 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:38 smithi071 ceph-mon[32789]: osdmap e67: 10 total, 9 up, 10 in 2024-03-04T08:54:39.112 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:38 smithi071 ceph-mon[32789]: from='osd.9 [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-04T08:54:39.112 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:38 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T08:54:39.112 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:38 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:39.112 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:38 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:39.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:38 smithi158 ceph-mon[37241]: from='osd.9 [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-04T08:54:39.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:38 smithi158 ceph-mon[37241]: osdmap e67: 10 total, 9 up, 10 in 2024-03-04T08:54:39.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:38 smithi158 ceph-mon[37241]: from='osd.9 [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-03-04T08:54:39.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:38 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T08:54:39.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:38 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:39.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:38 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:39.862 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 08:54:39 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T08:54:39.553+0000 7f4fbe743700 -1 osd.9 0 waiting for initial osdmap 2024-03-04T08:54:39.862 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 08:54:39 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T08:54:39.565+0000 7f4fb652e700 -1 osd.9 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T08:54:40.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:39 smithi150 ceph-mon[36807]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:40.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:39 smithi150 ceph-mon[36807]: from='osd.9 [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-03-04T08:54:40.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:39 smithi150 ceph-mon[36807]: osdmap e68: 10 total, 9 up, 10 in 2024-03-04T08:54:40.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:39 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T08:54:40.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:39 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T08:54:40.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:39 smithi150 ceph-mon[36807]: from='osd.9 [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347]' entity='osd.9' 2024-03-04T08:54:40.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:39 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T08:54:40.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:39 smithi071 ceph-mon[32789]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:40.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:39 smithi071 ceph-mon[32789]: from='osd.9 [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-03-04T08:54:40.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:39 smithi071 ceph-mon[32789]: osdmap e68: 10 total, 9 up, 10 in 2024-03-04T08:54:40.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:39 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T08:54:40.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:39 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T08:54:40.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:39 smithi071 ceph-mon[32789]: from='osd.9 [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347]' entity='osd.9' 2024-03-04T08:54:40.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:39 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T08:54:40.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:39 smithi158 ceph-mon[37241]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:40.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:39 smithi158 ceph-mon[37241]: from='osd.9 [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-03-04T08:54:40.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:39 smithi158 ceph-mon[37241]: osdmap e68: 10 total, 9 up, 10 in 2024-03-04T08:54:40.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:39 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T08:54:40.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:39 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T08:54:40.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:39 smithi158 ceph-mon[37241]: from='osd.9 [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347]' entity='osd.9' 2024-03-04T08:54:40.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:39 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T08:54:40.377 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-04T08:54:40.377 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-04T08:54:40.377 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-03-04T08:54:40.377 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-03-04T08:54:40.377 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110891 s, 946 MB/s 2024-03-04T08:54:40.377 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-03-04T08:54:40.964 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:40 smithi150 ceph-mon[36807]: purged_snaps scrub starts 2024-03-04T08:54:40.964 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:40 smithi150 ceph-mon[36807]: purged_snaps scrub ok 2024-03-04T08:54:40.964 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:40 smithi150 ceph-mon[36807]: osd.9 [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347] boot 2024-03-04T08:54:40.965 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:40 smithi150 ceph-mon[36807]: osdmap e69: 10 total, 10 up, 10 in 2024-03-04T08:54:40.965 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:40 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T08:54:40.990 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi150:vg_nvme/lv_1 2024-03-04T08:54:41.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:40 smithi071 ceph-mon[32789]: purged_snaps scrub starts 2024-03-04T08:54:41.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:40 smithi071 ceph-mon[32789]: purged_snaps scrub ok 2024-03-04T08:54:41.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:40 smithi071 ceph-mon[32789]: osd.9 [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347] boot 2024-03-04T08:54:41.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:40 smithi071 ceph-mon[32789]: osdmap e69: 10 total, 10 up, 10 in 2024-03-04T08:54:41.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:40 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T08:54:41.227 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T08:54:41.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:40 smithi158 ceph-mon[37241]: purged_snaps scrub starts 2024-03-04T08:54:41.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:40 smithi158 ceph-mon[37241]: purged_snaps scrub ok 2024-03-04T08:54:41.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:40 smithi158 ceph-mon[37241]: osd.9 [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347] boot 2024-03-04T08:54:41.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:40 smithi158 ceph-mon[37241]: osdmap e69: 10 total, 10 up, 10 in 2024-03-04T08:54:41.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:40 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T08:54:42.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:41 smithi071 ceph-mon[32789]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:42.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:41 smithi150 ceph-mon[36807]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:42.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:41 smithi158 ceph-mon[37241]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T08:54:42.931 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.933+0000 7faa302a5700 1 -- 172.21.15.150:0/3204322372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa281032f0 msgr2=0x7faa28103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:54:42.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.933+0000 7faa302a5700 1 --2- 172.21.15.150:0/3204322372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa281032f0 0x7faa28103750 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7faa240099c0 tx=0x7faa24009cd0 comp rx=0 tx=0).stop 2024-03-04T08:54:42.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.933+0000 7faa302a5700 1 -- 172.21.15.150:0/3204322372 shutdown_connections 2024-03-04T08:54:42.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.933+0000 7faa302a5700 1 --2- 172.21.15.150:0/3204322372 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faa281020f0 0x7faa28102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:54:42.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.933+0000 7faa302a5700 1 --2- 172.21.15.150:0/3204322372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa28103c90 0x7faa2810a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:54:42.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.933+0000 7faa302a5700 1 --2- 172.21.15.150:0/3204322372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa281032f0 0x7faa28103750 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:54:42.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.933+0000 7faa302a5700 1 -- 172.21.15.150:0/3204322372 >> 172.21.15.150:0/3204322372 conn(0x7faa280fd6d0 msgr2=0x7faa280ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:54:42.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.933+0000 7faa302a5700 1 -- 172.21.15.150:0/3204322372 shutdown_connections 2024-03-04T08:54:42.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.933+0000 7faa302a5700 1 -- 172.21.15.150:0/3204322372 wait complete. 2024-03-04T08:54:42.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.934+0000 7faa302a5700 1 Processor -- start 2024-03-04T08:54:42.932 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.934+0000 7faa302a5700 1 -- start start 2024-03-04T08:54:42.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.934+0000 7faa302a5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa281020f0 0x7faa28075e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:54:42.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.934+0000 7faa302a5700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faa281032f0 0x7faa280744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:54:42.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.934+0000 7faa302a5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa28103c90 0x7faa280749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:54:42.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.934+0000 7faa302a5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2810dcd0 con 0x7faa281020f0 2024-03-04T08:54:42.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.934+0000 7faa302a5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2810db50 con 0x7faa28103c90 2024-03-04T08:54:42.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.934+0000 7faa302a5700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2810de50 con 0x7faa281032f0 2024-03-04T08:54:42.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.934+0000 7faa2e842700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa28103c90 0x7faa280749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:54:42.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.934+0000 7faa2e842700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa28103c90 0x7faa280749f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:49500/0 (socket says 172.21.15.150:49500) 2024-03-04T08:54:42.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.934+0000 7faa2e842700 1 -- 172.21.15.150:0/96630654 learned_addr learned my addr 172.21.15.150:0/96630654 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-03-04T08:54:42.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.934+0000 7faa2e041700 1 --2- 172.21.15.150:0/96630654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa281020f0 0x7faa28075e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:54:42.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.934+0000 7faa2d840700 1 --2- 172.21.15.150:0/96630654 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faa281032f0 0x7faa280744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:54:42.933 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.934+0000 7faa2e842700 1 -- 172.21.15.150:0/96630654 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faa281032f0 msgr2=0x7faa280744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:54:42.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.934+0000 7faa2e842700 1 --2- 172.21.15.150:0/96630654 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faa281032f0 0x7faa280744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:54:42.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.934+0000 7faa2e842700 1 -- 172.21.15.150:0/96630654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa281020f0 msgr2=0x7faa28075e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:54:42.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.934+0000 7faa2e842700 1 --2- 172.21.15.150:0/96630654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa281020f0 0x7faa28075e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:54:42.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.934+0000 7faa2e842700 1 -- 172.21.15.150:0/96630654 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa280752a0 con 0x7faa28103c90 2024-03-04T08:54:42.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.935+0000 7faa2e842700 1 --2- 172.21.15.150:0/96630654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa28103c90 0x7faa280749f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faa20007ee0 tx=0x7faa2000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:54:42.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.935+0000 7faa1f7fe700 1 -- 172.21.15.150:0/96630654 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20004bb0 con 0x7faa28103c90 2024-03-04T08:54:42.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.935+0000 7faa1f7fe700 1 -- 172.21.15.150:0/96630654 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa20004d50 con 0x7faa28103c90 2024-03-04T08:54:42.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.935+0000 7faa302a5700 1 -- 172.21.15.150:0/96630654 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa281a24a0 con 0x7faa28103c90 2024-03-04T08:54:42.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.935+0000 7faa1f7fe700 1 -- 172.21.15.150:0/96630654 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20005960 con 0x7faa28103c90 2024-03-04T08:54:42.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.935+0000 7faa302a5700 1 -- 172.21.15.150:0/96630654 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa281a2960 con 0x7faa28103c90 2024-03-04T08:54:42.934 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.936+0000 7faa302a5700 1 -- 172.21.15.150:0/96630654 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa28068d80 con 0x7faa28103c90 2024-03-04T08:54:42.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.937+0000 7faa1f7fe700 1 -- 172.21.15.150:0/96630654 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7faa2001a020 con 0x7faa28103c90 2024-03-04T08:54:42.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.937+0000 7faa1f7fe700 1 --2- 172.21.15.150:0/96630654 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7faa140a1e10 0x7faa140a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:54:42.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.937+0000 7faa1f7fe700 1 -- 172.21.15.150:0/96630654 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6261+0+0 (secure 0 0 0) 0x7faa200ccd00 con 0x7faa28103c90 2024-03-04T08:54:42.935 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.938+0000 7faa2e041700 1 --2- 172.21.15.150:0/96630654 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7faa140a1e10 0x7faa140a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:54:42.936 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.938+0000 7faa2e041700 1 --2- 172.21.15.150:0/96630654 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7faa140a1e10 0x7faa140a42d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faa28103150 tx=0x7faa1800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:54:42.938 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:42.942+0000 7faa1f7fe700 1 -- 172.21.15.150:0/96630654 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7faa20010070 con 0x7faa28103c90 2024-03-04T08:54:43.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:42 smithi150 ceph-mon[36807]: osdmap e70: 10 total, 10 up, 10 in 2024-03-04T08:54:43.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:42 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:43.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:42 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:43.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:42 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:54:43.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:42 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:54:43.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:42 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:54:43.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:42 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:54:43.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:42 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:43.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:42 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:54:43.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:42 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:54:43.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:42 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:43.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:42 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:54:43.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:42 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:54:43.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:42 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:54:43.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:42 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:43.141 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:54:43.145+0000 7faa302a5700 1 -- 172.21.15.150:0/96630654 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7faa28062500 con 0x7faa140a1e10 2024-03-04T08:54:43.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:42 smithi071 ceph-mon[32789]: osdmap e70: 10 total, 10 up, 10 in 2024-03-04T08:54:43.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:42 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:43.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:42 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:43.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:42 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:54:43.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:42 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:54:43.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:42 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:54:43.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:42 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:54:43.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:42 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:43.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:42 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:54:43.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:42 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:54:43.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:42 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:43.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:42 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:54:43.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:42 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:54:43.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:42 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:54:43.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:42 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:43.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:42 smithi158 ceph-mon[37241]: osdmap e70: 10 total, 10 up, 10 in 2024-03-04T08:54:43.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:42 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:43.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:42 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:43.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:42 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:54:43.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:42 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:54:43.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:42 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:54:43.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:42 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:54:43.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:42 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:43.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:42 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:54:43.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:42 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:54:43.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:42 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:43.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:42 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:54:43.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:42 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:54:43.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:42 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:54:43.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:42 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:44.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:43 smithi150 ceph-mon[36807]: Detected new or changed devices on smithi071 2024-03-04T08:54:44.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:43 smithi150 ceph-mon[36807]: Adjusting osd_memory_target on smithi071 to 4272M 2024-03-04T08:54:44.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:43 smithi150 ceph-mon[36807]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:44.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:43 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:54:44.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:43 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:54:44.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:43 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:54:44.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:43 smithi071 ceph-mon[32789]: Detected new or changed devices on smithi071 2024-03-04T08:54:44.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:43 smithi071 ceph-mon[32789]: Adjusting osd_memory_target on smithi071 to 4272M 2024-03-04T08:54:44.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:43 smithi071 ceph-mon[32789]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:44.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:43 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:54:44.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:43 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:54:44.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:43 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:54:44.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:43 smithi158 ceph-mon[37241]: Detected new or changed devices on smithi071 2024-03-04T08:54:44.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:43 smithi158 ceph-mon[37241]: Adjusting osd_memory_target on smithi071 to 4272M 2024-03-04T08:54:44.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:43 smithi158 ceph-mon[37241]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:44.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:43 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:54:44.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:43 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:54:44.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:43 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:54:45.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:44 smithi071 ceph-mon[32789]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:54:45.187 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:44 smithi150 ceph-mon[36807]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:54:45.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:44 smithi158 ceph-mon[37241]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:54:46.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:45 smithi071 ceph-mon[32789]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:46.270 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:45 smithi150 ceph-mon[36807]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:46.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:45 smithi158 ceph-mon[37241]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:47.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:46 smithi071 ceph-mon[32789]: from='client.? 172.21.15.150:0/2538049213' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18c85e24-068d-4f1e-a3b8-f7915b363234"}]: dispatch 2024-03-04T08:54:47.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:46 smithi071 ceph-mon[32789]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18c85e24-068d-4f1e-a3b8-f7915b363234"}]: dispatch 2024-03-04T08:54:47.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:46 smithi071 ceph-mon[32789]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18c85e24-068d-4f1e-a3b8-f7915b363234"}]': finished 2024-03-04T08:54:47.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:46 smithi071 ceph-mon[32789]: osdmap e71: 11 total, 10 up, 11 in 2024-03-04T08:54:47.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:46 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T08:54:47.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:46 smithi071 ceph-mon[32789]: from='client.? 172.21.15.150:0/968023404' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:54:47.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:46 smithi071 ceph-mon[32789]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:47.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:46 smithi150 ceph-mon[36807]: from='client.? 172.21.15.150:0/2538049213' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18c85e24-068d-4f1e-a3b8-f7915b363234"}]: dispatch 2024-03-04T08:54:47.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:46 smithi150 ceph-mon[36807]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18c85e24-068d-4f1e-a3b8-f7915b363234"}]: dispatch 2024-03-04T08:54:47.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:46 smithi150 ceph-mon[36807]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18c85e24-068d-4f1e-a3b8-f7915b363234"}]': finished 2024-03-04T08:54:47.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:46 smithi150 ceph-mon[36807]: osdmap e71: 11 total, 10 up, 11 in 2024-03-04T08:54:47.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:46 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T08:54:47.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:46 smithi150 ceph-mon[36807]: from='client.? 172.21.15.150:0/968023404' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:54:47.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:46 smithi150 ceph-mon[36807]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:47.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:46 smithi158 ceph-mon[37241]: from='client.? 172.21.15.150:0/2538049213' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18c85e24-068d-4f1e-a3b8-f7915b363234"}]: dispatch 2024-03-04T08:54:47.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:46 smithi158 ceph-mon[37241]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18c85e24-068d-4f1e-a3b8-f7915b363234"}]: dispatch 2024-03-04T08:54:47.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:46 smithi158 ceph-mon[37241]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18c85e24-068d-4f1e-a3b8-f7915b363234"}]': finished 2024-03-04T08:54:47.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:46 smithi158 ceph-mon[37241]: osdmap e71: 11 total, 10 up, 11 in 2024-03-04T08:54:47.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:46 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T08:54:47.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:46 smithi158 ceph-mon[37241]: from='client.? 172.21.15.150:0/968023404' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:54:47.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:46 smithi158 ceph-mon[37241]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:49.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:49 smithi071 ceph-mon[32789]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:49.942 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:49 smithi150 ceph-mon[36807]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:50.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:49 smithi158 ceph-mon[37241]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:51.981 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:51 smithi150 ceph-mon[36807]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:52.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:51 smithi158 ceph-mon[37241]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:52.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:51 smithi071 ceph-mon[32789]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:53.990 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:53 smithi150 ceph-mon[36807]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:53.990 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:53 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-04T08:54:53.991 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:53 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:54:54.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:53 smithi158 ceph-mon[37241]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:54.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:53 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-04T08:54:54.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:53 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:54:54.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:53 smithi071 ceph-mon[32789]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:54.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:53 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-04T08:54:54.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:53 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:54:55.001 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:54 smithi150 ceph-mon[36807]: Deploying daemon osd.10 on smithi150 2024-03-04T08:54:55.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:54 smithi158 ceph-mon[37241]: Deploying daemon osd.10 on smithi150 2024-03-04T08:54:55.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:54 smithi071 ceph-mon[32789]: Deploying daemon osd.10 on smithi150 2024-03-04T08:54:56.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:55 smithi150 ceph-mon[36807]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:56.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:55 smithi158 ceph-mon[37241]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:56.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:55 smithi071 ceph-mon[32789]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:57.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:57 smithi150 ceph-mon[36807]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:58.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:57 smithi158 ceph-mon[37241]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:58.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:57 smithi071 ceph-mon[32789]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:59.826 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:59 smithi150 ceph-mon[36807]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:54:59.827 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:59 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:59.827 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:59 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:54:59.827 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:54:59 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:55:00.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:59 smithi158 ceph-mon[37241]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:55:00.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:59 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:00.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:59 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:00.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:54:59 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:55:00.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:59 smithi071 ceph-mon[32789]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:55:00.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:59 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:00.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:59 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:00.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:54:59 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:55:01.890 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:01 smithi150 ceph-mon[36807]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:55:02.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:01 smithi158 ceph-mon[37241]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:55:02.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:01 smithi071 ceph-mon[32789]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:55:03.107 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:55:03.111+0000 7faa1f7fe700 1 -- 172.21.15.150:0/96630654 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7faa28062500 con 0x7faa140a1e10 2024-03-04T08:55:03.108 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 10 on host 'smithi150' 2024-03-04T08:55:03.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:55:03.113+0000 7faa302a5700 1 -- 172.21.15.150:0/96630654 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7faa140a1e10 msgr2=0x7faa140a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:03.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:55:03.113+0000 7faa302a5700 1 --2- 172.21.15.150:0/96630654 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7faa140a1e10 0x7faa140a42d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faa28103150 tx=0x7faa1800b410 comp rx=0 tx=0).stop 2024-03-04T08:55:03.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:55:03.114+0000 7faa302a5700 1 -- 172.21.15.150:0/96630654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa28103c90 msgr2=0x7faa280749f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:03.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:55:03.114+0000 7faa302a5700 1 --2- 172.21.15.150:0/96630654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa28103c90 0x7faa280749f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faa20007ee0 tx=0x7faa2000d3b0 comp rx=0 tx=0).stop 2024-03-04T08:55:03.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:55:03.114+0000 7faa302a5700 1 -- 172.21.15.150:0/96630654 shutdown_connections 2024-03-04T08:55:03.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:55:03.114+0000 7faa302a5700 1 --2- 172.21.15.150:0/96630654 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7faa140a1e10 0x7faa140a42d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:03.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:55:03.114+0000 7faa302a5700 1 --2- 172.21.15.150:0/96630654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa28103c90 0x7faa280749f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:03.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:55:03.114+0000 7faa302a5700 1 --2- 172.21.15.150:0/96630654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa281020f0 0x7faa28075e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:03.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:55:03.114+0000 7faa302a5700 1 --2- 172.21.15.150:0/96630654 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faa281032f0 0x7faa280744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:03.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:55:03.114+0000 7faa302a5700 1 -- 172.21.15.150:0/96630654 >> 172.21.15.150:0/96630654 conn(0x7faa280fd6d0 msgr2=0x7faa28106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:55:03.110 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:55:03.114+0000 7faa302a5700 1 -- 172.21.15.150:0/96630654 shutdown_connections 2024-03-04T08:55:03.111 INFO:teuthology.orchestra.run.smithi150.stderr:2024-03-04T08:55:03.114+0000 7faa302a5700 1 -- 172.21.15.150:0/96630654 wait complete. 2024-03-04T08:55:03.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:03 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:03.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:03 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:03.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:03 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:03.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:03 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:55:03.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:03 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:03.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:03 smithi150 ceph-mon[36807]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:55:03.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:03 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:55:03.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:03 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:03.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:03 smithi150 ceph-mon[36807]: from='osd.10 [v2:172.21.15.150:6824/3295420996,v1:172.21.15.150:6825/3295420996]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-04T08:55:03.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:03 smithi150 ceph-mon[36807]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-04T08:55:03.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:03 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:03.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:03 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:03.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:03 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:03.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:03 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:03.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:03 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:55:03.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:03 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:03.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:03 smithi158 ceph-mon[37241]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:55:03.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:03 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:55:03.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:03 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:03.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:03 smithi158 ceph-mon[37241]: from='osd.10 [v2:172.21.15.150:6824/3295420996,v1:172.21.15.150:6825/3295420996]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-04T08:55:03.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:03 smithi158 ceph-mon[37241]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-04T08:55:03.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:03 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:03.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:03 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:03.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:03 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:03.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:03 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:03.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:03 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:55:03.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:03 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:03.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:03 smithi071 ceph-mon[32789]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:55:03.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:03 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:55:03.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:03 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:03.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:03 smithi071 ceph-mon[32789]: from='osd.10 [v2:172.21.15.150:6824/3295420996,v1:172.21.15.150:6825/3295420996]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-04T08:55:03.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:03 smithi071 ceph-mon[32789]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-04T08:55:03.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:03 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:03.816 DEBUG:teuthology.orchestra.run.smithi150:osd.10> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.10.service 2024-03-04T08:55:03.818 INFO:tasks.cephadm:Deploying osd.11 on smithi158 with /dev/vg_nvme/lv_1... 2024-03-04T08:55:03.818 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-04T08:55:04.069 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T08:55:05.235 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:05 smithi158 ceph-mon[37241]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-04T08:55:05.235 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:05 smithi158 ceph-mon[37241]: from='osd.10 [v2:172.21.15.150:6824/3295420996,v1:172.21.15.150:6825/3295420996]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-04T08:55:05.235 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:05 smithi158 ceph-mon[37241]: osdmap e72: 11 total, 10 up, 11 in 2024-03-04T08:55:05.235 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:05 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T08:55:05.235 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:05 smithi158 ceph-mon[37241]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-04T08:55:05.236 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:05 smithi158 ceph-mon[37241]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:55:05.276 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:05 smithi150 ceph-mon[36807]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-04T08:55:05.276 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:05 smithi150 ceph-mon[36807]: from='osd.10 [v2:172.21.15.150:6824/3295420996,v1:172.21.15.150:6825/3295420996]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-04T08:55:05.276 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:05 smithi150 ceph-mon[36807]: osdmap e72: 11 total, 10 up, 11 in 2024-03-04T08:55:05.276 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:05 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T08:55:05.277 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:05 smithi150 ceph-mon[36807]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-04T08:55:05.277 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:05 smithi150 ceph-mon[36807]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:55:05.277 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 08:55:05 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T08:55:05.065+0000 7f2ed09ef700 -1 osd.10 0 waiting for initial osdmap 2024-03-04T08:55:05.277 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 08:55:05 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T08:55:05.078+0000 7f2ecc7e2700 -1 osd.10 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T08:55:05.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:05 smithi071 ceph-mon[32789]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-04T08:55:05.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:05 smithi071 ceph-mon[32789]: from='osd.10 [v2:172.21.15.150:6824/3295420996,v1:172.21.15.150:6825/3295420996]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-04T08:55:05.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:05 smithi071 ceph-mon[32789]: osdmap e72: 11 total, 10 up, 11 in 2024-03-04T08:55:05.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:05 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T08:55:05.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:05 smithi071 ceph-mon[32789]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-03-04T08:55:05.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:05 smithi071 ceph-mon[32789]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-03-04T08:55:06.103 INFO:teuthology.orchestra.run.smithi158.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-04T08:55:06.103 INFO:teuthology.orchestra.run.smithi158.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-04T08:55:06.103 INFO:teuthology.orchestra.run.smithi158.stderr: stderr: 10+0 records in 2024-03-04T08:55:06.103 INFO:teuthology.orchestra.run.smithi158.stderr:10+0 records out 2024-03-04T08:55:06.103 INFO:teuthology.orchestra.run.smithi158.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0086146 s, 1.2 GB/s 2024-03-04T08:55:06.103 INFO:teuthology.orchestra.run.smithi158.stderr:--> Zapping successful for: 2024-03-04T08:55:06.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:06 smithi071 ceph-mon[32789]: purged_snaps scrub starts 2024-03-04T08:55:06.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:06 smithi071 ceph-mon[32789]: purged_snaps scrub ok 2024-03-04T08:55:06.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:06 smithi071 ceph-mon[32789]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-03-04T08:55:06.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:06 smithi071 ceph-mon[32789]: osdmap e73: 11 total, 10 up, 11 in 2024-03-04T08:55:06.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:06 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T08:55:06.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:06 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T08:55:06.455 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:06 smithi158 ceph-mon[37241]: purged_snaps scrub starts 2024-03-04T08:55:06.455 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:06 smithi158 ceph-mon[37241]: purged_snaps scrub ok 2024-03-04T08:55:06.455 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:06 smithi158 ceph-mon[37241]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-03-04T08:55:06.456 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:06 smithi158 ceph-mon[37241]: osdmap e73: 11 total, 10 up, 11 in 2024-03-04T08:55:06.456 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:06 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T08:55:06.456 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:06 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T08:55:06.499 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:06 smithi150 ceph-mon[36807]: purged_snaps scrub starts 2024-03-04T08:55:06.499 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:06 smithi150 ceph-mon[36807]: purged_snaps scrub ok 2024-03-04T08:55:06.499 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:06 smithi150 ceph-mon[36807]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-03-04T08:55:06.499 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:06 smithi150 ceph-mon[36807]: osdmap e73: 11 total, 10 up, 11 in 2024-03-04T08:55:06.499 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:06 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T08:55:06.499 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:06 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T08:55:06.730 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi158:vg_nvme/lv_1 2024-03-04T08:55:06.969 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T08:55:07.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:07 smithi150 ceph-mon[36807]: osd.10 [v2:172.21.15.150:6824/3295420996,v1:172.21.15.150:6825/3295420996] boot 2024-03-04T08:55:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:07 smithi150 ceph-mon[36807]: osdmap e74: 11 total, 11 up, 11 in 2024-03-04T08:55:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:07 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T08:55:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:07 smithi150 ceph-mon[36807]: pgmap v266: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-04T08:55:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:07 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:07 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:07 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:55:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:07 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:55:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:07 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:55:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:07 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:55:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:07 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:07.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:07 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:07.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:07 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:55:07.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:07 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:07.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:07 smithi150 ceph-mon[36807]: osdmap e75: 11 total, 11 up, 11 in 2024-03-04T08:55:07.330 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:07 smithi158 ceph-mon[37241]: osd.10 [v2:172.21.15.150:6824/3295420996,v1:172.21.15.150:6825/3295420996] boot 2024-03-04T08:55:07.330 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:07 smithi158 ceph-mon[37241]: osdmap e74: 11 total, 11 up, 11 in 2024-03-04T08:55:07.330 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:07 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T08:55:07.330 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:07 smithi158 ceph-mon[37241]: pgmap v266: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-04T08:55:07.330 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:07 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:07.330 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:07 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:07.330 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:07 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:55:07.330 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:07 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:55:07.331 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:07 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:55:07.331 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:07 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:55:07.331 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:07 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:07.331 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:07 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:07.331 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:07 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:55:07.331 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:07 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:07.331 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:07 smithi158 ceph-mon[37241]: osdmap e75: 11 total, 11 up, 11 in 2024-03-04T08:55:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:07 smithi071 ceph-mon[32789]: osd.10 [v2:172.21.15.150:6824/3295420996,v1:172.21.15.150:6825/3295420996] boot 2024-03-04T08:55:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:07 smithi071 ceph-mon[32789]: osdmap e74: 11 total, 11 up, 11 in 2024-03-04T08:55:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:07 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T08:55:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:07 smithi071 ceph-mon[32789]: pgmap v266: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-04T08:55:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:07 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:07 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:07 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:55:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:07 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:55:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:07 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:55:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:07 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:55:07.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:07 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:07.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:07 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:07.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:07 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:55:07.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:07 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:07.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:07 smithi071 ceph-mon[32789]: osdmap e75: 11 total, 11 up, 11 in 2024-03-04T08:55:08.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:08 smithi158 ceph-mon[37241]: Detected new or changed devices on smithi150 2024-03-04T08:55:08.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:08 smithi158 ceph-mon[37241]: Adjusting osd_memory_target on smithi150 to 4270M 2024-03-04T08:55:08.322 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:08 smithi071 ceph-mon[32789]: Detected new or changed devices on smithi150 2024-03-04T08:55:08.322 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:08 smithi071 ceph-mon[32789]: Adjusting osd_memory_target on smithi150 to 4270M 2024-03-04T08:55:08.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:08 smithi150 ceph-mon[36807]: Detected new or changed devices on smithi150 2024-03-04T08:55:08.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:08 smithi150 ceph-mon[36807]: Adjusting osd_memory_target on smithi150 to 4270M 2024-03-04T08:55:08.562 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.562+0000 7f742701d700 1 -- 172.21.15.158:0/4195845352 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7420107e60 msgr2=0x7f742010a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:08.562 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.562+0000 7f742701d700 1 --2- 172.21.15.158:0/4195845352 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7420107e60 0x7f742010a250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7418009990 tx=0x7f7418009ca0 comp rx=0 tx=0).stop 2024-03-04T08:55:08.562 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.562+0000 7f742701d700 1 -- 172.21.15.158:0/4195845352 shutdown_connections 2024-03-04T08:55:08.562 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.562+0000 7f742701d700 1 --2- 172.21.15.158:0/4195845352 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7420107e60 0x7f742010a250 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:08.562 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.562+0000 7f742701d700 1 --2- 172.21.15.158:0/4195845352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7420102c00 0x7f7420104ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:08.562 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.562+0000 7f742701d700 1 --2- 172.21.15.158:0/4195845352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7420105530 0x7f7420107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:08.563 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.562+0000 7f742701d700 1 -- 172.21.15.158:0/4195845352 >> 172.21.15.158:0/4195845352 conn(0x7f74200fc590 msgr2=0x7f74200fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:55:08.563 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.562+0000 7f742701d700 1 -- 172.21.15.158:0/4195845352 shutdown_connections 2024-03-04T08:55:08.563 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.562+0000 7f742701d700 1 -- 172.21.15.158:0/4195845352 wait complete. 2024-03-04T08:55:08.563 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.563+0000 7f742701d700 1 Processor -- start 2024-03-04T08:55:08.563 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.563+0000 7f742701d700 1 -- start start 2024-03-04T08:55:08.563 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.563+0000 7f742701d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7420102c00 0x7f74201982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:08.563 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.563+0000 7f742701d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7420105530 0x7f74201987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:08.563 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.563+0000 7f742701d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7420107e60 0x7f742019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:08.564 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.563+0000 7f742701d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742010dcc0 con 0x7f7420107e60 2024-03-04T08:55:08.564 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.563+0000 7f742701d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742010db40 con 0x7f7420105530 2024-03-04T08:55:08.564 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.563+0000 7f742701d700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742010de40 con 0x7f7420102c00 2024-03-04T08:55:08.564 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.563+0000 7f7424db9700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7420102c00 0x7f74201982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:08.564 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.563+0000 7f7424db9700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7420102c00 0x7f74201982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.158:41144/0 (socket says 172.21.15.158:41144) 2024-03-04T08:55:08.564 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.563+0000 7f7424db9700 1 -- 172.21.15.158:0/3890545964 learned_addr learned my addr 172.21.15.158:0/3890545964 (peer_addr_for_me v2:172.21.15.158:0/0) 2024-03-04T08:55:08.564 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.563+0000 7f7417fff700 1 --2- 172.21.15.158:0/3890545964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7420105530 0x7f74201987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:08.564 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.563+0000 7f74255ba700 1 --2- 172.21.15.158:0/3890545964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7420107e60 0x7f742019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:08.564 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.563+0000 7f7424db9700 1 -- 172.21.15.158:0/3890545964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7420105530 msgr2=0x7f74201987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:08.565 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.563+0000 7f7424db9700 1 --2- 172.21.15.158:0/3890545964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7420105530 0x7f74201987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:08.565 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.563+0000 7f7424db9700 1 -- 172.21.15.158:0/3890545964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7420107e60 msgr2=0x7f742019f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:08.565 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.563+0000 7f7424db9700 1 --2- 172.21.15.158:0/3890545964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7420107e60 0x7f742019f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:08.565 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.563+0000 7f7424db9700 1 -- 172.21.15.158:0/3890545964 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742019ff10 con 0x7f7420102c00 2024-03-04T08:55:08.565 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.563+0000 7f74255ba700 1 --2- 172.21.15.158:0/3890545964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7420107e60 0x7f742019f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:55:08.565 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.563+0000 7f7424db9700 1 --2- 172.21.15.158:0/3890545964 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7420102c00 0x7f74201982b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f741c007f40 tx=0x7f741c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:55:08.565 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.564+0000 7f7415ffb700 1 -- 172.21.15.158:0/3890545964 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c004bb0 con 0x7f7420102c00 2024-03-04T08:55:08.565 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.564+0000 7f742701d700 1 -- 172.21.15.158:0/3890545964 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74201a0200 con 0x7f7420102c00 2024-03-04T08:55:08.565 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.564+0000 7f7415ffb700 1 -- 172.21.15.158:0/3890545964 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f741c004d50 con 0x7f7420102c00 2024-03-04T08:55:08.565 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.564+0000 7f7415ffb700 1 -- 172.21.15.158:0/3890545964 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c0133f0 con 0x7f7420102c00 2024-03-04T08:55:08.565 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.564+0000 7f742701d700 1 -- 172.21.15.158:0/3890545964 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74201a0740 con 0x7f7420102c00 2024-03-04T08:55:08.566 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.565+0000 7f742701d700 1 -- 172.21.15.158:0/3890545964 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7420068d80 con 0x7f7420102c00 2024-03-04T08:55:08.567 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.566+0000 7f7415ffb700 1 -- 172.21.15.158:0/3890545964 <== mon.1 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f741c01a020 con 0x7f7420102c00 2024-03-04T08:55:08.567 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.566+0000 7f7415ffb700 1 --2- 172.21.15.158:0/3890545964 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f74080a1e10 0x7f74080a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:08.567 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.566+0000 7f7415ffb700 1 -- 172.21.15.158:0/3890545964 <== mon.1 v2:172.21.15.158:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f741c0ccf50 con 0x7f7420102c00 2024-03-04T08:55:08.567 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.567+0000 7f7417fff700 1 --2- 172.21.15.158:0/3890545964 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f74080a1e10 0x7f74080a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:08.567 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.567+0000 7f7417fff700 1 --2- 172.21.15.158:0/3890545964 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f74080a1e10 0x7f74080a42d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7410009c60 tx=0x7f7410009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:55:08.568 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.571+0000 7f7415ffb700 1 -- 172.21.15.158:0/3890545964 <== mon.1 v2:172.21.15.158:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f741c010070 con 0x7f7420102c00 2024-03-04T08:55:08.771 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:08.774+0000 7f742701d700 1 -- 172.21.15.158:0/3890545964 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7420062500 con 0x7f74080a1e10 2024-03-04T08:55:09.348 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:09 smithi071 ceph-mon[32789]: osdmap e76: 11 total, 11 up, 11 in 2024-03-04T08:55:09.348 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:09 smithi071 ceph-mon[32789]: pgmap v269: 1 pgs: 1 peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-03-04T08:55:09.348 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:09 smithi071 ceph-mon[32789]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:55:09.348 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:09 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:55:09.348 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:09 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:55:09.348 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:09 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:09.375 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:09 smithi158 ceph-mon[37241]: osdmap e76: 11 total, 11 up, 11 in 2024-03-04T08:55:09.375 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:09 smithi158 ceph-mon[37241]: pgmap v269: 1 pgs: 1 peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-03-04T08:55:09.375 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:09 smithi158 ceph-mon[37241]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:55:09.375 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:09 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:55:09.375 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:09 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:55:09.376 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:09 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:09.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:09 smithi150 ceph-mon[36807]: osdmap e76: 11 total, 11 up, 11 in 2024-03-04T08:55:09.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:09 smithi150 ceph-mon[36807]: pgmap v269: 1 pgs: 1 peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-03-04T08:55:09.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:09 smithi150 ceph-mon[36807]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi158:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:55:09.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:09 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T08:55:09.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:09 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T08:55:09.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:09 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:10.614 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:10 smithi158 ceph-mon[37241]: osdmap e77: 11 total, 11 up, 11 in 2024-03-04T08:55:10.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:10 smithi071 ceph-mon[32789]: osdmap e77: 11 total, 11 up, 11 in 2024-03-04T08:55:10.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:10 smithi150 ceph-mon[36807]: osdmap e77: 11 total, 11 up, 11 in 2024-03-04T08:55:11.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:11 smithi071 ceph-mon[32789]: osdmap e78: 11 total, 11 up, 11 in 2024-03-04T08:55:11.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:11 smithi071 ceph-mon[32789]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-04T08:55:11.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:11 smithi150 ceph-mon[36807]: osdmap e78: 11 total, 11 up, 11 in 2024-03-04T08:55:11.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:11 smithi150 ceph-mon[36807]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-04T08:55:11.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:11 smithi158 ceph-mon[37241]: osdmap e78: 11 total, 11 up, 11 in 2024-03-04T08:55:11.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:11 smithi158 ceph-mon[37241]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-04T08:55:12.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:12 smithi071 ceph-mon[32789]: from='client.? 172.21.15.158:0/1587041275' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f6b3ddf-c482-4740-b1fd-de836e047c2d"}]: dispatch 2024-03-04T08:55:12.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:12 smithi071 ceph-mon[32789]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f6b3ddf-c482-4740-b1fd-de836e047c2d"}]: dispatch 2024-03-04T08:55:12.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:12 smithi071 ceph-mon[32789]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f6b3ddf-c482-4740-b1fd-de836e047c2d"}]': finished 2024-03-04T08:55:12.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:12 smithi071 ceph-mon[32789]: osdmap e79: 12 total, 11 up, 12 in 2024-03-04T08:55:12.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:12 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T08:55:12.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:12 smithi071 ceph-mon[32789]: from='client.? 172.21.15.158:0/2491455450' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:55:12.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:12 smithi150 ceph-mon[36807]: from='client.? 172.21.15.158:0/1587041275' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f6b3ddf-c482-4740-b1fd-de836e047c2d"}]: dispatch 2024-03-04T08:55:12.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:12 smithi150 ceph-mon[36807]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f6b3ddf-c482-4740-b1fd-de836e047c2d"}]: dispatch 2024-03-04T08:55:12.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:12 smithi150 ceph-mon[36807]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f6b3ddf-c482-4740-b1fd-de836e047c2d"}]': finished 2024-03-04T08:55:12.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:12 smithi150 ceph-mon[36807]: osdmap e79: 12 total, 11 up, 12 in 2024-03-04T08:55:12.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:12 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T08:55:12.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:12 smithi150 ceph-mon[36807]: from='client.? 172.21.15.158:0/2491455450' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:55:12.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:12 smithi158 ceph-mon[37241]: from='client.? 172.21.15.158:0/1587041275' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f6b3ddf-c482-4740-b1fd-de836e047c2d"}]: dispatch 2024-03-04T08:55:12.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:12 smithi158 ceph-mon[37241]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f6b3ddf-c482-4740-b1fd-de836e047c2d"}]: dispatch 2024-03-04T08:55:12.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:12 smithi158 ceph-mon[37241]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f6b3ddf-c482-4740-b1fd-de836e047c2d"}]': finished 2024-03-04T08:55:12.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:12 smithi158 ceph-mon[37241]: osdmap e79: 12 total, 11 up, 12 in 2024-03-04T08:55:12.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:12 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T08:55:12.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:12 smithi158 ceph-mon[37241]: from='client.? 172.21.15.158:0/2491455450' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T08:55:13.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:13 smithi071 ceph-mon[32789]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-03-04T08:55:13.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:13 smithi150 ceph-mon[36807]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-03-04T08:55:13.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:13 smithi158 ceph-mon[37241]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-03-04T08:55:15.907 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:15 smithi158 ceph-mon[37241]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-04T08:55:15.975 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:15 smithi150 ceph-mon[36807]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-04T08:55:16.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:15 smithi071 ceph-mon[32789]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-04T08:55:17.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:17 smithi158 ceph-mon[37241]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-04T08:55:18.034 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:17 smithi150 ceph-mon[36807]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-04T08:55:18.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:17 smithi071 ceph-mon[32789]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-04T08:55:20.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:19 smithi150 ceph-mon[36807]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-04T08:55:20.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:19 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-04T08:55:20.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:19 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:20.034 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:19 smithi071 ceph-mon[32789]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-04T08:55:20.034 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:19 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-04T08:55:20.034 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:19 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:20.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:19 smithi158 ceph-mon[37241]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-04T08:55:20.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:19 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-04T08:55:20.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:19 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:20.902 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:20 smithi158 ceph-mon[37241]: Deploying daemon osd.11 on smithi158 2024-03-04T08:55:20.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:20 smithi071 ceph-mon[32789]: Deploying daemon osd.11 on smithi158 2024-03-04T08:55:21.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:20 smithi150 ceph-mon[36807]: Deploying daemon osd.11 on smithi158 2024-03-04T08:55:21.790 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:21 smithi158 ceph-mon[37241]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-03-04T08:55:22.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:21 smithi150 ceph-mon[36807]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-03-04T08:55:22.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:21 smithi071 ceph-mon[32789]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-03-04T08:55:23.884 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:23 smithi158 ceph-mon[37241]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-03-04T08:55:24.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:23 smithi150 ceph-mon[36807]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-03-04T08:55:24.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:23 smithi071 ceph-mon[32789]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-03-04T08:55:25.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:25 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:25.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:25 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:55:25.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:25 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:25.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:25 smithi150 ceph-mon[36807]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-03-04T08:55:25.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:25 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:25.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:25 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:55:25.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:25 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:25.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:25 smithi158 ceph-mon[37241]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-03-04T08:55:25.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:25 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:25.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:25 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:55:25.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:25 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:25.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:25 smithi071 ceph-mon[32789]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-03-04T08:55:28.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:27 smithi150 ceph-mon[36807]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-03-04T08:55:28.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:27 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:28.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:27 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:28.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:27 smithi158 ceph-mon[37241]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-03-04T08:55:28.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:27 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:28.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:27 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:28.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:27 smithi071 ceph-mon[32789]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-03-04T08:55:28.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:27 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:28.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:27 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:29.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:28 smithi150 ceph-mon[36807]: from='osd.11 [v2:172.21.15.158:6824/3332083865,v1:172.21.15.158:6825/3332083865]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-04T08:55:29.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:28 smithi150 ceph-mon[36807]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-04T08:55:29.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:28 smithi158 ceph-mon[37241]: from='osd.11 [v2:172.21.15.158:6824/3332083865,v1:172.21.15.158:6825/3332083865]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-04T08:55:29.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:28 smithi158 ceph-mon[37241]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-04T08:55:29.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:28 smithi071 ceph-mon[32789]: from='osd.11 [v2:172.21.15.158:6824/3332083865,v1:172.21.15.158:6825/3332083865]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-04T08:55:29.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:28 smithi071 ceph-mon[32789]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-04T08:55:30.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:29 smithi150 ceph-mon[36807]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-03-04T08:55:30.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:29 smithi150 ceph-mon[36807]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-04T08:55:30.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:29 smithi150 ceph-mon[36807]: from='osd.11 [v2:172.21.15.158:6824/3332083865,v1:172.21.15.158:6825/3332083865]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-03-04T08:55:30.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:29 smithi150 ceph-mon[36807]: osdmap e80: 12 total, 11 up, 12 in 2024-03-04T08:55:30.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:29 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T08:55:30.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:29 smithi150 ceph-mon[36807]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-03-04T08:55:30.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:29 smithi158 ceph-mon[37241]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-03-04T08:55:30.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:29 smithi158 ceph-mon[37241]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-04T08:55:30.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:29 smithi158 ceph-mon[37241]: from='osd.11 [v2:172.21.15.158:6824/3332083865,v1:172.21.15.158:6825/3332083865]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-03-04T08:55:30.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:29 smithi158 ceph-mon[37241]: osdmap e80: 12 total, 11 up, 12 in 2024-03-04T08:55:30.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:29 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T08:55:30.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:29 smithi158 ceph-mon[37241]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-03-04T08:55:30.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:29 smithi071 ceph-mon[32789]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-03-04T08:55:30.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:29 smithi071 ceph-mon[32789]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-04T08:55:30.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:29 smithi071 ceph-mon[32789]: from='osd.11 [v2:172.21.15.158:6824/3332083865,v1:172.21.15.158:6825/3332083865]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-03-04T08:55:30.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:29 smithi071 ceph-mon[32789]: osdmap e80: 12 total, 11 up, 12 in 2024-03-04T08:55:30.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:29 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T08:55:30.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:29 smithi071 ceph-mon[32789]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]: dispatch 2024-03-04T08:55:31.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:30 smithi150 ceph-mon[36807]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]': finished 2024-03-04T08:55:31.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:30 smithi150 ceph-mon[36807]: osdmap e81: 12 total, 11 up, 12 in 2024-03-04T08:55:31.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:30 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T08:55:31.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:30 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T08:55:31.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:30 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T08:55:31.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:30 smithi158 ceph-mon[37241]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]': finished 2024-03-04T08:55:31.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:30 smithi158 ceph-mon[37241]: osdmap e81: 12 total, 11 up, 12 in 2024-03-04T08:55:31.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:30 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T08:55:31.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:30 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T08:55:31.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:30 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T08:55:31.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:30 smithi071 ceph-mon[32789]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi158", "root=default"]}]': finished 2024-03-04T08:55:31.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:30 smithi071 ceph-mon[32789]: osdmap e81: 12 total, 11 up, 12 in 2024-03-04T08:55:31.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:30 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T08:55:31.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:30 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T08:55:31.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:30 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T08:55:32.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:31 smithi150 ceph-mon[36807]: purged_snaps scrub starts 2024-03-04T08:55:32.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:31 smithi150 ceph-mon[36807]: purged_snaps scrub ok 2024-03-04T08:55:32.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:31 smithi150 ceph-mon[36807]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-03-04T08:55:32.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:31 smithi150 ceph-mon[36807]: osd.11 [v2:172.21.15.158:6824/3332083865,v1:172.21.15.158:6825/3332083865] boot 2024-03-04T08:55:32.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:31 smithi150 ceph-mon[36807]: osdmap e82: 12 total, 12 up, 12 in 2024-03-04T08:55:32.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:31 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T08:55:32.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:31 smithi158 ceph-mon[37241]: purged_snaps scrub starts 2024-03-04T08:55:32.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:31 smithi158 ceph-mon[37241]: purged_snaps scrub ok 2024-03-04T08:55:32.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:31 smithi158 ceph-mon[37241]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-03-04T08:55:32.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:31 smithi158 ceph-mon[37241]: osd.11 [v2:172.21.15.158:6824/3332083865,v1:172.21.15.158:6825/3332083865] boot 2024-03-04T08:55:32.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:31 smithi158 ceph-mon[37241]: osdmap e82: 12 total, 12 up, 12 in 2024-03-04T08:55:32.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:31 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T08:55:32.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:31 smithi071 ceph-mon[32789]: purged_snaps scrub starts 2024-03-04T08:55:32.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:31 smithi071 ceph-mon[32789]: purged_snaps scrub ok 2024-03-04T08:55:32.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:31 smithi071 ceph-mon[32789]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-03-04T08:55:32.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:31 smithi071 ceph-mon[32789]: osd.11 [v2:172.21.15.158:6824/3332083865,v1:172.21.15.158:6825/3332083865] boot 2024-03-04T08:55:32.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:31 smithi071 ceph-mon[32789]: osdmap e82: 12 total, 12 up, 12 in 2024-03-04T08:55:32.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:31 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T08:55:33.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:32 smithi150 ceph-mon[36807]: osdmap e83: 12 total, 12 up, 12 in 2024-03-04T08:55:33.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:32 smithi158 ceph-mon[37241]: osdmap e83: 12 total, 12 up, 12 in 2024-03-04T08:55:33.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:32 smithi071 ceph-mon[32789]: osdmap e83: 12 total, 12 up, 12 in 2024-03-04T08:55:36.012 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:35 smithi150 ceph-mon[36807]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:36.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:35 smithi158 ceph-mon[37241]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:36.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:35 smithi071 ceph-mon[32789]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:36.609 INFO:teuthology.orchestra.run.smithi158.stdout:Created osd(s) 11 on host 'smithi158' 2024-03-04T08:55:36.609 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:36.612+0000 7f7415ffb700 1 -- 172.21.15.158:0/3890545964 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f7420062500 con 0x7f74080a1e10 2024-03-04T08:55:36.611 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:36.615+0000 7f742701d700 1 -- 172.21.15.158:0/3890545964 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f74080a1e10 msgr2=0x7f74080a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:36.611 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:36.615+0000 7f742701d700 1 --2- 172.21.15.158:0/3890545964 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f74080a1e10 0x7f74080a42d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7410009c60 tx=0x7f7410009260 comp rx=0 tx=0).stop 2024-03-04T08:55:36.611 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:36.615+0000 7f742701d700 1 -- 172.21.15.158:0/3890545964 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7420102c00 msgr2=0x7f74201982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:36.611 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:36.615+0000 7f742701d700 1 --2- 172.21.15.158:0/3890545964 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7420102c00 0x7f74201982b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f741c007f40 tx=0x7f741c00d3b0 comp rx=0 tx=0).stop 2024-03-04T08:55:36.612 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:36.615+0000 7f742701d700 1 -- 172.21.15.158:0/3890545964 shutdown_connections 2024-03-04T08:55:36.612 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:36.615+0000 7f742701d700 1 --2- 172.21.15.158:0/3890545964 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f74080a1e10 0x7f74080a42d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:36.612 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:36.615+0000 7f742701d700 1 --2- 172.21.15.158:0/3890545964 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7420102c00 0x7f74201982b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:36.612 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:36.615+0000 7f742701d700 1 --2- 172.21.15.158:0/3890545964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7420107e60 0x7f742019f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:36.612 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:36.615+0000 7f742701d700 1 --2- 172.21.15.158:0/3890545964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7420105530 0x7f74201987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:36.613 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:36.615+0000 7f742701d700 1 -- 172.21.15.158:0/3890545964 >> 172.21.15.158:0/3890545964 conn(0x7f74200fc590 msgr2=0x7f7420103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:55:36.613 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:36.615+0000 7f742701d700 1 -- 172.21.15.158:0/3890545964 shutdown_connections 2024-03-04T08:55:36.613 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:36.615+0000 7f742701d700 1 -- 172.21.15.158:0/3890545964 wait complete. 2024-03-04T08:55:36.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:36 smithi158 ceph-mon[37241]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:36.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:36 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:36.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:36 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:37.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:36 smithi150 ceph-mon[36807]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:37.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:36 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:37.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:36 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:37.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:36 smithi071 ceph-mon[32789]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:37.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:36 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:37.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:36 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:37.287 DEBUG:teuthology.orchestra.run.smithi158:osd.11> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.11.service 2024-03-04T08:55:37.295 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-04T08:55:37.295 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph osd stat -f json 2024-03-04T08:55:37.553 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:55:37.940 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:37 smithi071 ceph-mon[32789]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:38.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:37 smithi150 ceph-mon[36807]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:38.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:37 smithi158 ceph-mon[37241]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:39.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.374+0000 7f25b2de3700 1 -- 172.21.15.71:0/1968449783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25ac101020 msgr2=0x7f25ac101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:39.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.374+0000 7f25b2de3700 1 --2- 172.21.15.71:0/1968449783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25ac101020 0x7f25ac101440 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f25a00099c0 tx=0x7f25a0009cd0 comp rx=0 tx=0).stop 2024-03-04T08:55:39.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.374+0000 7f25b2de3700 1 -- 172.21.15.71:0/1968449783 shutdown_connections 2024-03-04T08:55:39.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.374+0000 7f25b2de3700 1 --2- 172.21.15.71:0/1968449783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25ac102bc0 0x7f25ac109470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:39.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.374+0000 7f25b2de3700 1 --2- 172.21.15.71:0/1968449783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25ac101020 0x7f25ac101440 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:39.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.374+0000 7f25b2de3700 1 --2- 172.21.15.71:0/1968449783 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25ac102220 0x7f25ac102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:39.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.374+0000 7f25b2de3700 1 -- 172.21.15.71:0/1968449783 >> 172.21.15.71:0/1968449783 conn(0x7f25ac0fc5c0 msgr2=0x7f25ac0fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:55:39.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.374+0000 7f25b2de3700 1 -- 172.21.15.71:0/1968449783 shutdown_connections 2024-03-04T08:55:39.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.374+0000 7f25b2de3700 1 -- 172.21.15.71:0/1968449783 wait complete. 2024-03-04T08:55:39.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.374+0000 7f25b2de3700 1 Processor -- start 2024-03-04T08:55:39.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.375+0000 7f25b2de3700 1 -- start start 2024-03-04T08:55:39.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.375+0000 7f25b2de3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25ac101020 0x7f25ac198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:39.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.375+0000 7f25b2de3700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25ac102220 0x7f25ac1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:39.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.375+0000 7f25b2de3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25ac102bc0 0x7f25ac19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:39.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.375+0000 7f25b2de3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac10cbf0 con 0x7f25ac102bc0 2024-03-04T08:55:39.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.375+0000 7f25b2de3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac10ca70 con 0x7f25ac101020 2024-03-04T08:55:39.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.375+0000 7f25b2de3700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac10cd70 con 0x7f25ac102220 2024-03-04T08:55:39.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.375+0000 7f25b1380700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25ac102bc0 0x7f25ac19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:39.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.375+0000 7f25b1380700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25ac102bc0 0x7f25ac19f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51892/0 (socket says 172.21.15.71:51892) 2024-03-04T08:55:39.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.375+0000 7f25b1380700 1 -- 172.21.15.71:0/4240907203 learned_addr learned my addr 172.21.15.71:0/4240907203 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:55:39.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.375+0000 7f25abfff700 1 --2- 172.21.15.71:0/4240907203 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25ac102220 0x7f25ac1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:39.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.375+0000 7f25b0b7f700 1 --2- 172.21.15.71:0/4240907203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25ac101020 0x7f25ac198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:39.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.375+0000 7f25b1380700 1 -- 172.21.15.71:0/4240907203 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25ac102220 msgr2=0x7f25ac1987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:39.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.375+0000 7f25b1380700 1 --2- 172.21.15.71:0/4240907203 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25ac102220 0x7f25ac1987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:39.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.375+0000 7f25b1380700 1 -- 172.21.15.71:0/4240907203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25ac101020 msgr2=0x7f25ac198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:39.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.375+0000 7f25b1380700 1 --2- 172.21.15.71:0/4240907203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25ac101020 0x7f25ac198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:39.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.375+0000 7f25b1380700 1 -- 172.21.15.71:0/4240907203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac19fef0 con 0x7f25ac102bc0 2024-03-04T08:55:39.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.375+0000 7f25b1380700 1 --2- 172.21.15.71:0/4240907203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25ac102bc0 0x7f25ac19f850 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f25a4007da0 tx=0x7f25a400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:55:39.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.375+0000 7f25a9ffb700 1 -- 172.21.15.71:0/4240907203 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4016020 con 0x7f25ac102bc0 2024-03-04T08:55:39.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.375+0000 7f25a9ffb700 1 -- 172.21.15.71:0/4240907203 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25a400ac50 con 0x7f25ac102bc0 2024-03-04T08:55:39.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.375+0000 7f25b2de3700 1 -- 172.21.15.71:0/4240907203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25ac1a01e0 con 0x7f25ac102bc0 2024-03-04T08:55:39.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.375+0000 7f25a9ffb700 1 -- 172.21.15.71:0/4240907203 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a40115e0 con 0x7f25ac102bc0 2024-03-04T08:55:39.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.376+0000 7f25b2de3700 1 -- 172.21.15.71:0/4240907203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ac1a06d0 con 0x7f25ac102bc0 2024-03-04T08:55:39.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.377+0000 7f25b2de3700 1 -- 172.21.15.71:0/4240907203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25ac068d80 con 0x7f25ac102bc0 2024-03-04T08:55:39.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.378+0000 7f25a9ffb700 1 -- 172.21.15.71:0/4240907203 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f25a4019020 con 0x7f25ac102bc0 2024-03-04T08:55:39.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.378+0000 7f25a9ffb700 1 --2- 172.21.15.71:0/4240907203 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f259c0a1e10 0x7f259c0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:39.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.378+0000 7f25b0b7f700 1 --2- 172.21.15.71:0/4240907203 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f259c0a1e10 0x7f259c0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:39.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.379+0000 7f25a9ffb700 1 -- 172.21.15.71:0/4240907203 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f25a40cdb90 con 0x7f25ac102bc0 2024-03-04T08:55:39.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.379+0000 7f25b0b7f700 1 --2- 172.21.15.71:0/4240907203 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f259c0a1e10 0x7f259c0a42d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f25a0003890 tx=0x7f25a0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:55:39.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.386+0000 7f25a9ffb700 1 -- 172.21.15.71:0/4240907203 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f25a4020020 con 0x7f25ac102bc0 2024-03-04T08:55:39.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.590+0000 7f25b2de3700 1 -- 172.21.15.71:0/4240907203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f25ac1a0b50 con 0x7f25ac102bc0 2024-03-04T08:55:39.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.591+0000 7f25a9ffb700 1 -- 172.21.15.71:0/4240907203 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v83) v1 ==== 74+0+133 (secure 0 0 0) 0x7f25a400ae00 con 0x7f25ac102bc0 2024-03-04T08:55:39.587 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:55:39.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.594+0000 7f25b2de3700 1 -- 172.21.15.71:0/4240907203 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f259c0a1e10 msgr2=0x7f259c0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:39.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.594+0000 7f25b2de3700 1 --2- 172.21.15.71:0/4240907203 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f259c0a1e10 0x7f259c0a42d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f25a0003890 tx=0x7f25a0005fb0 comp rx=0 tx=0).stop 2024-03-04T08:55:39.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.594+0000 7f25b2de3700 1 -- 172.21.15.71:0/4240907203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25ac102bc0 msgr2=0x7f25ac19f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:39.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.594+0000 7f25b2de3700 1 --2- 172.21.15.71:0/4240907203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25ac102bc0 0x7f25ac19f850 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f25a4007da0 tx=0x7f25a400a340 comp rx=0 tx=0).stop 2024-03-04T08:55:39.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.594+0000 7f25b2de3700 1 -- 172.21.15.71:0/4240907203 shutdown_connections 2024-03-04T08:55:39.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.594+0000 7f25b2de3700 1 --2- 172.21.15.71:0/4240907203 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f259c0a1e10 0x7f259c0a42d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:39.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.594+0000 7f25b2de3700 1 --2- 172.21.15.71:0/4240907203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25ac102bc0 0x7f25ac19f850 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:39.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.594+0000 7f25b2de3700 1 --2- 172.21.15.71:0/4240907203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25ac101020 0x7f25ac198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:39.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.594+0000 7f25b2de3700 1 --2- 172.21.15.71:0/4240907203 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f25ac102220 0x7f25ac1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:39.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.594+0000 7f25b2de3700 1 -- 172.21.15.71:0/4240907203 >> 172.21.15.71:0/4240907203 conn(0x7f25ac0fc5c0 msgr2=0x7f25ac06a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:55:39.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.594+0000 7f25b2de3700 1 -- 172.21.15.71:0/4240907203 shutdown_connections 2024-03-04T08:55:39.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:39.594+0000 7f25b2de3700 1 -- 172.21.15.71:0/4240907203 wait complete. 2024-03-04T08:55:39.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:39 smithi071 ceph-mon[32789]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:39.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:39 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/4240907203' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-04T08:55:39.953 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:39 smithi158 ceph-mon[37241]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:39.953 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:39 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/4240907203' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-04T08:55:40.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:39 smithi150 ceph-mon[36807]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:40.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:39 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/4240907203' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-04T08:55:40.265 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":83,"num_osds":12,"num_up_osds":12,"osd_up_since":1709542530,"num_in_osds":12,"osd_in_since":1709542511,"num_remapped_pgs":0} 2024-03-04T08:55:40.266 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph osd dump --format=json 2024-03-04T08:55:40.512 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:55:41.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:41 smithi071 ceph-mon[32789]: Detected new or changed devices on smithi158 2024-03-04T08:55:41.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:41.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:41.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:55:41.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:55:41.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:55:41.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:55:41.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:41 smithi071 ceph-mon[32789]: Adjusting osd_memory_target on smithi158 to 4270M 2024-03-04T08:55:41.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:41.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:41.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:55:41.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:41.673 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:41 smithi071 ceph-mon[32789]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:41.673 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:55:41.673 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:41.673 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:55:41.673 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:41 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:41.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:41 smithi150 ceph-mon[36807]: Detected new or changed devices on smithi158 2024-03-04T08:55:41.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:41.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:41.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:55:41.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:55:41.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:55:41.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:55:41.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:41 smithi150 ceph-mon[36807]: Adjusting osd_memory_target on smithi158 to 4270M 2024-03-04T08:55:41.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:41.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:41.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:55:41.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:41.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:41 smithi150 ceph-mon[36807]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:41.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:55:41.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:41.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:55:41.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:41 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:41.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:41 smithi158 ceph-mon[37241]: Detected new or changed devices on smithi158 2024-03-04T08:55:41.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:41.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:41.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:55:41.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:55:41.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:55:41.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:55:41.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:41 smithi158 ceph-mon[37241]: Adjusting osd_memory_target on smithi158 to 4270M 2024-03-04T08:55:41.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:41.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:41.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:55:41.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:41.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:41 smithi158 ceph-mon[37241]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:41.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:55:41.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:41.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:55:41.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:41 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:42.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.303+0000 7f8b3777e700 1 -- 172.21.15.71:0/2137685292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b30101c60 msgr2=0x7f8b3010de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:42.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.303+0000 7f8b3777e700 1 --2- 172.21.15.71:0/2137685292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b30101c60 0x7f8b3010de90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800b0a0 tx=0x7f8b2800b3b0 comp rx=0 tx=0).stop 2024-03-04T08:55:42.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.303+0000 7f8b3777e700 1 -- 172.21.15.71:0/2137685292 shutdown_connections 2024-03-04T08:55:42.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.303+0000 7f8b3777e700 1 --2- 172.21.15.71:0/2137685292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b30101c60 0x7f8b3010de90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:42.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.303+0000 7f8b3777e700 1 --2- 172.21.15.71:0/2137685292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b301071b0 0x7f8b30107590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:42.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.303+0000 7f8b3777e700 1 --2- 172.21.15.71:0/2137685292 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8b301012a0 0x7f8b30101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:42.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.303+0000 7f8b3777e700 1 -- 172.21.15.71:0/2137685292 >> 172.21.15.71:0/2137685292 conn(0x7f8b30077e20 msgr2=0x7f8b30078230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:55:42.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.303+0000 7f8b3777e700 1 -- 172.21.15.71:0/2137685292 shutdown_connections 2024-03-04T08:55:42.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.303+0000 7f8b3777e700 1 -- 172.21.15.71:0/2137685292 wait complete. 2024-03-04T08:55:42.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.304+0000 7f8b3777e700 1 Processor -- start 2024-03-04T08:55:42.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.304+0000 7f8b3777e700 1 -- start start 2024-03-04T08:55:42.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.304+0000 7f8b3777e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8b301012a0 0x7f8b30197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:42.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.304+0000 7f8b3777e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b30101c60 0x7f8b30197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:42.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.304+0000 7f8b3777e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b301071b0 0x7f8b3019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:42.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.304+0000 7f8b3777e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b30111410 con 0x7f8b301071b0 2024-03-04T08:55:42.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.304+0000 7f8b3777e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b30111290 con 0x7f8b30101c60 2024-03-04T08:55:42.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.304+0000 7f8b3777e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b30111590 con 0x7f8b301012a0 2024-03-04T08:55:42.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.304+0000 7f8b35d1b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b301071b0 0x7f8b3019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:42.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.304+0000 7f8b35d1b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b301071b0 0x7f8b3019c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51900/0 (socket says 172.21.15.71:51900) 2024-03-04T08:55:42.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.304+0000 7f8b35d1b700 1 -- 172.21.15.71:0/3749711811 learned_addr learned my addr 172.21.15.71:0/3749711811 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:55:42.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.305+0000 7f8b3551a700 1 --2- 172.21.15.71:0/3749711811 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8b301012a0 0x7f8b30197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:42.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.305+0000 7f8b34d19700 1 --2- 172.21.15.71:0/3749711811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b30101c60 0x7f8b30197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:42.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.305+0000 7f8b35d1b700 1 -- 172.21.15.71:0/3749711811 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8b301012a0 msgr2=0x7f8b30197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:42.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.305+0000 7f8b35d1b700 1 --2- 172.21.15.71:0/3749711811 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8b301012a0 0x7f8b30197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:42.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.305+0000 7f8b35d1b700 1 -- 172.21.15.71:0/3749711811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b30101c60 msgr2=0x7f8b30197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:42.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.305+0000 7f8b35d1b700 1 --2- 172.21.15.71:0/3749711811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b30101c60 0x7f8b30197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:42.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.305+0000 7f8b35d1b700 1 -- 172.21.15.71:0/3749711811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3019cad0 con 0x7f8b301071b0 2024-03-04T08:55:42.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.305+0000 7f8b3551a700 1 --2- 172.21.15.71:0/3749711811 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8b301012a0 0x7f8b30197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:55:42.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.305+0000 7f8b34d19700 1 --2- 172.21.15.71:0/3749711811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b30101c60 0x7f8b30197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:55:42.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.305+0000 7f8b35d1b700 1 --2- 172.21.15.71:0/3749711811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b301071b0 0x7f8b3019c590 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8b280062a0 tx=0x7f8b28004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:55:42.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.305+0000 7f8b267fc700 1 -- 172.21.15.71:0/3749711811 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b28021070 con 0x7f8b301071b0 2024-03-04T08:55:42.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.306+0000 7f8b3777e700 1 -- 172.21.15.71:0/3749711811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3019cd60 con 0x7f8b301071b0 2024-03-04T08:55:42.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.306+0000 7f8b267fc700 1 -- 172.21.15.71:0/3749711811 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b28007df0 con 0x7f8b301071b0 2024-03-04T08:55:42.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.306+0000 7f8b267fc700 1 -- 172.21.15.71:0/3749711811 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b28005430 con 0x7f8b301071b0 2024-03-04T08:55:42.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.306+0000 7f8b3777e700 1 -- 172.21.15.71:0/3749711811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3019d250 con 0x7f8b301071b0 2024-03-04T08:55:42.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.307+0000 7f8b3777e700 1 -- 172.21.15.71:0/3749711811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b30102740 con 0x7f8b301071b0 2024-03-04T08:55:42.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.308+0000 7f8b267fc700 1 -- 172.21.15.71:0/3749711811 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f8b28024070 con 0x7f8b301071b0 2024-03-04T08:55:42.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.309+0000 7f8b267fc700 1 --2- 172.21.15.71:0/3749711811 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f8b1c0a1e10 0x7f8b1c0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:42.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.309+0000 7f8b3551a700 1 --2- 172.21.15.71:0/3749711811 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f8b1c0a1e10 0x7f8b1c0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:42.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.310+0000 7f8b267fc700 1 -- 172.21.15.71:0/3749711811 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8b280cebb0 con 0x7f8b301071b0 2024-03-04T08:55:42.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.310+0000 7f8b3551a700 1 --2- 172.21.15.71:0/3749711811 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f8b1c0a1e10 0x7f8b1c0a42d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8b20007900 tx=0x7f8b20008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:55:42.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.313+0000 7f8b267fc700 1 -- 172.21.15.71:0/3749711811 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8b2809c470 con 0x7f8b301071b0 2024-03-04T08:55:42.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.510+0000 7f8b3777e700 1 -- 172.21.15.71:0/3749711811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b3006ae50 con 0x7f8b301071b0 2024-03-04T08:55:42.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.511+0000 7f8b267fc700 1 -- 172.21.15.71:0/3749711811 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+18481 (secure 0 0 0) 0x7f8b2809bd70 con 0x7f8b301071b0 2024-03-04T08:55:42.507 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:55:42.507 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":83,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","created":"2024-03-04T08:47:28.546102+0000","modified":"2024-03-04T08:55:31.753527+0000","last_up_change":"2024-03-04T08:55:30.744961+0000","last_in_change":"2024-03-04T08:55:11.570990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T08:51:14.933487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e43b47d-8888-4631-bc96-8a96fcf8cd46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6803","nonce":1173641530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6805","nonce":1173641530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6809","nonce":1173641530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6807","nonce":1173641530}]},"public_addr":"172.21.15.71:6803/1173641530","cluster_addr":"172.21.15.71:6805/1173641530","heartbeat_back_addr":"172.21.15.71:6809/1173641530","heartbeat_front_addr":"172.21.15.71:6807/1173641530","state":["exists","up"]},{"osd":1,"uuid":"1209f214-f2e2-4209-bf69-4528a43a2123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6801","nonce":3383948110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6803","nonce":3383948110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6807","nonce":3383948110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6805","nonce":3383948110}]},"public_addr":"172.21.15.150:6801/3383948110","cluster_addr":"172.21.15.150:6803/3383948110","heartbeat_back_addr":"172.21.15.150:6807/3383948110","heartbeat_front_addr":"172.21.15.150:6805/3383948110","state":["exists","up"]},{"osd":2,"uuid":"b88c3082-93a9-4c9b-9221-a7105a44c91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6801","nonce":586944144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6803","nonce":586944144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6807","nonce":586944144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6805","nonce":586944144}]},"public_addr":"172.21.15.158:6801/586944144","cluster_addr":"172.21.15.158:6803/586944144","heartbeat_back_addr":"172.21.15.158:6807/586944144","heartbeat_front_addr":"172.21.15.158:6805/586944144","state":["exists","up"]},{"osd":3,"uuid":"e4fc7723-fd73-40b2-b9d0-7f9bda282ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6811","nonce":1311903262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6813","nonce":1311903262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6817","nonce":1311903262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6815","nonce":1311903262}]},"public_addr":"172.21.15.71:6811/1311903262","cluster_addr":"172.21.15.71:6813/1311903262","heartbeat_back_addr":"172.21.15.71:6817/1311903262","heartbeat_front_addr":"172.21.15.71:6815/1311903262","state":["exists","up"]},{"osd":4,"uuid":"e7eb72ac-f066-4ccd-b2bf-22a6a26daf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6809","nonce":2300837573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6811","nonce":2300837573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6815","nonce":2300837573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6813","nonce":2300837573}]},"public_addr":"172.21.15.150:6809/2300837573","cluster_addr":"172.21.15.150:6811/2300837573","heartbeat_back_addr":"172.21.15.150:6815/2300837573","heartbeat_front_addr":"172.21.15.150:6813/2300837573","state":["exists","up"]},{"osd":5,"uuid":"9b295840-e104-4a66-9cc4-7020627c1765","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6809","nonce":1922694045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6811","nonce":1922694045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6815","nonce":1922694045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6813","nonce":1922694045}]},"public_addr":"172.21.15.158:6809/1922694045","cluster_addr":"172.21.15.158:6811/1922694045","heartbeat_back_addr":"172.21.15.158:6815/1922694045","heartbeat_front_addr":"172.21.15.158:6813/1922694045","state":["exists","up"]},{"osd":6,"uuid":"6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6819","nonce":3848842707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6821","nonce":3848842707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6825","nonce":3848842707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6823","nonce":3848842707}]},"public_addr":"172.21.15.71:6819/3848842707","cluster_addr":"172.21.15.71:6821/3848842707","heartbeat_back_addr":"172.21.15.71:6825/3848842707","heartbeat_front_addr":"172.21.15.71:6823/3848842707","state":["exists","up"]},{"osd":7,"uuid":"6160a40a-d30f-4569-949c-0afb1bb3d133","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6817","nonce":1328330305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6819","nonce":1328330305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6823","nonce":1328330305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6821","nonce":1328330305}]},"public_addr":"172.21.15.150:6817/1328330305","cluster_addr":"172.21.15.150:6819/1328330305","heartbeat_back_addr":"172.21.15.150:6823/1328330305","heartbeat_front_addr":"172.21.15.150:6821/1328330305","state":["exists","up"]},{"osd":8,"uuid":"95ed2341-84a4-465c-9948-ac667680ce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6817","nonce":2195304577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6819","nonce":2195304577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6823","nonce":2195304577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6821","nonce":2195304577}]},"public_addr":"172.21.15.158:6817/2195304577","cluster_addr":"172.21.15.158:6819/2195304577","heartbeat_back_addr":"172.21.15.158:6823/2195304577","heartbeat_front_addr":"172.21.15.158:6821/2195304577","state":["exists","up"]},{"osd":9,"uuid":"088f6419-63a0-4768-afae-dbb8a5f5a565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6827","nonce":1913779347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6829","nonce":1913779347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6833","nonce":1913779347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6831","nonce":1913779347}]},"public_addr":"172.21.15.71:6827/1913779347","cluster_addr":"172.21.15.71:6829/1913779347","heartbeat_back_addr":"172.21.15.71:6833/1913779347","heartbeat_front_addr":"172.21.15.71:6831/1913779347","state":["exists","up"]},{"osd":10,"uuid":"18c85e24-068d-4f1e-a3b8-f7915b363234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6825","nonce":3295420996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6827","nonce":3295420996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6831","nonce":3295420996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6829","nonce":3295420996}]},"public_addr":"172.21.15.150:6825/3295420996","cluster_addr":"172.21.15.150:6827/3295420996","heartbeat_back_addr":"172.21.15.150:6831/3295420996","heartbeat_front_addr":"172.21.15.150:6829/3295420996","state":["exists","up"]},{"osd":11,"uuid":"4f6b3ddf-c482-4740-b1fd-de836e047c2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6825","nonce":3332083865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6827","nonce":3332083865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6831","nonce":3332083865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6829","nonce":3332083865}]},"public_addr":"172.21.15.158:6825/3332083865","cluster_addr":"172.21.15.158:6827/3332083865","heartbeat_back_addr":"172.21.15.158:6831/3332083865","heartbeat_front_addr":"172.21.15.158:6829/3332083865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:05.997832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:29.314235+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:08.819465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:41.194275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:06.032479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:38.814498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:13.067960+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:38.568479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:03.416713+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:38.349479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:04.096945+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:29.425250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1287712358":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/3770452860":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/718743094":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/1904717532":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:6800/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6800/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/2862184313":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/3621754630":"2024-03-05T08:48:22.593931+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T08:55:42.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.513+0000 7f8b3777e700 1 -- 172.21.15.71:0/3749711811 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f8b1c0a1e10 msgr2=0x7f8b1c0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:42.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.513+0000 7f8b3777e700 1 --2- 172.21.15.71:0/3749711811 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f8b1c0a1e10 0x7f8b1c0a42d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8b20007900 tx=0x7f8b20008040 comp rx=0 tx=0).stop 2024-03-04T08:55:42.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.513+0000 7f8b3777e700 1 -- 172.21.15.71:0/3749711811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b301071b0 msgr2=0x7f8b3019c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:42.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.513+0000 7f8b3777e700 1 --2- 172.21.15.71:0/3749711811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b301071b0 0x7f8b3019c590 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8b280062a0 tx=0x7f8b28004ef0 comp rx=0 tx=0).stop 2024-03-04T08:55:42.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.513+0000 7f8b3777e700 1 -- 172.21.15.71:0/3749711811 shutdown_connections 2024-03-04T08:55:42.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.513+0000 7f8b3777e700 1 --2- 172.21.15.71:0/3749711811 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f8b1c0a1e10 0x7f8b1c0a42d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:42.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.513+0000 7f8b3777e700 1 --2- 172.21.15.71:0/3749711811 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8b301012a0 0x7f8b30197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:42.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.513+0000 7f8b3777e700 1 --2- 172.21.15.71:0/3749711811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b301071b0 0x7f8b3019c590 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:42.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.513+0000 7f8b3777e700 1 --2- 172.21.15.71:0/3749711811 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b30101c60 0x7f8b30197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:42.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.513+0000 7f8b3777e700 1 -- 172.21.15.71:0/3749711811 >> 172.21.15.71:0/3749711811 conn(0x7f8b30077e20 msgr2=0x7f8b300ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:55:42.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.513+0000 7f8b3777e700 1 -- 172.21.15.71:0/3749711811 shutdown_connections 2024-03-04T08:55:42.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:42.513+0000 7f8b3777e700 1 -- 172.21.15.71:0/3749711811 wait complete. 2024-03-04T08:55:42.838 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:42 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3749711811' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:55:43.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:42 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3749711811' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:55:43.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:42 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3749711811' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:55:43.200 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T08:51:14.933487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-04T08:55:43.201 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-04T08:55:43.439 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:55:43.843 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:43 smithi071 ceph-mon[32789]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:44.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:43 smithi150 ceph-mon[36807]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:44.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:43 smithi158 ceph-mon[37241]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:45.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.157+0000 7f7214aa8700 1 -- 172.21.15.71:0/463555092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f721006b410 msgr2=0x7f7210106dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:45.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.157+0000 7f7214aa8700 1 --2- 172.21.15.71:0/463555092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f721006b410 0x7f7210106dd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7204009990 tx=0x7f7204009ca0 comp rx=0 tx=0).stop 2024-03-04T08:55:45.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.157+0000 7f7214aa8700 1 -- 172.21.15.71:0/463555092 shutdown_connections 2024-03-04T08:55:45.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.157+0000 7f7214aa8700 1 --2- 172.21.15.71:0/463555092 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7210107500 0x7f721010f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:45.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.157+0000 7f7214aa8700 1 --2- 172.21.15.71:0/463555092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f721006aa60 0x7f721006ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:45.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.157+0000 7f7214aa8700 1 --2- 172.21.15.71:0/463555092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f721006b410 0x7f7210106dd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:45.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.157+0000 7f7214aa8700 1 -- 172.21.15.71:0/463555092 >> 172.21.15.71:0/463555092 conn(0x7f7210077e20 msgr2=0x7f7210078230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:55:45.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.157+0000 7f7214aa8700 1 -- 172.21.15.71:0/463555092 shutdown_connections 2024-03-04T08:55:45.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.157+0000 7f7214aa8700 1 -- 172.21.15.71:0/463555092 wait complete. 2024-03-04T08:55:45.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.158+0000 7f7214aa8700 1 Processor -- start 2024-03-04T08:55:45.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.158+0000 7f7214aa8700 1 -- start start 2024-03-04T08:55:45.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.158+0000 7f7214aa8700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f721006aa60 0x7f721019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:45.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.158+0000 7f7214aa8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f721006b410 0x7f721019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:45.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.158+0000 7f7214aa8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7210107500 0x7f72101986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:45.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.158+0000 7f7214aa8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7210112440 con 0x7f7210107500 2024-03-04T08:55:45.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.158+0000 7f7214aa8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72101122c0 con 0x7f721006b410 2024-03-04T08:55:45.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.158+0000 7f7214aa8700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72101125c0 con 0x7f721006aa60 2024-03-04T08:55:45.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.158+0000 7f720ed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7210107500 0x7f72101986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:45.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.158+0000 7f720ed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7210107500 0x7f72101986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51914/0 (socket says 172.21.15.71:51914) 2024-03-04T08:55:45.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.158+0000 7f720ed9d700 1 -- 172.21.15.71:0/717581072 learned_addr learned my addr 172.21.15.71:0/717581072 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:55:45.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.158+0000 7f720dd9b700 1 --2- 172.21.15.71:0/717581072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f721006b410 0x7f721019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:45.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.158+0000 7f720e59c700 1 --2- 172.21.15.71:0/717581072 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f721006aa60 0x7f721019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:45.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.158+0000 7f720ed9d700 1 -- 172.21.15.71:0/717581072 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f721006aa60 msgr2=0x7f721019e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:45.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.159+0000 7f720ed9d700 1 --2- 172.21.15.71:0/717581072 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f721006aa60 0x7f721019e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:45.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.159+0000 7f720ed9d700 1 -- 172.21.15.71:0/717581072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f721006b410 msgr2=0x7f721019eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:45.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.159+0000 7f720ed9d700 1 --2- 172.21.15.71:0/717581072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f721006b410 0x7f721019eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:45.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.159+0000 7f720ed9d700 1 -- 172.21.15.71:0/717581072 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7210198fa0 con 0x7f7210107500 2024-03-04T08:55:45.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.159+0000 7f720ed9d700 1 --2- 172.21.15.71:0/717581072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7210107500 0x7f72101986c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7208007f10 tx=0x7f720800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:55:45.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.159+0000 7f71ff7fe700 1 -- 172.21.15.71:0/717581072 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7208009e20 con 0x7f7210107500 2024-03-04T08:55:45.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.159+0000 7f71ff7fe700 1 -- 172.21.15.71:0/717581072 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7208004bb0 con 0x7f7210107500 2024-03-04T08:55:45.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.159+0000 7f71ff7fe700 1 -- 172.21.15.71:0/717581072 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7208005960 con 0x7f7210107500 2024-03-04T08:55:45.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.159+0000 7f7214aa8700 1 -- 172.21.15.71:0/717581072 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7210199290 con 0x7f7210107500 2024-03-04T08:55:45.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.159+0000 7f7214aa8700 1 -- 172.21.15.71:0/717581072 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72101a5520 con 0x7f7210107500 2024-03-04T08:55:45.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.160+0000 7f7214aa8700 1 -- 172.21.15.71:0/717581072 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7210103880 con 0x7f7210107500 2024-03-04T08:55:45.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.161+0000 7f71ff7fe700 1 -- 172.21.15.71:0/717581072 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f720801a020 con 0x7f7210107500 2024-03-04T08:55:45.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.161+0000 7f71ff7fe700 1 --2- 172.21.15.71:0/717581072 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f71f80a1e90 0x7f71f80a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:45.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.162+0000 7f720e59c700 1 --2- 172.21.15.71:0/717581072 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f71f80a1e90 0x7f71f80a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:45.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.162+0000 7f71ff7fe700 1 -- 172.21.15.71:0/717581072 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f72080cd520 con 0x7f7210107500 2024-03-04T08:55:45.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.162+0000 7f720e59c700 1 --2- 172.21.15.71:0/717581072 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f71f80a1e90 0x7f71f80a4350 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7200005950 tx=0x7f720000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:55:45.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.165+0000 7f71ff7fe700 1 -- 172.21.15.71:0/717581072 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7208010070 con 0x7f7210107500 2024-03-04T08:55:45.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.357+0000 7f7214aa8700 1 -- 172.21.15.71:0/717581072 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7210199ba0 con 0x7f7210107500 2024-03-04T08:55:45.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.357+0000 7f71ff7fe700 1 -- 172.21.15.71:0/717581072 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f72080192c0 con 0x7f7210107500 2024-03-04T08:55:45.355 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-03-04T08:55:45.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.360+0000 7f7214aa8700 1 -- 172.21.15.71:0/717581072 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f71f80a1e90 msgr2=0x7f71f80a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:45.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.360+0000 7f7214aa8700 1 --2- 172.21.15.71:0/717581072 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f71f80a1e90 0x7f71f80a4350 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7200005950 tx=0x7f720000b410 comp rx=0 tx=0).stop 2024-03-04T08:55:45.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.360+0000 7f7214aa8700 1 -- 172.21.15.71:0/717581072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7210107500 msgr2=0x7f72101986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:45.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.360+0000 7f7214aa8700 1 --2- 172.21.15.71:0/717581072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7210107500 0x7f72101986c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7208007f10 tx=0x7f720800d3b0 comp rx=0 tx=0).stop 2024-03-04T08:55:45.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.360+0000 7f7214aa8700 1 -- 172.21.15.71:0/717581072 shutdown_connections 2024-03-04T08:55:45.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.360+0000 7f7214aa8700 1 --2- 172.21.15.71:0/717581072 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f71f80a1e90 0x7f71f80a4350 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:45.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.360+0000 7f7214aa8700 1 --2- 172.21.15.71:0/717581072 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f721006aa60 0x7f721019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:45.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.360+0000 7f7214aa8700 1 --2- 172.21.15.71:0/717581072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7210107500 0x7f72101986c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:45.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.360+0000 7f7214aa8700 1 --2- 172.21.15.71:0/717581072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f721006b410 0x7f721019eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:45.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.360+0000 7f7214aa8700 1 -- 172.21.15.71:0/717581072 >> 172.21.15.71:0/717581072 conn(0x7f7210077e20 msgr2=0x7f72100ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:55:45.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.360+0000 7f7214aa8700 1 -- 172.21.15.71:0/717581072 shutdown_connections 2024-03-04T08:55:45.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:45.360+0000 7f7214aa8700 1 -- 172.21.15.71:0/717581072 wait complete. 2024-03-04T08:55:46.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:45 smithi150 ceph-mon[36807]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:46.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:45 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/717581072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:55:46.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:45 smithi158 ceph-mon[37241]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:46.056 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:45 smithi071 ceph-mon[32789]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:46.056 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:45 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/717581072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:55:46.084 INFO:tasks.cephadm:Adding mds.a on smithi071 2024-03-04T08:55:46.084 INFO:tasks.cephadm:Adding mds.d on smithi071 2024-03-04T08:55:46.084 INFO:tasks.cephadm:Adding mds.g on smithi071 2024-03-04T08:55:46.084 INFO:tasks.cephadm:Adding mds.j on smithi071 2024-03-04T08:55:46.084 INFO:tasks.cephadm:Adding mds.b on smithi150 2024-03-04T08:55:46.084 INFO:tasks.cephadm:Adding mds.e on smithi150 2024-03-04T08:55:46.084 INFO:tasks.cephadm:Adding mds.h on smithi150 2024-03-04T08:55:46.084 INFO:tasks.cephadm:Adding mds.k on smithi150 2024-03-04T08:55:46.084 INFO:tasks.cephadm:Adding mds.c on smithi158 2024-03-04T08:55:46.084 INFO:tasks.cephadm:Adding mds.f on smithi158 2024-03-04T08:55:46.084 INFO:tasks.cephadm:Adding mds.i on smithi158 2024-03-04T08:55:46.084 INFO:tasks.cephadm:Adding mds.l on smithi158 2024-03-04T08:55:46.085 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph orch apply mds all '12;smithi071=a;smithi071=d;smithi071=g;smithi071=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi158=c;smithi158=f;smithi158=i;smithi158=l' 2024-03-04T08:55:49.506 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T08:55:49.628 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:49 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/717581072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:55:49.628 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:49 smithi158 ceph-mon[37241]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:49.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:49 smithi071 ceph-mon[32789]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:49.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:49 smithi150 ceph-mon[36807]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:50.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:50 smithi071 ceph-mon[32789]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:50.723 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:50 smithi158 ceph-mon[37241]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:50.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:50 smithi150 ceph-mon[36807]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:51.333 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.331+0000 7f05f61ac700 1 -- 172.21.15.158:0/2743417423 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05f0103c90 msgr2=0x7f05f010a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:51.333 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.331+0000 7f05f61ac700 1 --2- 172.21.15.158:0/2743417423 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05f0103c90 0x7f05f010a540 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f05e800b0a0 tx=0x7f05e800b3b0 comp rx=0 tx=0).stop 2024-03-04T08:55:51.333 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.331+0000 7f05f61ac700 1 -- 172.21.15.158:0/2743417423 shutdown_connections 2024-03-04T08:55:51.333 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.331+0000 7f05f61ac700 1 --2- 172.21.15.158:0/2743417423 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05f0103c90 0x7f05f010a540 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:51.333 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.331+0000 7f05f61ac700 1 --2- 172.21.15.158:0/2743417423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05f01020f0 0x7f05f0102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:51.333 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.331+0000 7f05f61ac700 1 --2- 172.21.15.158:0/2743417423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05f01032f0 0x7f05f0103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:51.333 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.331+0000 7f05f61ac700 1 -- 172.21.15.158:0/2743417423 >> 172.21.15.158:0/2743417423 conn(0x7f05f00fd6d0 msgr2=0x7f05f00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:55:51.333 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.331+0000 7f05f61ac700 1 -- 172.21.15.158:0/2743417423 shutdown_connections 2024-03-04T08:55:51.334 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.331+0000 7f05f61ac700 1 -- 172.21.15.158:0/2743417423 wait complete. 2024-03-04T08:55:51.334 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.332+0000 7f05f61ac700 1 Processor -- start 2024-03-04T08:55:51.334 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.332+0000 7f05f61ac700 1 -- start start 2024-03-04T08:55:51.334 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.333+0000 7f05f61ac700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05f01020f0 0x7f05f0198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:51.334 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.333+0000 7f05f61ac700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05f01032f0 0x7f05f01987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:51.334 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.333+0000 7f05f61ac700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05f0103c90 0x7f05f019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:51.334 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.333+0000 7f05f61ac700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f010dcc0 con 0x7f05f0103c90 2024-03-04T08:55:51.334 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.333+0000 7f05f61ac700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f010db40 con 0x7f05f01020f0 2024-03-04T08:55:51.334 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.333+0000 7f05f61ac700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f010de40 con 0x7f05f01032f0 2024-03-04T08:55:51.334 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.333+0000 7f05e6ffd700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05f01032f0 0x7f05f01987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:51.334 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.333+0000 7f05e6ffd700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05f01032f0 0x7f05f01987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.158:52226/0 (socket says 172.21.15.158:52226) 2024-03-04T08:55:51.335 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.333+0000 7f05e6ffd700 1 -- 172.21.15.158:0/2309619309 learned_addr learned my addr 172.21.15.158:0/2309619309 (peer_addr_for_me v2:172.21.15.158:0/0) 2024-03-04T08:55:51.335 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.333+0000 7f05ef7fe700 1 --2- 172.21.15.158:0/2309619309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05f01020f0 0x7f05f0198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:51.335 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.333+0000 7f05effff700 1 --2- 172.21.15.158:0/2309619309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05f0103c90 0x7f05f019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:51.335 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.333+0000 7f05e6ffd700 1 -- 172.21.15.158:0/2309619309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05f01020f0 msgr2=0x7f05f0198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:51.335 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.333+0000 7f05e6ffd700 1 --2- 172.21.15.158:0/2309619309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05f01020f0 0x7f05f0198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:51.335 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.333+0000 7f05e6ffd700 1 -- 172.21.15.158:0/2309619309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05f0103c90 msgr2=0x7f05f019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:51.335 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.333+0000 7f05e6ffd700 1 --2- 172.21.15.158:0/2309619309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05f0103c90 0x7f05f019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:51.335 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.333+0000 7f05e6ffd700 1 -- 172.21.15.158:0/2309619309 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f019fee0 con 0x7f05f01032f0 2024-03-04T08:55:51.335 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.333+0000 7f05effff700 1 --2- 172.21.15.158:0/2309619309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05f0103c90 0x7f05f019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:55:51.336 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.333+0000 7f05e6ffd700 1 --2- 172.21.15.158:0/2309619309 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05f01032f0 0x7f05f01987c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f05d800d980 tx=0x7f05d800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:55:51.336 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.333+0000 7f05ed7fa700 1 -- 172.21.15.158:0/2309619309 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d8016070 con 0x7f05f01032f0 2024-03-04T08:55:51.336 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.333+0000 7f05ed7fa700 1 -- 172.21.15.158:0/2309619309 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05d800de70 con 0x7f05f01032f0 2024-03-04T08:55:51.336 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.333+0000 7f05f61ac700 1 -- 172.21.15.158:0/2309619309 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f01a01d0 con 0x7f05f01032f0 2024-03-04T08:55:51.336 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.333+0000 7f05ed7fa700 1 -- 172.21.15.158:0/2309619309 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d800f3c0 con 0x7f05f01032f0 2024-03-04T08:55:51.336 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.334+0000 7f05f61ac700 1 -- 172.21.15.158:0/2309619309 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f01a0710 con 0x7f05f01032f0 2024-03-04T08:55:51.336 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.334+0000 7f05f61ac700 1 -- 172.21.15.158:0/2309619309 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f0068d80 con 0x7f05f01032f0 2024-03-04T08:55:51.336 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.335+0000 7f05ed7fa700 1 -- 172.21.15.158:0/2309619309 <== mon.1 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f05d8019070 con 0x7f05f01032f0 2024-03-04T08:55:51.336 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.336+0000 7f05ed7fa700 1 --2- 172.21.15.158:0/2309619309 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f05dc0a1e10 0x7f05dc0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:51.336 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.336+0000 7f05ed7fa700 1 -- 172.21.15.158:0/2309619309 <== mon.1 v2:172.21.15.158:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f05d80cd440 con 0x7f05f01032f0 2024-03-04T08:55:51.337 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.336+0000 7f05ef7fe700 1 --2- 172.21.15.158:0/2309619309 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f05dc0a1e10 0x7f05dc0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:51.337 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.337+0000 7f05ef7fe700 1 --2- 172.21.15.158:0/2309619309 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f05dc0a1e10 0x7f05dc0a42d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f05f0103150 tx=0x7f05e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:55:51.337 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.340+0000 7f05ed7fa700 1 -- 172.21.15.158:0/2309619309 <== mon.1 v2:172.21.15.158:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f05d809ad10 con 0x7f05f01032f0 2024-03-04T08:55:51.546 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.550+0000 7f05f61ac700 1 -- 172.21.15.158:0/2309619309 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi071=a;smithi071=d;smithi071=g;smithi071=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi158=c;smithi158=f;smithi158=i;smithi158=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f05f00627c0 con 0x7f05dc0a1e10 2024-03-04T08:55:51.553 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.558+0000 7f05ed7fa700 1 -- 172.21.15.158:0/2309619309 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f05f00627c0 con 0x7f05dc0a1e10 2024-03-04T08:55:51.554 INFO:teuthology.orchestra.run.smithi158.stdout:Scheduled mds.all update... 2024-03-04T08:55:51.556 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.560+0000 7f05f61ac700 1 -- 172.21.15.158:0/2309619309 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f05dc0a1e10 msgr2=0x7f05dc0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:51.556 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.560+0000 7f05f61ac700 1 --2- 172.21.15.158:0/2309619309 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f05dc0a1e10 0x7f05dc0a42d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f05f0103150 tx=0x7f05e0009260 comp rx=0 tx=0).stop 2024-03-04T08:55:51.556 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.560+0000 7f05f61ac700 1 -- 172.21.15.158:0/2309619309 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05f01032f0 msgr2=0x7f05f01987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:51.556 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.560+0000 7f05f61ac700 1 --2- 172.21.15.158:0/2309619309 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05f01032f0 0x7f05f01987c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f05d800d980 tx=0x7f05d800dc90 comp rx=0 tx=0).stop 2024-03-04T08:55:51.556 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.560+0000 7f05f61ac700 1 -- 172.21.15.158:0/2309619309 shutdown_connections 2024-03-04T08:55:51.556 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.560+0000 7f05f61ac700 1 --2- 172.21.15.158:0/2309619309 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f05dc0a1e10 0x7f05dc0a42d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:51.556 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.560+0000 7f05f61ac700 1 --2- 172.21.15.158:0/2309619309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05f0103c90 0x7f05f019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:51.557 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.560+0000 7f05f61ac700 1 --2- 172.21.15.158:0/2309619309 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05f01020f0 0x7f05f0198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:51.557 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.560+0000 7f05f61ac700 1 --2- 172.21.15.158:0/2309619309 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f05f01032f0 0x7f05f01987c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:51.557 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.560+0000 7f05f61ac700 1 -- 172.21.15.158:0/2309619309 >> 172.21.15.158:0/2309619309 conn(0x7f05f00fd6d0 msgr2=0x7f05f0106590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:55:51.557 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.560+0000 7f05f61ac700 1 -- 172.21.15.158:0/2309619309 shutdown_connections 2024-03-04T08:55:51.557 INFO:teuthology.orchestra.run.smithi158.stderr:2024-03-04T08:55:51.560+0000 7f05f61ac700 1 -- 172.21.15.158:0/2309619309 wait complete. 2024-03-04T08:55:51.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:51 smithi150 ceph-mon[36807]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:51.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:51 smithi158 ceph-mon[37241]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:51.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:51 smithi071 ceph-mon[32789]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:52.351 DEBUG:teuthology.orchestra.run.smithi071:mds.a> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.a.service 2024-03-04T08:55:52.354 DEBUG:teuthology.orchestra.run.smithi071:mds.d> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.d.service 2024-03-04T08:55:52.397 DEBUG:teuthology.orchestra.run.smithi071:mds.g> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.g.service 2024-03-04T08:55:52.440 DEBUG:teuthology.orchestra.run.smithi071:mds.j> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.j.service 2024-03-04T08:55:52.484 DEBUG:teuthology.orchestra.run.smithi150:mds.b> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.b.service 2024-03-04T08:55:52.487 DEBUG:teuthology.orchestra.run.smithi150:mds.e> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.e.service 2024-03-04T08:55:52.531 DEBUG:teuthology.orchestra.run.smithi150:mds.h> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.h.service 2024-03-04T08:55:52.575 DEBUG:teuthology.orchestra.run.smithi150:mds.k> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.k.service 2024-03-04T08:55:52.589 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:52 smithi158 ceph-mon[37241]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi071=a;smithi071=d;smithi071=g;smithi071=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi158=c;smithi158=f;smithi158=i;smithi158=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:55:52.589 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:52 smithi158 ceph-mon[37241]: Saving service mds.all spec with placement smithi071=a;smithi071=d;smithi071=g;smithi071=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi158=c;smithi158=f;smithi158=i;smithi158=l;count:12 2024-03-04T08:55:52.589 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:52 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:52.589 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:52 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:55:52.589 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:52 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:52.589 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:52 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:55:52.589 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:52 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:52.589 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:52 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:55:52.589 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:52 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:55:52.589 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:52 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:52.590 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:52 smithi158 ceph-mon[37241]: Deploying daemon mds.l on smithi158 2024-03-04T08:55:52.618 DEBUG:teuthology.orchestra.run.smithi158:mds.c> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.c.service 2024-03-04T08:55:52.620 DEBUG:teuthology.orchestra.run.smithi158:mds.f> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.f.service 2024-03-04T08:55:52.664 DEBUG:teuthology.orchestra.run.smithi158:mds.i> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.i.service 2024-03-04T08:55:52.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:52 smithi071 ceph-mon[32789]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi071=a;smithi071=d;smithi071=g;smithi071=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi158=c;smithi158=f;smithi158=i;smithi158=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:55:52.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:52 smithi071 ceph-mon[32789]: Saving service mds.all spec with placement smithi071=a;smithi071=d;smithi071=g;smithi071=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi158=c;smithi158=f;smithi158=i;smithi158=l;count:12 2024-03-04T08:55:52.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:52 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:52.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:52 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:55:52.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:52 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:52.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:52 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:55:52.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:52 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:52.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:52 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:55:52.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:52 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:55:52.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:52 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:52.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:52 smithi071 ceph-mon[32789]: Deploying daemon mds.l on smithi158 2024-03-04T08:55:52.708 DEBUG:teuthology.orchestra.run.smithi158:mds.l> sudo journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.l.service 2024-03-04T08:55:52.750 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-04T08:55:52.751 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T08:55:52.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:52 smithi150 ceph-mon[36807]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi071=a;smithi071=d;smithi071=g;smithi071=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi158=c;smithi158=f;smithi158=i;smithi158=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:55:52.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:52 smithi150 ceph-mon[36807]: Saving service mds.all spec with placement smithi071=a;smithi071=d;smithi071=g;smithi071=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi158=c;smithi158=f;smithi158=i;smithi158=l;count:12 2024-03-04T08:55:52.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:52 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:52.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:52 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:55:52.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:52 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:52.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:52 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:55:52.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:52 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:52.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:52 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:55:52.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:52 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:55:52.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:52 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:52.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:52 smithi150 ceph-mon[36807]: Deploying daemon mds.l on smithi158 2024-03-04T08:55:52.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.866+0000 7f6e0fed4640 1 -- 172.21.15.71:0/1686429657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e08075610 msgr2=0x7f6e08073a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:52.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.866+0000 7f6e0fed4640 1 --2- 172.21.15.71:0/1686429657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e08075610 0x7f6e08073a60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009930 tx=0x7f6e0002bc30 comp rx=0 tx=0).stop 2024-03-04T08:55:52.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.866+0000 7f6e0fed4640 1 -- 172.21.15.71:0/1686429657 shutdown_connections 2024-03-04T08:55:52.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.866+0000 7f6e0fed4640 1 --2- 172.21.15.71:0/1686429657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e0810dbd0 0x7f6e0810ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:52.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.866+0000 7f6e0fed4640 1 --2- 172.21.15.71:0/1686429657 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e08074030 0x7f6e080744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:52.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.866+0000 7f6e0fed4640 1 --2- 172.21.15.71:0/1686429657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e08075610 0x7f6e08073a60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:52.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.866+0000 7f6e0fed4640 1 -- 172.21.15.71:0/1686429657 >> 172.21.15.71:0/1686429657 conn(0x7f6e080fcbb0 msgr2=0x7f6e080fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:55:52.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.866+0000 7f6e0fed4640 1 -- 172.21.15.71:0/1686429657 shutdown_connections 2024-03-04T08:55:52.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.866+0000 7f6e0fed4640 1 -- 172.21.15.71:0/1686429657 wait complete. 2024-03-04T08:55:52.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.867+0000 7f6e0fed4640 1 Processor -- start 2024-03-04T08:55:52.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.867+0000 7f6e0fed4640 1 -- start start 2024-03-04T08:55:52.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.867+0000 7f6e0fed4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e08074030 0x7f6e0819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:52.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.867+0000 7f6e0fed4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e08075610 0x7f6e0819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:52.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.867+0000 7f6e0fed4640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e0810dbd0 0x7f6e081a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:52.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.867+0000 7f6e0fed4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e08113850 con 0x7f6e08074030 2024-03-04T08:55:52.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.867+0000 7f6e0fed4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e081136d0 con 0x7f6e08075610 2024-03-04T08:55:52.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.867+0000 7f6e0fed4640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e081139d0 con 0x7f6e0810dbd0 2024-03-04T08:55:52.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.867+0000 7f6e0dc70640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e08074030 0x7f6e0819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:52.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.867+0000 7f6e0dc70640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e08074030 0x7f6e0819df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35094/0 (socket says 172.21.15.71:35094) 2024-03-04T08:55:52.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.867+0000 7f6e0dc70640 1 -- 172.21.15.71:0/1151687915 learned_addr learned my addr 172.21.15.71:0/1151687915 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:55:52.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.867+0000 7f6e0e471640 1 --2- 172.21.15.71:0/1151687915 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e0810dbd0 0x7f6e081a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:52.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.867+0000 7f6e0d46f640 1 --2- 172.21.15.71:0/1151687915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e08075610 0x7f6e0819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:52.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.867+0000 7f6e0dc70640 1 -- 172.21.15.71:0/1151687915 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e0810dbd0 msgr2=0x7f6e081a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:52.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.867+0000 7f6e0dc70640 1 --2- 172.21.15.71:0/1151687915 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e0810dbd0 0x7f6e081a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:52.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.867+0000 7f6e0dc70640 1 -- 172.21.15.71:0/1151687915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e08075610 msgr2=0x7f6e0819e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:52.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.867+0000 7f6e0dc70640 1 --2- 172.21.15.71:0/1151687915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e08075610 0x7f6e0819e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:52.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.867+0000 7f6e0dc70640 1 -- 172.21.15.71:0/1151687915 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e081a5b70 con 0x7f6e08074030 2024-03-04T08:55:52.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.867+0000 7f6e0d46f640 1 --2- 172.21.15.71:0/1151687915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e08075610 0x7f6e0819e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:55:52.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.868+0000 7f6e0dc70640 1 --2- 172.21.15.71:0/1151687915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e08074030 0x7f6e0819df50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6e00005ab0 tx=0x7f6e00005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:55:52.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.868+0000 7f6dfeffd640 1 -- 172.21.15.71:0/1151687915 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e00041070 con 0x7f6e08074030 2024-03-04T08:55:52.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.868+0000 7f6dfeffd640 1 -- 172.21.15.71:0/1151687915 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e000054e0 con 0x7f6e08074030 2024-03-04T08:55:52.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.868+0000 7f6e0fed4640 1 -- 172.21.15.71:0/1151687915 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e081a5e00 con 0x7f6e08074030 2024-03-04T08:55:52.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.868+0000 7f6dfeffd640 1 -- 172.21.15.71:0/1151687915 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e00034440 con 0x7f6e08074030 2024-03-04T08:55:52.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.868+0000 7f6e0fed4640 1 -- 172.21.15.71:0/1151687915 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e081a6210 con 0x7f6e08074030 2024-03-04T08:55:52.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.869+0000 7f6e0fed4640 1 -- 172.21.15.71:0/1151687915 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e08103da0 con 0x7f6e08074030 2024-03-04T08:55:52.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.870+0000 7f6dfeffd640 1 -- 172.21.15.71:0/1151687915 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f6e00044070 con 0x7f6e08074030 2024-03-04T08:55:52.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.870+0000 7f6dfeffd640 1 --2- 172.21.15.71:0/1151687915 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6df40a1d80 0x7f6df40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:52.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.870+0000 7f6e0d46f640 1 --2- 172.21.15.71:0/1151687915 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6df40a1d80 0x7f6df40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:52.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.870+0000 7f6dfeffd640 1 -- 172.21.15.71:0/1151687915 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6e000ef570 con 0x7f6e08074030 2024-03-04T08:55:52.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.871+0000 7f6e0d46f640 1 --2- 172.21.15.71:0/1151687915 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6df40a1d80 0x7f6df40a4240 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6df80058c0 tx=0x7f6df8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:55:52.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:52.875+0000 7f6dfeffd640 1 -- 172.21.15.71:0/1151687915 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6e000bce40 con 0x7f6e08074030 2024-03-04T08:55:53.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.050+0000 7f6e0fed4640 1 -- 172.21.15.71:0/1151687915 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e0802d170 con 0x7f6e08074030 2024-03-04T08:55:53.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.051+0000 7f6dfeffd640 1 -- 172.21.15.71:0/1151687915 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+18481 (secure 0 0 0) 0x7f6e000bc740 con 0x7f6e08074030 2024-03-04T08:55:53.048 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:55:53.048 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":83,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","created":"2024-03-04T08:47:28.546102+0000","modified":"2024-03-04T08:55:31.753527+0000","last_up_change":"2024-03-04T08:55:30.744961+0000","last_in_change":"2024-03-04T08:55:11.570990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T08:51:14.933487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e43b47d-8888-4631-bc96-8a96fcf8cd46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6803","nonce":1173641530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6805","nonce":1173641530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6809","nonce":1173641530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6807","nonce":1173641530}]},"public_addr":"172.21.15.71:6803/1173641530","cluster_addr":"172.21.15.71:6805/1173641530","heartbeat_back_addr":"172.21.15.71:6809/1173641530","heartbeat_front_addr":"172.21.15.71:6807/1173641530","state":["exists","up"]},{"osd":1,"uuid":"1209f214-f2e2-4209-bf69-4528a43a2123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6801","nonce":3383948110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6803","nonce":3383948110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6807","nonce":3383948110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6805","nonce":3383948110}]},"public_addr":"172.21.15.150:6801/3383948110","cluster_addr":"172.21.15.150:6803/3383948110","heartbeat_back_addr":"172.21.15.150:6807/3383948110","heartbeat_front_addr":"172.21.15.150:6805/3383948110","state":["exists","up"]},{"osd":2,"uuid":"b88c3082-93a9-4c9b-9221-a7105a44c91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6801","nonce":586944144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6803","nonce":586944144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6807","nonce":586944144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6805","nonce":586944144}]},"public_addr":"172.21.15.158:6801/586944144","cluster_addr":"172.21.15.158:6803/586944144","heartbeat_back_addr":"172.21.15.158:6807/586944144","heartbeat_front_addr":"172.21.15.158:6805/586944144","state":["exists","up"]},{"osd":3,"uuid":"e4fc7723-fd73-40b2-b9d0-7f9bda282ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6811","nonce":1311903262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6813","nonce":1311903262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6817","nonce":1311903262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6815","nonce":1311903262}]},"public_addr":"172.21.15.71:6811/1311903262","cluster_addr":"172.21.15.71:6813/1311903262","heartbeat_back_addr":"172.21.15.71:6817/1311903262","heartbeat_front_addr":"172.21.15.71:6815/1311903262","state":["exists","up"]},{"osd":4,"uuid":"e7eb72ac-f066-4ccd-b2bf-22a6a26daf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6809","nonce":2300837573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6811","nonce":2300837573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6815","nonce":2300837573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6813","nonce":2300837573}]},"public_addr":"172.21.15.150:6809/2300837573","cluster_addr":"172.21.15.150:6811/2300837573","heartbeat_back_addr":"172.21.15.150:6815/2300837573","heartbeat_front_addr":"172.21.15.150:6813/2300837573","state":["exists","up"]},{"osd":5,"uuid":"9b295840-e104-4a66-9cc4-7020627c1765","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6809","nonce":1922694045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6811","nonce":1922694045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6815","nonce":1922694045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6813","nonce":1922694045}]},"public_addr":"172.21.15.158:6809/1922694045","cluster_addr":"172.21.15.158:6811/1922694045","heartbeat_back_addr":"172.21.15.158:6815/1922694045","heartbeat_front_addr":"172.21.15.158:6813/1922694045","state":["exists","up"]},{"osd":6,"uuid":"6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6819","nonce":3848842707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6821","nonce":3848842707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6825","nonce":3848842707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6823","nonce":3848842707}]},"public_addr":"172.21.15.71:6819/3848842707","cluster_addr":"172.21.15.71:6821/3848842707","heartbeat_back_addr":"172.21.15.71:6825/3848842707","heartbeat_front_addr":"172.21.15.71:6823/3848842707","state":["exists","up"]},{"osd":7,"uuid":"6160a40a-d30f-4569-949c-0afb1bb3d133","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6817","nonce":1328330305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6819","nonce":1328330305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6823","nonce":1328330305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6821","nonce":1328330305}]},"public_addr":"172.21.15.150:6817/1328330305","cluster_addr":"172.21.15.150:6819/1328330305","heartbeat_back_addr":"172.21.15.150:6823/1328330305","heartbeat_front_addr":"172.21.15.150:6821/1328330305","state":["exists","up"]},{"osd":8,"uuid":"95ed2341-84a4-465c-9948-ac667680ce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6817","nonce":2195304577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6819","nonce":2195304577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6823","nonce":2195304577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6821","nonce":2195304577}]},"public_addr":"172.21.15.158:6817/2195304577","cluster_addr":"172.21.15.158:6819/2195304577","heartbeat_back_addr":"172.21.15.158:6823/2195304577","heartbeat_front_addr":"172.21.15.158:6821/2195304577","state":["exists","up"]},{"osd":9,"uuid":"088f6419-63a0-4768-afae-dbb8a5f5a565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6827","nonce":1913779347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6829","nonce":1913779347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6833","nonce":1913779347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6831","nonce":1913779347}]},"public_addr":"172.21.15.71:6827/1913779347","cluster_addr":"172.21.15.71:6829/1913779347","heartbeat_back_addr":"172.21.15.71:6833/1913779347","heartbeat_front_addr":"172.21.15.71:6831/1913779347","state":["exists","up"]},{"osd":10,"uuid":"18c85e24-068d-4f1e-a3b8-f7915b363234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6825","nonce":3295420996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6827","nonce":3295420996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6831","nonce":3295420996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6829","nonce":3295420996}]},"public_addr":"172.21.15.150:6825/3295420996","cluster_addr":"172.21.15.150:6827/3295420996","heartbeat_back_addr":"172.21.15.150:6831/3295420996","heartbeat_front_addr":"172.21.15.150:6829/3295420996","state":["exists","up"]},{"osd":11,"uuid":"4f6b3ddf-c482-4740-b1fd-de836e047c2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6825","nonce":3332083865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6827","nonce":3332083865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6831","nonce":3332083865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6829","nonce":3332083865}]},"public_addr":"172.21.15.158:6825/3332083865","cluster_addr":"172.21.15.158:6827/3332083865","heartbeat_back_addr":"172.21.15.158:6831/3332083865","heartbeat_front_addr":"172.21.15.158:6829/3332083865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:05.997832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:29.314235+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:08.819465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:41.194275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:06.032479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:38.814498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:13.067960+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:38.568479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:03.416713+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:38.349479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:04.096945+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:29.425250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1287712358":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/3770452860":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/718743094":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/1904717532":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:6800/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6800/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/2862184313":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/3621754630":"2024-03-05T08:48:22.593931+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T08:55:53.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.053+0000 7f6e0fed4640 1 -- 172.21.15.71:0/1151687915 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6df40a1d80 msgr2=0x7f6df40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:53.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.053+0000 7f6e0fed4640 1 --2- 172.21.15.71:0/1151687915 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6df40a1d80 0x7f6df40a4240 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6df80058c0 tx=0x7f6df8005850 comp rx=0 tx=0).stop 2024-03-04T08:55:53.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.053+0000 7f6e0fed4640 1 -- 172.21.15.71:0/1151687915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e08074030 msgr2=0x7f6e0819df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:53.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.053+0000 7f6e0fed4640 1 --2- 172.21.15.71:0/1151687915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e08074030 0x7f6e0819df50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6e00005ab0 tx=0x7f6e00005dc0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.053+0000 7f6e0fed4640 1 -- 172.21.15.71:0/1151687915 shutdown_connections 2024-03-04T08:55:53.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.053+0000 7f6e0fed4640 1 --2- 172.21.15.71:0/1151687915 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6df40a1d80 0x7f6df40a4240 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.053+0000 7f6e0fed4640 1 --2- 172.21.15.71:0/1151687915 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6e0810dbd0 0x7f6e081a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.053+0000 7f6e0fed4640 1 --2- 172.21.15.71:0/1151687915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e08075610 0x7f6e0819e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.053+0000 7f6e0fed4640 1 --2- 172.21.15.71:0/1151687915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e08074030 0x7f6e0819df50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.053+0000 7f6e0fed4640 1 -- 172.21.15.71:0/1151687915 >> 172.21.15.71:0/1151687915 conn(0x7f6e080fcbb0 msgr2=0x7f6e08100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:55:53.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.054+0000 7f6e0fed4640 1 -- 172.21.15.71:0/1151687915 shutdown_connections 2024-03-04T08:55:53.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.054+0000 7f6e0fed4640 1 -- 172.21.15.71:0/1151687915 wait complete. 2024-03-04T08:55:53.061 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T08:51:14.933487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-04T08:55:53.061 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T08:55:53.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.170+0000 7fc8b5239640 1 -- 172.21.15.71:0/806055887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8b01004d0 msgr2=0x7fc8b01008f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:53.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.170+0000 7fc8b5239640 1 --2- 172.21.15.71:0/806055887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8b01004d0 0x7fc8b01008f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009900 tx=0x7fc8a002bc30 comp rx=0 tx=0).stop 2024-03-04T08:55:53.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.170+0000 7fc8b5239640 1 -- 172.21.15.71:0/806055887 shutdown_connections 2024-03-04T08:55:53.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.170+0000 7fc8b5239640 1 --2- 172.21.15.71:0/806055887 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8b0102080 0x7fc8b0108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.170+0000 7fc8b5239640 1 --2- 172.21.15.71:0/806055887 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8b01016c0 0x7fc8b0101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.170+0000 7fc8b5239640 1 --2- 172.21.15.71:0/806055887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8b01004d0 0x7fc8b01008f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.170+0000 7fc8b5239640 1 -- 172.21.15.71:0/806055887 >> 172.21.15.71:0/806055887 conn(0x7fc8b00fba40 msgr2=0x7fc8b00fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:55:53.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.170+0000 7fc8b5239640 1 -- 172.21.15.71:0/806055887 shutdown_connections 2024-03-04T08:55:53.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.170+0000 7fc8b5239640 1 -- 172.21.15.71:0/806055887 wait complete. 2024-03-04T08:55:53.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.171+0000 7fc8b5239640 1 Processor -- start 2024-03-04T08:55:53.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.171+0000 7fc8b5239640 1 -- start start 2024-03-04T08:55:53.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.171+0000 7fc8b5239640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8b01004d0 0x7fc8b006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:53.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.171+0000 7fc8b5239640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8b01016c0 0x7fc8b019c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:53.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.171+0000 7fc8b5239640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8b0102080 0x7fc8b019ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:53.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.171+0000 7fc8b5239640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b010c160 con 0x7fc8b0102080 2024-03-04T08:55:53.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.171+0000 7fc8b5239640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b010bfe0 con 0x7fc8b01004d0 2024-03-04T08:55:53.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.171+0000 7fc8b5239640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b010c2e0 con 0x7fc8b01016c0 2024-03-04T08:55:53.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.171+0000 7fc8af59e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8b0102080 0x7fc8b019ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:53.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.171+0000 7fc8af59e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8b0102080 0x7fc8b019ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35124/0 (socket says 172.21.15.71:35124) 2024-03-04T08:55:53.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.171+0000 7fc8af59e640 1 -- 172.21.15.71:0/1487406560 learned_addr learned my addr 172.21.15.71:0/1487406560 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:55:53.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.171+0000 7fc8a7fff640 1 --2- 172.21.15.71:0/1487406560 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8b01016c0 0x7fc8b019c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:53.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.171+0000 7fc8aed9d640 1 --2- 172.21.15.71:0/1487406560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8b01004d0 0x7fc8b006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:53.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.171+0000 7fc8af59e640 1 -- 172.21.15.71:0/1487406560 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8b01016c0 msgr2=0x7fc8b019c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:53.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.171+0000 7fc8af59e640 1 --2- 172.21.15.71:0/1487406560 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8b01016c0 0x7fc8b019c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.171+0000 7fc8af59e640 1 -- 172.21.15.71:0/1487406560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8b01004d0 msgr2=0x7fc8b006a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:53.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.171+0000 7fc8af59e640 1 --2- 172.21.15.71:0/1487406560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8b01004d0 0x7fc8b006a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.171+0000 7fc8af59e640 1 -- 172.21.15.71:0/1487406560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b019f5c0 con 0x7fc8b0102080 2024-03-04T08:55:53.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.171+0000 7fc8af59e640 1 --2- 172.21.15.71:0/1487406560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8b0102080 0x7fc8b019ecb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800cad0 tx=0x7fc8a800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:55:53.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.172+0000 7fc8acd99640 1 -- 172.21.15.71:0/1487406560 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a800ae20 con 0x7fc8b0102080 2024-03-04T08:55:53.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.172+0000 7fc8acd99640 1 -- 172.21.15.71:0/1487406560 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8a8005020 con 0x7fc8b0102080 2024-03-04T08:55:53.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.172+0000 7fc8b5239640 1 -- 172.21.15.71:0/1487406560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b019f850 con 0x7fc8b0102080 2024-03-04T08:55:53.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.172+0000 7fc8acd99640 1 -- 172.21.15.71:0/1487406560 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a800f6f0 con 0x7fc8b0102080 2024-03-04T08:55:53.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.172+0000 7fc8b5239640 1 -- 172.21.15.71:0/1487406560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b019fda0 con 0x7fc8b0102080 2024-03-04T08:55:53.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.173+0000 7fc8b5239640 1 -- 172.21.15.71:0/1487406560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b01078c0 con 0x7fc8b0102080 2024-03-04T08:55:53.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.174+0000 7fc8acd99640 1 -- 172.21.15.71:0/1487406560 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fc8a80051d0 con 0x7fc8b0102080 2024-03-04T08:55:53.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.174+0000 7fc8acd99640 1 --2- 172.21.15.71:0/1487406560 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc89c0a1d80 0x7fc89c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:53.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.174+0000 7fc8aed9d640 1 --2- 172.21.15.71:0/1487406560 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc89c0a1d80 0x7fc89c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:53.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.174+0000 7fc8acd99640 1 -- 172.21.15.71:0/1487406560 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc8a80cdc70 con 0x7fc8b0102080 2024-03-04T08:55:53.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.175+0000 7fc8aed9d640 1 --2- 172.21.15.71:0/1487406560 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc89c0a1d80 0x7fc89c0a4240 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc8a002c7b0 tx=0x7fc8a0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:55:53.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.179+0000 7fc8acd99640 1 -- 172.21.15.71:0/1487406560 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc8a8014030 con 0x7fc8b0102080 2024-03-04T08:55:53.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.353+0000 7fc8b5239640 1 -- 172.21.15.71:0/1487406560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc8b002d1f0 con 0x7fc8b0102080 2024-03-04T08:55:53.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.353+0000 7fc8acd99640 1 -- 172.21.15.71:0/1487406560 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc8a809b540 con 0x7fc8b0102080 2024-03-04T08:55:53.350 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-03-04T08:55:53.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.355+0000 7fc8b5239640 1 -- 172.21.15.71:0/1487406560 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc89c0a1d80 msgr2=0x7fc89c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:53.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.355+0000 7fc8b5239640 1 --2- 172.21.15.71:0/1487406560 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc89c0a1d80 0x7fc89c0a4240 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc8a002c7b0 tx=0x7fc8a0005fd0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.355+0000 7fc8b5239640 1 -- 172.21.15.71:0/1487406560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8b0102080 msgr2=0x7fc8b019ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:53.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.355+0000 7fc8b5239640 1 --2- 172.21.15.71:0/1487406560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8b0102080 0x7fc8b019ecb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800cad0 tx=0x7fc8a800cf90 comp rx=0 tx=0).stop 2024-03-04T08:55:53.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.355+0000 7fc8b5239640 1 -- 172.21.15.71:0/1487406560 shutdown_connections 2024-03-04T08:55:53.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.355+0000 7fc8b5239640 1 --2- 172.21.15.71:0/1487406560 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc89c0a1d80 0x7fc89c0a4240 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.355+0000 7fc8b5239640 1 --2- 172.21.15.71:0/1487406560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8b0102080 0x7fc8b019ecb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.355+0000 7fc8b5239640 1 --2- 172.21.15.71:0/1487406560 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc8b01016c0 0x7fc8b019c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.355+0000 7fc8b5239640 1 --2- 172.21.15.71:0/1487406560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8b01004d0 0x7fc8b006a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.355+0000 7fc8b5239640 1 -- 172.21.15.71:0/1487406560 >> 172.21.15.71:0/1487406560 conn(0x7fc8b00fba40 msgr2=0x7fc8b0104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:55:53.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.355+0000 7fc8b5239640 1 -- 172.21.15.71:0/1487406560 shutdown_connections 2024-03-04T08:55:53.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.355+0000 7fc8b5239640 1 -- 172.21.15.71:0/1487406560 wait complete. 2024-03-04T08:55:53.363 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:55:53.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.469+0000 7fd2c5f56640 1 -- 172.21.15.71:0/2298590363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2c0102090 msgr2=0x7fd2c0104480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:53.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.469+0000 7fd2c5f56640 1 --2- 172.21.15.71:0/2298590363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2c0102090 0x7fd2c0104480 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0009900 tx=0x7fd2b002bc30 comp rx=0 tx=0).stop 2024-03-04T08:55:53.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.469+0000 7fd2c5f56640 1 -- 172.21.15.71:0/2298590363 shutdown_connections 2024-03-04T08:55:53.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.469+0000 7fd2c5f56640 1 --2- 172.21.15.71:0/2298590363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2c01072f0 0x7fd2c0109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.469+0000 7fd2c5f56640 1 --2- 172.21.15.71:0/2298590363 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2c01049c0 0x7fd2c0106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.469+0000 7fd2c5f56640 1 --2- 172.21.15.71:0/2298590363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2c0102090 0x7fd2c0104480 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.469+0000 7fd2c5f56640 1 -- 172.21.15.71:0/2298590363 >> 172.21.15.71:0/2298590363 conn(0x7fd2c00fba80 msgr2=0x7fd2c00fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:55:53.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.469+0000 7fd2c5f56640 1 -- 172.21.15.71:0/2298590363 shutdown_connections 2024-03-04T08:55:53.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.469+0000 7fd2c5f56640 1 -- 172.21.15.71:0/2298590363 wait complete. 2024-03-04T08:55:53.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.470+0000 7fd2c5f56640 1 Processor -- start 2024-03-04T08:55:53.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.470+0000 7fd2c5f56640 1 -- start start 2024-03-04T08:55:53.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.470+0000 7fd2c5f56640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2c0102090 0x7fd2c0101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:53.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.470+0000 7fd2c5f56640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2c01049c0 0x7fd2c0100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:53.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.470+0000 7fd2c5f56640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2c01072f0 0x7fd2c0100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:53.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.470+0000 7fd2c5f56640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c010d2a0 con 0x7fd2c01049c0 2024-03-04T08:55:53.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.470+0000 7fd2c5f56640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c010d120 con 0x7fd2c01072f0 2024-03-04T08:55:53.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.470+0000 7fd2c5f56640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c010d420 con 0x7fd2c0102090 2024-03-04T08:55:53.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.470+0000 7fd2beffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2c01049c0 0x7fd2c0100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:53.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.470+0000 7fd2beffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2c01049c0 0x7fd2c0100040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35150/0 (socket says 172.21.15.71:35150) 2024-03-04T08:55:53.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.470+0000 7fd2beffd640 1 -- 172.21.15.71:0/2813360047 learned_addr learned my addr 172.21.15.71:0/2813360047 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:55:53.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.470+0000 7fd2bffff640 1 --2- 172.21.15.71:0/2813360047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2c01072f0 0x7fd2c0100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:53.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.470+0000 7fd2beffd640 1 -- 172.21.15.71:0/2813360047 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2c0102090 msgr2=0x7fd2c0101a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:53.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.470+0000 7fd2bf7fe640 1 --2- 172.21.15.71:0/2813360047 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2c0102090 0x7fd2c0101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:53.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.470+0000 7fd2beffd640 1 --2- 172.21.15.71:0/2813360047 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2c0102090 0x7fd2c0101a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.470+0000 7fd2beffd640 1 -- 172.21.15.71:0/2813360047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2c01072f0 msgr2=0x7fd2c0100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:53.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.470+0000 7fd2beffd640 1 --2- 172.21.15.71:0/2813360047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2c01072f0 0x7fd2c0100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.470+0000 7fd2beffd640 1 -- 172.21.15.71:0/2813360047 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c0100e10 con 0x7fd2c01049c0 2024-03-04T08:55:53.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.471+0000 7fd2bf7fe640 1 --2- 172.21.15.71:0/2813360047 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2c0102090 0x7fd2c0101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:55:53.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.471+0000 7fd2beffd640 1 --2- 172.21.15.71:0/2813360047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2c01049c0 0x7fd2c0100040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd2b400d740 tx=0x7fd2b400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:55:53.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.471+0000 7fd2bcff9640 1 -- 172.21.15.71:0/2813360047 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4017070 con 0x7fd2c01049c0 2024-03-04T08:55:53.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.471+0000 7fd2bcff9640 1 -- 172.21.15.71:0/2813360047 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2b400de50 con 0x7fd2c01049c0 2024-03-04T08:55:53.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.471+0000 7fd2c5f56640 1 -- 172.21.15.71:0/2813360047 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c01a1aa0 con 0x7fd2c01049c0 2024-03-04T08:55:53.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.471+0000 7fd2bcff9640 1 -- 172.21.15.71:0/2813360047 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4004e00 con 0x7fd2c01049c0 2024-03-04T08:55:53.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.471+0000 7fd2c5f56640 1 -- 172.21.15.71:0/2813360047 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c01a1fa0 con 0x7fd2c01049c0 2024-03-04T08:55:53.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.472+0000 7fd2c5f56640 1 -- 172.21.15.71:0/2813360047 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c00fd570 con 0x7fd2c01049c0 2024-03-04T08:55:53.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.473+0000 7fd2bcff9640 1 -- 172.21.15.71:0/2813360047 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fd2b4004fa0 con 0x7fd2c01049c0 2024-03-04T08:55:53.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.473+0000 7fd2bcff9640 1 --2- 172.21.15.71:0/2813360047 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd2a80a1d80 0x7fd2a80a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:53.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.474+0000 7fd2bf7fe640 1 --2- 172.21.15.71:0/2813360047 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd2a80a1d80 0x7fd2a80a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:53.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.474+0000 7fd2bcff9640 1 -- 172.21.15.71:0/2813360047 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd2b40cd2b0 con 0x7fd2c01049c0 2024-03-04T08:55:53.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.474+0000 7fd2bf7fe640 1 --2- 172.21.15.71:0/2813360047 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd2a80a1d80 0x7fd2a80a4240 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd2b00096c0 tx=0x7fd2b0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:55:53.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.478+0000 7fd2bcff9640 1 -- 172.21.15.71:0/2813360047 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd2b409ab80 con 0x7fd2c01049c0 2024-03-04T08:55:53.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:53 smithi071 ceph-mon[32789]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:53.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:53 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1151687915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:55:53.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:53 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1487406560' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:55:53.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.691+0000 7fd2c5f56640 1 -- 172.21.15.71:0/2813360047 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2c002d200 con 0x7fd2c01049c0 2024-03-04T08:55:53.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.692+0000 7fd2bcff9640 1 -- 172.21.15.71:0/2813360047 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7fd2b409a480 con 0x7fd2c01049c0 2024-03-04T08:55:53.688 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:55:53.688 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-04T08:55:53.688 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 1 2024-03-04T08:55:53.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.693+0000 7fd2c5f56640 1 -- 172.21.15.71:0/2813360047 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd2a80a1d80 msgr2=0x7fd2a80a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:53.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.693+0000 7fd2c5f56640 1 --2- 172.21.15.71:0/2813360047 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd2a80a1d80 0x7fd2a80a4240 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd2b00096c0 tx=0x7fd2b0000bc0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.693+0000 7fd2c5f56640 1 -- 172.21.15.71:0/2813360047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2c01049c0 msgr2=0x7fd2c0100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:53.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.693+0000 7fd2c5f56640 1 --2- 172.21.15.71:0/2813360047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2c01049c0 0x7fd2c0100040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd2b400d740 tx=0x7fd2b400dc00 comp rx=0 tx=0).stop 2024-03-04T08:55:53.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.694+0000 7fd2c5f56640 1 -- 172.21.15.71:0/2813360047 shutdown_connections 2024-03-04T08:55:53.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.694+0000 7fd2c5f56640 1 --2- 172.21.15.71:0/2813360047 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd2a80a1d80 0x7fd2a80a4240 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.694+0000 7fd2c5f56640 1 --2- 172.21.15.71:0/2813360047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2c01072f0 0x7fd2c0100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.694+0000 7fd2c5f56640 1 --2- 172.21.15.71:0/2813360047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2c01049c0 0x7fd2c0100040 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.694+0000 7fd2c5f56640 1 --2- 172.21.15.71:0/2813360047 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd2c0102090 0x7fd2c0101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:53.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.694+0000 7fd2c5f56640 1 -- 172.21.15.71:0/2813360047 >> 172.21.15.71:0/2813360047 conn(0x7fd2c00fba80 msgr2=0x7fd2c01029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:55:53.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.694+0000 7fd2c5f56640 1 -- 172.21.15.71:0/2813360047 shutdown_connections 2024-03-04T08:55:53.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:53.694+0000 7fd2c5f56640 1 -- 172.21.15.71:0/2813360047 wait complete. 2024-03-04T08:55:53.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:53 smithi158 ceph-mon[37241]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:53.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:53 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1151687915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:55:53.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:53 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1487406560' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:55:54.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:53 smithi150 ceph-mon[36807]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:54.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:53 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1151687915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:55:54.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:53 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1487406560' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:55:54.241 INFO:journalctl@ceph.mds.l.smithi158.stdout:Mar 04 08:55:54 smithi158 systemd[1]: Starting Ceph mds.l for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T08:55:54.503 INFO:journalctl@ceph.mds.l.smithi158.stdout:Mar 04 08:55:54 smithi158 podman[67712]: 2024-03-04 08:55:54.245991712 +0000 UTC m=+0.112784526 container create d8193eec104a98d92ec701293a821d2cdc8b3d4c8513bed6bae1d1e870af7632 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux ) 2024-03-04T08:55:54.504 INFO:journalctl@ceph.mds.l.smithi158.stdout:Mar 04 08:55:54 smithi158 podman[67712]: 2024-03-04 08:55:54.150668662 +0000 UTC m=+0.017461502 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T08:55:54.504 INFO:journalctl@ceph.mds.l.smithi158.stdout:Mar 04 08:55:54 smithi158 podman[67712]: 2024-03-04 08:55:54.482301854 +0000 UTC m=+0.349094691 container init d8193eec104a98d92ec701293a821d2cdc8b3d4c8513bed6bae1d1e870af7632 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-l, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS) 2024-03-04T08:55:54.504 INFO:journalctl@ceph.mds.l.smithi158.stdout:Mar 04 08:55:54 smithi158 podman[67712]: 2024-03-04 08:55:54.485373623 +0000 UTC m=+0.352166448 container start d8193eec104a98d92ec701293a821d2cdc8b3d4c8513bed6bae1d1e870af7632 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-l, RELEASE=wip-lusov-qdb-1bc250c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-04T08:55:54.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:54 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2813360047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:55:54.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:54 smithi158 ceph-mon[37241]: mds.? [v2:172.21.15.158:6832/178179528,v1:172.21.15.158:6833/178179528] up:boot 2024-03-04T08:55:54.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:54 smithi158 ceph-mon[37241]: fsmap 1 up:standby 2024-03-04T08:55:54.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:54 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-04T08:55:54.803 INFO:journalctl@ceph.mds.l.smithi158.stdout:Mar 04 08:55:54 smithi158 ceph-mds[67731]: -- 172.21.15.158:0/1292573498 <== mon.1 v2:172.21.15.158:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e07ed05a40 con 0x55e07ecd8800 2024-03-04T08:55:54.803 INFO:journalctl@ceph.mds.l.smithi158.stdout:Mar 04 08:55:54 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-l[67727]: starting mds.l at 2024-03-04T08:55:54.803 INFO:journalctl@ceph.mds.l.smithi158.stdout:Mar 04 08:55:54 smithi158 bash[67712]: d8193eec104a98d92ec701293a821d2cdc8b3d4c8513bed6bae1d1e870af7632 2024-03-04T08:55:54.803 INFO:journalctl@ceph.mds.l.smithi158.stdout:Mar 04 08:55:54 smithi158 systemd[1]: Started Ceph mds.l for b3c85c40-da03-11ee-95c6-87774f69a715. 2024-03-04T08:55:54.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:54 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2813360047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:55:54.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:54 smithi071 ceph-mon[32789]: mds.? [v2:172.21.15.158:6832/178179528,v1:172.21.15.158:6833/178179528] up:boot 2024-03-04T08:55:54.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:54 smithi071 ceph-mon[32789]: fsmap 1 up:standby 2024-03-04T08:55:54.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:54 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-04T08:55:55.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:54 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2813360047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:55:55.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:54 smithi150 ceph-mon[36807]: mds.? [v2:172.21.15.158:6832/178179528,v1:172.21.15.158:6833/178179528] up:boot 2024-03-04T08:55:55.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:54 smithi150 ceph-mon[36807]: fsmap 1 up:standby 2024-03-04T08:55:55.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:54 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-04T08:55:55.702 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:55:55.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.819+0000 7f3d28c29640 1 -- 172.21.15.71:0/115636655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d24101610 msgr2=0x7f3d24101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:55.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.819+0000 7f3d28c29640 1 --2- 172.21.15.71:0/115636655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d24101610 0x7f3d24101a30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3d14009ca0 tx=0x7f3d1402c0f0 comp rx=0 tx=0).stop 2024-03-04T08:55:55.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.820+0000 7f3d28c29640 1 -- 172.21.15.71:0/115636655 shutdown_connections 2024-03-04T08:55:55.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.820+0000 7f3d28c29640 1 --2- 172.21.15.71:0/115636655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d241031c0 0x7f3d24109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:55.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.820+0000 7f3d28c29640 1 --2- 172.21.15.71:0/115636655 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d24102800 0x7f3d24102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:55.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.820+0000 7f3d28c29640 1 --2- 172.21.15.71:0/115636655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d24101610 0x7f3d24101a30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:55.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.820+0000 7f3d28c29640 1 -- 172.21.15.71:0/115636655 >> 172.21.15.71:0/115636655 conn(0x7f3d240fcb80 msgr2=0x7f3d240fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:55:55.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.820+0000 7f3d28c29640 1 -- 172.21.15.71:0/115636655 shutdown_connections 2024-03-04T08:55:55.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.820+0000 7f3d28c29640 1 -- 172.21.15.71:0/115636655 wait complete. 2024-03-04T08:55:55.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.820+0000 7f3d28c29640 1 Processor -- start 2024-03-04T08:55:55.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.820+0000 7f3d28c29640 1 -- start start 2024-03-04T08:55:55.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.821+0000 7f3d28c29640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d24101610 0x7f3d24197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:55.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.821+0000 7f3d28c29640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d24102800 0x7f3d24197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:55.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.821+0000 7f3d28c29640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d241031c0 0x7f3d2419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:55.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.821+0000 7f3d28c29640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2410d280 con 0x7f3d24101610 2024-03-04T08:55:55.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.821+0000 7f3d28c29640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2410d100 con 0x7f3d24102800 2024-03-04T08:55:55.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.821+0000 7f3d28c29640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2410d400 con 0x7f3d241031c0 2024-03-04T08:55:55.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.821+0000 7f3d2259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d24101610 0x7f3d24197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:55.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.821+0000 7f3d2259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d24101610 0x7f3d24197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48270/0 (socket says 172.21.15.71:48270) 2024-03-04T08:55:55.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.821+0000 7f3d2259c640 1 -- 172.21.15.71:0/1341642703 learned_addr learned my addr 172.21.15.71:0/1341642703 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:55:55.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.821+0000 7f3d21d9b640 1 --2- 172.21.15.71:0/1341642703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d24102800 0x7f3d24197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:55.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.821+0000 7f3d22d9d640 1 --2- 172.21.15.71:0/1341642703 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d241031c0 0x7f3d2419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:55.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.821+0000 7f3d2259c640 1 -- 172.21.15.71:0/1341642703 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d241031c0 msgr2=0x7f3d2419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:55.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.821+0000 7f3d2259c640 1 --2- 172.21.15.71:0/1341642703 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d241031c0 0x7f3d2419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:55.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.821+0000 7f3d2259c640 1 -- 172.21.15.71:0/1341642703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d24102800 msgr2=0x7f3d24197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:55.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.821+0000 7f3d2259c640 1 --2- 172.21.15.71:0/1341642703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d24102800 0x7f3d24197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:55.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.821+0000 7f3d2259c640 1 -- 172.21.15.71:0/1341642703 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2419f570 con 0x7f3d24101610 2024-03-04T08:55:55.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.821+0000 7f3d22d9d640 1 --2- 172.21.15.71:0/1341642703 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d241031c0 0x7f3d2419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:55:55.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.821+0000 7f3d2259c640 1 --2- 172.21.15.71:0/1341642703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d24101610 0x7f3d24197950 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3d140324c0 tx=0x7f3d14032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:55:55.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.821+0000 7f3d137fe640 1 -- 172.21.15.71:0/1341642703 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d14041020 con 0x7f3d24101610 2024-03-04T08:55:55.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.821+0000 7f3d28c29640 1 -- 172.21.15.71:0/1341642703 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d2419f800 con 0x7f3d24101610 2024-03-04T08:55:55.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.821+0000 7f3d137fe640 1 -- 172.21.15.71:0/1341642703 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3d14034cd0 con 0x7f3d24101610 2024-03-04T08:55:55.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.821+0000 7f3d137fe640 1 -- 172.21.15.71:0/1341642703 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d14039040 con 0x7f3d24101610 2024-03-04T08:55:55.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.821+0000 7f3d28c29640 1 -- 172.21.15.71:0/1341642703 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d2419fcf0 con 0x7f3d24101610 2024-03-04T08:55:55.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.822+0000 7f3d28c29640 1 -- 172.21.15.71:0/1341642703 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d24108a00 con 0x7f3d24101610 2024-03-04T08:55:55.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.823+0000 7f3d137fe640 1 -- 172.21.15.71:0/1341642703 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f3d14044020 con 0x7f3d24101610 2024-03-04T08:55:55.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.824+0000 7f3d137fe640 1 --2- 172.21.15.71:0/1341642703 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3d0c0a1d80 0x7f3d0c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:55.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.824+0000 7f3d21d9b640 1 --2- 172.21.15.71:0/1341642703 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3d0c0a1d80 0x7f3d0c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:55.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.824+0000 7f3d137fe640 1 -- 172.21.15.71:0/1341642703 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3d140ef810 con 0x7f3d24101610 2024-03-04T08:55:55.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.824+0000 7f3d21d9b640 1 --2- 172.21.15.71:0/1341642703 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3d0c0a1d80 0x7f3d0c0a4240 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3d24198e60 tx=0x7f3d1800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:55:55.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:55.829+0000 7f3d137fe640 1 -- 172.21.15.71:0/1341642703 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3d1404b020 con 0x7f3d24101610 2024-03-04T08:55:55.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:55 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:55.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:55 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:55.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:55 smithi071 ceph-mon[32789]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:55.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:55 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:55.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:55 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:55:55.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:55 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:55:55.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:55 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:55.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:55 smithi071 ceph-mon[32789]: Deploying daemon mds.k on smithi150 2024-03-04T08:55:56.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:55 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:56.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:55 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:56.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:55 smithi150 ceph-mon[36807]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:56.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:55 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:56.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:55 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:55:56.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:55 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:55:56.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:55 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:56.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:55 smithi150 ceph-mon[36807]: Deploying daemon mds.k on smithi150 2024-03-04T08:55:56.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:56.039+0000 7f3d28c29640 1 -- 172.21.15.71:0/1341642703 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d2402cff0 con 0x7f3d24101610 2024-03-04T08:55:56.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:56.040+0000 7f3d137fe640 1 -- 172.21.15.71:0/1341642703 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7f3d140bd170 con 0x7f3d24101610 2024-03-04T08:55:56.036 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:55:56.036 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/178179528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":178179528},{"type":"v1","addr":"172.21.15.158:6833","nonce":178179528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-04T08:55:56.036 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 2 2024-03-04T08:55:56.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:56.042+0000 7f3d28c29640 1 -- 172.21.15.71:0/1341642703 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3d0c0a1d80 msgr2=0x7f3d0c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:56.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:56.042+0000 7f3d28c29640 1 --2- 172.21.15.71:0/1341642703 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3d0c0a1d80 0x7f3d0c0a4240 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3d24198e60 tx=0x7f3d1800a2d0 comp rx=0 tx=0).stop 2024-03-04T08:55:56.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:56.042+0000 7f3d28c29640 1 -- 172.21.15.71:0/1341642703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d24101610 msgr2=0x7f3d24197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:56.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:56.042+0000 7f3d28c29640 1 --2- 172.21.15.71:0/1341642703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d24101610 0x7f3d24197950 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3d140324c0 tx=0x7f3d14032f40 comp rx=0 tx=0).stop 2024-03-04T08:55:56.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:56.042+0000 7f3d28c29640 1 -- 172.21.15.71:0/1341642703 shutdown_connections 2024-03-04T08:55:56.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:56.042+0000 7f3d28c29640 1 --2- 172.21.15.71:0/1341642703 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3d0c0a1d80 0x7f3d0c0a4240 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:56.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:56.042+0000 7f3d28c29640 1 --2- 172.21.15.71:0/1341642703 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d241031c0 0x7f3d2419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:56.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:56.042+0000 7f3d28c29640 1 --2- 172.21.15.71:0/1341642703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d24102800 0x7f3d24197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:56.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:56.042+0000 7f3d28c29640 1 --2- 172.21.15.71:0/1341642703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d24101610 0x7f3d24197950 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:56.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:56.042+0000 7f3d28c29640 1 -- 172.21.15.71:0/1341642703 >> 172.21.15.71:0/1341642703 conn(0x7f3d240fcb80 msgr2=0x7f3d24105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:55:56.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:56.042+0000 7f3d28c29640 1 -- 172.21.15.71:0/1341642703 shutdown_connections 2024-03-04T08:55:56.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:56.042+0000 7f3d28c29640 1 -- 172.21.15.71:0/1341642703 wait complete. 2024-03-04T08:55:56.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:55 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:56.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:55 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:56.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:55 smithi158 ceph-mon[37241]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:56.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:55 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:56.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:55 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:55:56.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:55 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:55:56.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:55 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:56.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:55 smithi158 ceph-mon[37241]: Deploying daemon mds.k on smithi150 2024-03-04T08:55:56.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:56 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1341642703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:55:57.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:56 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1341642703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:55:57.030 INFO:journalctl@ceph.mds.k.smithi150.stdout:Mar 04 08:55:56 smithi150 systemd[1]: Starting Ceph mds.k for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T08:55:57.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:56 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1341642703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:55:57.372 INFO:journalctl@ceph.mds.k.smithi150.stdout:Mar 04 08:55:57 smithi150 podman[65709]: 2024-03-04 08:55:57.01106558 +0000 UTC m=+0.017850981 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T08:55:57.372 INFO:journalctl@ceph.mds.k.smithi150.stdout:Mar 04 08:55:57 smithi150 podman[65709]: 2024-03-04 08:55:57.123884864 +0000 UTC m=+0.130670246 container create 12678faf940d5a21c6a64db34d2d4c67474b936ee78ffc5acdc721b8395c5d4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-k, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-04T08:55:57.372 INFO:journalctl@ceph.mds.k.smithi150.stdout:Mar 04 08:55:57 smithi150 podman[65709]: 2024-03-04 08:55:57.376767014 +0000 UTC m=+0.383552406 container init 12678faf940d5a21c6a64db34d2d4c67474b936ee78ffc5acdc721b8395c5d4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-04T08:55:57.646 INFO:journalctl@ceph.mds.k.smithi150.stdout:Mar 04 08:55:57 smithi150 podman[65709]: 2024-03-04 08:55:57.379849706 +0000 UTC m=+0.386635089 container start 12678faf940d5a21c6a64db34d2d4c67474b936ee78ffc5acdc721b8395c5d4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-k, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-04T08:55:57.646 INFO:journalctl@ceph.mds.k.smithi150.stdout:Mar 04 08:55:57 smithi150 ceph-mds[65729]: -- 172.21.15.150:0/3660172459 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b77e571a40 con 0x55b77e544c00 2024-03-04T08:55:57.646 INFO:journalctl@ceph.mds.k.smithi150.stdout:Mar 04 08:55:57 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-k[65724]: starting mds.k at 2024-03-04T08:55:57.646 INFO:journalctl@ceph.mds.k.smithi150.stdout:Mar 04 08:55:57 smithi150 bash[65709]: 12678faf940d5a21c6a64db34d2d4c67474b936ee78ffc5acdc721b8395c5d4d 2024-03-04T08:55:57.646 INFO:journalctl@ceph.mds.k.smithi150.stdout:Mar 04 08:55:57 smithi150 systemd[1]: Started Ceph mds.k for b3c85c40-da03-11ee-95c6-87774f69a715. 2024-03-04T08:55:57.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:57 smithi071 ceph-mon[32789]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:57.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:57 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:57.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:57 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:57.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:57 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:57.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:57 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:55:57.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:57 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:55:57.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:57 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:58.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:57 smithi150 ceph-mon[36807]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:58.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:57 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:58.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:57 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:58.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:57 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:58.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:57 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:55:58.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:57 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:55:58.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:57 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:58.049 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:55:58.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:57 smithi158 ceph-mon[37241]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:58.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:57 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:58.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:57 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:58.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:57 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:55:58.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:57 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:55:58.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:57 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:55:58.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:57 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:55:58.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.164+0000 7f4c68288640 1 -- 172.21.15.71:0/4007157925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c601031a0 msgr2=0x7f4c60109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:58.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.164+0000 7f4c68288640 1 --2- 172.21.15.71:0/4007157925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c601031a0 0x7f4c60109a30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c009900 tx=0x7f4c5c02bc30 comp rx=0 tx=0).stop 2024-03-04T08:55:58.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.164+0000 7f4c68288640 1 -- 172.21.15.71:0/4007157925 shutdown_connections 2024-03-04T08:55:58.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.164+0000 7f4c68288640 1 --2- 172.21.15.71:0/4007157925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c601031a0 0x7f4c60109a30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:58.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.164+0000 7f4c68288640 1 --2- 172.21.15.71:0/4007157925 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c601027e0 0x7f4c60102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:58.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.164+0000 7f4c68288640 1 --2- 172.21.15.71:0/4007157925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c601015f0 0x7f4c60101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:58.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.164+0000 7f4c68288640 1 -- 172.21.15.71:0/4007157925 >> 172.21.15.71:0/4007157925 conn(0x7f4c600fcb80 msgr2=0x7f4c600fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:55:58.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.164+0000 7f4c68288640 1 -- 172.21.15.71:0/4007157925 shutdown_connections 2024-03-04T08:55:58.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.164+0000 7f4c68288640 1 -- 172.21.15.71:0/4007157925 wait complete. 2024-03-04T08:55:58.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.164+0000 7f4c68288640 1 Processor -- start 2024-03-04T08:55:58.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.164+0000 7f4c68288640 1 -- start start 2024-03-04T08:55:58.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.165+0000 7f4c68288640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c601015f0 0x7f4c601978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:58.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.165+0000 7f4c68288640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c601027e0 0x7f4c60197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:58.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.165+0000 7f4c68288640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c601031a0 0x7f4c6019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:58.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.165+0000 7f4c68288640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6010d2d0 con 0x7f4c601027e0 2024-03-04T08:55:58.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.165+0000 7f4c68288640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6010d150 con 0x7f4c601015f0 2024-03-04T08:55:58.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.165+0000 7f4c68288640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6010d450 con 0x7f4c601031a0 2024-03-04T08:55:58.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.165+0000 7f4c65823640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c601027e0 0x7f4c60197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:58.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.165+0000 7f4c65823640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c601027e0 0x7f4c60197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48286/0 (socket says 172.21.15.71:48286) 2024-03-04T08:55:58.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.165+0000 7f4c66024640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c601015f0 0x7f4c601978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:58.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.165+0000 7f4c65823640 1 -- 172.21.15.71:0/3638583018 learned_addr learned my addr 172.21.15.71:0/3638583018 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:55:58.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.165+0000 7f4c66825640 1 --2- 172.21.15.71:0/3638583018 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c601031a0 0x7f4c6019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:58.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.165+0000 7f4c65823640 1 -- 172.21.15.71:0/3638583018 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c601031a0 msgr2=0x7f4c6019ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:58.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.165+0000 7f4c65823640 1 --2- 172.21.15.71:0/3638583018 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c601031a0 0x7f4c6019ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:58.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.165+0000 7f4c65823640 1 -- 172.21.15.71:0/3638583018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c601015f0 msgr2=0x7f4c601978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:58.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.165+0000 7f4c65823640 1 --2- 172.21.15.71:0/3638583018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c601015f0 0x7f4c601978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:58.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.165+0000 7f4c65823640 1 -- 172.21.15.71:0/3638583018 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6019f500 con 0x7f4c601027e0 2024-03-04T08:55:58.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.165+0000 7f4c66825640 1 --2- 172.21.15.71:0/3638583018 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c601031a0 0x7f4c6019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:55:58.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.165+0000 7f4c65823640 1 --2- 172.21.15.71:0/3638583018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c601027e0 0x7f4c60197e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000b4a0 tx=0x7f4c5000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:55:58.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.165+0000 7f4c577fe640 1 -- 172.21.15.71:0/3638583018 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c50014070 con 0x7f4c601027e0 2024-03-04T08:55:58.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.165+0000 7f4c577fe640 1 -- 172.21.15.71:0/3638583018 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c50010af0 con 0x7f4c601027e0 2024-03-04T08:55:58.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.165+0000 7f4c68288640 1 -- 172.21.15.71:0/3638583018 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c6019f7f0 con 0x7f4c601027e0 2024-03-04T08:55:58.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.165+0000 7f4c577fe640 1 -- 172.21.15.71:0/3638583018 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c50017d10 con 0x7f4c601027e0 2024-03-04T08:55:58.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.165+0000 7f4c68288640 1 -- 172.21.15.71:0/3638583018 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c6019fcf0 con 0x7f4c601027e0 2024-03-04T08:55:58.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.166+0000 7f4c68288640 1 -- 172.21.15.71:0/3638583018 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c60108910 con 0x7f4c601027e0 2024-03-04T08:55:58.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.167+0000 7f4c577fe640 1 -- 172.21.15.71:0/3638583018 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f4c50017070 con 0x7f4c601027e0 2024-03-04T08:55:58.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.168+0000 7f4c577fe640 1 --2- 172.21.15.71:0/3638583018 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f4c4c0a1d80 0x7f4c4c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:55:58.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.168+0000 7f4c66024640 1 --2- 172.21.15.71:0/3638583018 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f4c4c0a1d80 0x7f4c4c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:55:58.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.168+0000 7f4c577fe640 1 -- 172.21.15.71:0/3638583018 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4c500cd0d0 con 0x7f4c601027e0 2024-03-04T08:55:58.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.168+0000 7f4c66024640 1 --2- 172.21.15.71:0/3638583018 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f4c4c0a1d80 0x7f4c4c0a4240 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4c60102640 tx=0x7f4c5800a590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:55:58.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.171+0000 7f4c577fe640 1 -- 172.21.15.71:0/3638583018 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4c5009a9a0 con 0x7f4c601027e0 2024-03-04T08:55:58.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.392+0000 7f4c68288640 1 -- 172.21.15.71:0/3638583018 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c6002cff0 con 0x7f4c601027e0 2024-03-04T08:55:58.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.393+0000 7f4c577fe640 1 -- 172.21.15.71:0/3638583018 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7f4c5009a2a0 con 0x7f4c601027e0 2024-03-04T08:55:58.389 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:55:58.389 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/178179528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":178179528},{"type":"v1","addr":"172.21.15.158:6833","nonce":178179528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2468877675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2468877675},{"type":"v1","addr":"172.21.15.150:6833","nonce":2468877675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-04T08:55:58.390 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 3 2024-03-04T08:55:58.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.395+0000 7f4c68288640 1 -- 172.21.15.71:0/3638583018 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f4c4c0a1d80 msgr2=0x7f4c4c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:58.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.395+0000 7f4c68288640 1 --2- 172.21.15.71:0/3638583018 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f4c4c0a1d80 0x7f4c4c0a4240 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4c60102640 tx=0x7f4c5800a590 comp rx=0 tx=0).stop 2024-03-04T08:55:58.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.395+0000 7f4c68288640 1 -- 172.21.15.71:0/3638583018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c601027e0 msgr2=0x7f4c60197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:55:58.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.395+0000 7f4c68288640 1 --2- 172.21.15.71:0/3638583018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c601027e0 0x7f4c60197e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000b4a0 tx=0x7f4c5000b960 comp rx=0 tx=0).stop 2024-03-04T08:55:58.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.395+0000 7f4c68288640 1 -- 172.21.15.71:0/3638583018 shutdown_connections 2024-03-04T08:55:58.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.395+0000 7f4c68288640 1 --2- 172.21.15.71:0/3638583018 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f4c4c0a1d80 0x7f4c4c0a4240 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:58.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.395+0000 7f4c68288640 1 --2- 172.21.15.71:0/3638583018 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c601031a0 0x7f4c6019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:58.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.395+0000 7f4c68288640 1 --2- 172.21.15.71:0/3638583018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c601027e0 0x7f4c60197e20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:58.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.395+0000 7f4c68288640 1 --2- 172.21.15.71:0/3638583018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c601015f0 0x7f4c601978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:55:58.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.395+0000 7f4c68288640 1 -- 172.21.15.71:0/3638583018 >> 172.21.15.71:0/3638583018 conn(0x7f4c600fcb80 msgr2=0x7f4c60105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:55:58.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.395+0000 7f4c68288640 1 -- 172.21.15.71:0/3638583018 shutdown_connections 2024-03-04T08:55:58.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:55:58.395+0000 7f4c68288640 1 -- 172.21.15.71:0/3638583018 wait complete. 2024-03-04T08:55:58.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:58 smithi150 ceph-mon[36807]: Deploying daemon mds.j on smithi071 2024-03-04T08:55:58.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:58 smithi150 ceph-mon[36807]: mds.? [v2:172.21.15.150:6832/2468877675,v1:172.21.15.150:6833/2468877675] up:boot 2024-03-04T08:55:58.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:58 smithi150 ceph-mon[36807]: fsmap 2 up:standby 2024-03-04T08:55:58.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:58 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-04T08:55:58.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:58 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3638583018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:55:58.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:58 smithi071 ceph-mon[32789]: Deploying daemon mds.j on smithi071 2024-03-04T08:55:58.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:58 smithi071 ceph-mon[32789]: mds.? [v2:172.21.15.150:6832/2468877675,v1:172.21.15.150:6833/2468877675] up:boot 2024-03-04T08:55:58.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:58 smithi071 ceph-mon[32789]: fsmap 2 up:standby 2024-03-04T08:55:58.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:58 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-04T08:55:58.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:58 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3638583018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:55:59.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:58 smithi158 ceph-mon[37241]: Deploying daemon mds.j on smithi071 2024-03-04T08:55:59.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:58 smithi158 ceph-mon[37241]: mds.? [v2:172.21.15.150:6832/2468877675,v1:172.21.15.150:6833/2468877675] up:boot 2024-03-04T08:55:59.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:58 smithi158 ceph-mon[37241]: fsmap 2 up:standby 2024-03-04T08:55:59.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:58 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-04T08:55:59.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:58 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3638583018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:55:59.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:55:59 smithi071 ceph-mon[32789]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:55:59.922 INFO:journalctl@ceph.mds.j.smithi071.stdout:Mar 04 08:55:59 smithi071 systemd[1]: Starting Ceph mds.j for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T08:56:00.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:55:59 smithi150 ceph-mon[36807]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:00.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:55:59 smithi158 ceph-mon[37241]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:00.186 INFO:journalctl@ceph.mds.j.smithi071.stdout:Mar 04 08:55:59 smithi071 podman[73255]: 2024-03-04 08:55:59.837256212 +0000 UTC m=+0.018110165 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T08:56:00.187 INFO:journalctl@ceph.mds.j.smithi071.stdout:Mar 04 08:55:59 smithi071 podman[73255]: 2024-03-04 08:55:59.963730399 +0000 UTC m=+0.144584346 container create 32208bdaccc6609be16e44faacb1f09c8d3aa6fc763a40ca669ef9443d99e272 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-j, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, ceph=True) 2024-03-04T08:56:00.187 INFO:journalctl@ceph.mds.j.smithi071.stdout:Mar 04 08:56:00 smithi071 podman[73255]: 2024-03-04 08:56:00.191446695 +0000 UTC m=+0.372300643 container init 32208bdaccc6609be16e44faacb1f09c8d3aa6fc763a40ca669ef9443d99e272 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T08:56:00.403 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:56:00.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.516+0000 7f7395f13640 1 -- 172.21.15.71:0/1448351288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73901049a0 msgr2=0x7f7390106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:00.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.516+0000 7f7395f13640 1 --2- 172.21.15.71:0/1448351288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73901049a0 0x7f7390106d90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7378009900 tx=0x7f737802bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:00.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.516+0000 7f7395f13640 1 -- 172.21.15.71:0/1448351288 shutdown_connections 2024-03-04T08:56:00.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.516+0000 7f7395f13640 1 --2- 172.21.15.71:0/1448351288 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73901072d0 0x7f7390109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:00.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.516+0000 7f7395f13640 1 --2- 172.21.15.71:0/1448351288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73901049a0 0x7f7390106d90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:00.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.516+0000 7f7395f13640 1 --2- 172.21.15.71:0/1448351288 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7390102070 0x7f7390104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:00.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.516+0000 7f7395f13640 1 -- 172.21.15.71:0/1448351288 >> 172.21.15.71:0/1448351288 conn(0x7f73900fba80 msgr2=0x7f73900fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:00.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.516+0000 7f7395f13640 1 -- 172.21.15.71:0/1448351288 shutdown_connections 2024-03-04T08:56:00.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.516+0000 7f7395f13640 1 -- 172.21.15.71:0/1448351288 wait complete. 2024-03-04T08:56:00.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.517+0000 7f7395f13640 1 Processor -- start 2024-03-04T08:56:00.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.517+0000 7f7395f13640 1 -- start start 2024-03-04T08:56:00.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.517+0000 7f7395f13640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7390102070 0x7f7390197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:00.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.517+0000 7f7395f13640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73901049a0 0x7f7390197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:00.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.517+0000 7f7395f13640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73901072d0 0x7f739019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:00.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.517+0000 7f7395f13640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739010d270 con 0x7f7390102070 2024-03-04T08:56:00.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.517+0000 7f7395f13640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739010d0f0 con 0x7f73901049a0 2024-03-04T08:56:00.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.517+0000 7f7395f13640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739010d3f0 con 0x7f73901072d0 2024-03-04T08:56:00.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.517+0000 7f738f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7390102070 0x7f7390197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:00.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.517+0000 7f738f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7390102070 0x7f7390197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48322/0 (socket says 172.21.15.71:48322) 2024-03-04T08:56:00.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.517+0000 7f738f7fe640 1 -- 172.21.15.71:0/90960646 learned_addr learned my addr 172.21.15.71:0/90960646 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:00.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.517+0000 7f7387fff640 1 --2- 172.21.15.71:0/90960646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73901049a0 0x7f7390197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:00.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.517+0000 7f738ffff640 1 --2- 172.21.15.71:0/90960646 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73901072d0 0x7f739019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:00.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.517+0000 7f738f7fe640 1 -- 172.21.15.71:0/90960646 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73901072d0 msgr2=0x7f739019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:00.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.517+0000 7f738f7fe640 1 --2- 172.21.15.71:0/90960646 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73901072d0 0x7f739019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:00.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.517+0000 7f738f7fe640 1 -- 172.21.15.71:0/90960646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73901049a0 msgr2=0x7f7390197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:00.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.517+0000 7f738f7fe640 1 --2- 172.21.15.71:0/90960646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73901049a0 0x7f7390197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:00.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.517+0000 7f738f7fe640 1 -- 172.21.15.71:0/90960646 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739019f560 con 0x7f7390102070 2024-03-04T08:56:00.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.517+0000 7f738f7fe640 1 --2- 172.21.15.71:0/90960646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7390102070 0x7f7390197940 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f738000c970 tx=0x7f738000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:00.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.518+0000 7f738d7fa640 1 -- 172.21.15.71:0/90960646 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7380007bb0 con 0x7f7390102070 2024-03-04T08:56:00.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.518+0000 7f738d7fa640 1 -- 172.21.15.71:0/90960646 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7380007d50 con 0x7f7390102070 2024-03-04T08:56:00.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.518+0000 7f738d7fa640 1 -- 172.21.15.71:0/90960646 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738000f450 con 0x7f7390102070 2024-03-04T08:56:00.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.518+0000 7f7395f13640 1 -- 172.21.15.71:0/90960646 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f739019f850 con 0x7f7390102070 2024-03-04T08:56:00.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.518+0000 7f7395f13640 1 -- 172.21.15.71:0/90960646 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f739019fd70 con 0x7f7390102070 2024-03-04T08:56:00.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.519+0000 7f7395f13640 1 -- 172.21.15.71:0/90960646 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73900fd570 con 0x7f7390102070 2024-03-04T08:56:00.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.520+0000 7f738d7fa640 1 -- 172.21.15.71:0/90960646 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f7380005020 con 0x7f7390102070 2024-03-04T08:56:00.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.520+0000 7f738d7fa640 1 --2- 172.21.15.71:0/90960646 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f737c0a1e00 0x7f737c0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:00.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.520+0000 7f7387fff640 1 --2- 172.21.15.71:0/90960646 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f737c0a1e00 0x7f737c0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:00.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.521+0000 7f738d7fa640 1 -- 172.21.15.71:0/90960646 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f73800cd650 con 0x7f7390102070 2024-03-04T08:56:00.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.521+0000 7f7387fff640 1 --2- 172.21.15.71:0/90960646 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f737c0a1e00 0x7f737c0a42c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7378000c00 tx=0x7f7378004f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:00.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.525+0000 7f738d7fa640 1 -- 172.21.15.71:0/90960646 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7380014030 con 0x7f7390102070 2024-03-04T08:56:00.671 INFO:journalctl@ceph.mds.j.smithi071.stdout:Mar 04 08:56:00 smithi071 podman[73255]: 2024-03-04 08:56:00.194185751 +0000 UTC m=+0.375039700 container start 32208bdaccc6609be16e44faacb1f09c8d3aa6fc763a40ca669ef9443d99e272 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-j, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5) 2024-03-04T08:56:00.671 INFO:journalctl@ceph.mds.j.smithi071.stdout:Mar 04 08:56:00 smithi071 ceph-mds[73275]: -- 172.21.15.71:0/211260801 <== mon.0 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5563c83ffa40 con 0x5563c83d2c00 2024-03-04T08:56:00.672 INFO:journalctl@ceph.mds.j.smithi071.stdout:Mar 04 08:56:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-j[73271]: starting mds.j at 2024-03-04T08:56:00.672 INFO:journalctl@ceph.mds.j.smithi071.stdout:Mar 04 08:56:00 smithi071 bash[73255]: 32208bdaccc6609be16e44faacb1f09c8d3aa6fc763a40ca669ef9443d99e272 2024-03-04T08:56:00.672 INFO:journalctl@ceph.mds.j.smithi071.stdout:Mar 04 08:56:00 smithi071 systemd[1]: Started Ceph mds.j for b3c85c40-da03-11ee-95c6-87774f69a715. 2024-03-04T08:56:00.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.778+0000 7f7395f13640 1 -- 172.21.15.71:0/90960646 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f739002cff0 con 0x7f7390102070 2024-03-04T08:56:00.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.779+0000 7f738d7fa640 1 -- 172.21.15.71:0/90960646 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7f739002cff0 con 0x7f7390102070 2024-03-04T08:56:00.775 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:00.775 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3975379807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3975379807},{"type":"v1","addr":"172.21.15.71:6835","nonce":3975379807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/178179528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":178179528},{"type":"v1","addr":"172.21.15.158:6833","nonce":178179528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2468877675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2468877675},{"type":"v1","addr":"172.21.15.150:6833","nonce":2468877675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-04T08:56:00.775 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 4 2024-03-04T08:56:00.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.781+0000 7f7395f13640 1 -- 172.21.15.71:0/90960646 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f737c0a1e00 msgr2=0x7f737c0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:00.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.781+0000 7f7395f13640 1 --2- 172.21.15.71:0/90960646 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f737c0a1e00 0x7f737c0a42c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7378000c00 tx=0x7f7378004f10 comp rx=0 tx=0).stop 2024-03-04T08:56:00.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.781+0000 7f7395f13640 1 -- 172.21.15.71:0/90960646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7390102070 msgr2=0x7f7390197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:00.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.781+0000 7f7395f13640 1 --2- 172.21.15.71:0/90960646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7390102070 0x7f7390197940 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f738000c970 tx=0x7f738000ce30 comp rx=0 tx=0).stop 2024-03-04T08:56:00.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.781+0000 7f7395f13640 1 -- 172.21.15.71:0/90960646 shutdown_connections 2024-03-04T08:56:00.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.781+0000 7f7395f13640 1 --2- 172.21.15.71:0/90960646 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f737c0a1e00 0x7f737c0a42c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:00.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.781+0000 7f7395f13640 1 --2- 172.21.15.71:0/90960646 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f73901072d0 0x7f739019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:00.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.781+0000 7f7395f13640 1 --2- 172.21.15.71:0/90960646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73901049a0 0x7f7390197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:00.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.781+0000 7f7395f13640 1 --2- 172.21.15.71:0/90960646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7390102070 0x7f7390197940 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:00.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.781+0000 7f7395f13640 1 -- 172.21.15.71:0/90960646 >> 172.21.15.71:0/90960646 conn(0x7f73900fba80 msgr2=0x7f73901029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:00.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.781+0000 7f7395f13640 1 -- 172.21.15.71:0/90960646 shutdown_connections 2024-03-04T08:56:00.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:00.781+0000 7f7395f13640 1 -- 172.21.15.71:0/90960646 wait complete. 2024-03-04T08:56:01.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:01 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:01.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:01 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:01.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:01 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:01.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:01 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:01.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:01 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:01.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:01 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:01.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:01 smithi071 ceph-mon[32789]: Deploying daemon mds.d on smithi071 2024-03-04T08:56:01.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:01 smithi071 ceph-mon[32789]: mds.? [v2:172.21.15.71:6834/3975379807,v1:172.21.15.71:6835/3975379807] up:boot 2024-03-04T08:56:01.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:01 smithi071 ceph-mon[32789]: fsmap 3 up:standby 2024-03-04T08:56:01.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:01 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-04T08:56:01.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:01 smithi071 ceph-mon[32789]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:01.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:01 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/90960646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:01.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:01 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:01.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:01 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:01.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:01 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:01.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:01 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:01.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:01 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:01.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:01 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:01.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:01 smithi150 ceph-mon[36807]: Deploying daemon mds.d on smithi071 2024-03-04T08:56:01.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:01 smithi150 ceph-mon[36807]: mds.? [v2:172.21.15.71:6834/3975379807,v1:172.21.15.71:6835/3975379807] up:boot 2024-03-04T08:56:01.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:01 smithi150 ceph-mon[36807]: fsmap 3 up:standby 2024-03-04T08:56:01.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:01 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-04T08:56:01.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:01 smithi150 ceph-mon[36807]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:01.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:01 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/90960646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:01.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:01 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:01.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:01 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:01.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:01 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:01.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:01 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:01.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:01 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:01.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:01 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:01.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:01 smithi158 ceph-mon[37241]: Deploying daemon mds.d on smithi071 2024-03-04T08:56:01.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:01 smithi158 ceph-mon[37241]: mds.? [v2:172.21.15.71:6834/3975379807,v1:172.21.15.71:6835/3975379807] up:boot 2024-03-04T08:56:01.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:01 smithi158 ceph-mon[37241]: fsmap 3 up:standby 2024-03-04T08:56:01.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:01 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-04T08:56:01.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:01 smithi158 ceph-mon[37241]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:01.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:01 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/90960646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:02.671 INFO:journalctl@ceph.mds.d.smithi071.stdout:Mar 04 08:56:02 smithi071 systemd[1]: Starting Ceph mds.d for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T08:56:02.789 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:56:02.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.900+0000 7fb885823640 1 -- 172.21.15.71:0/1393959894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8801020a0 msgr2=0x7fb880108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:02.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.900+0000 7fb885823640 1 --2- 172.21.15.71:0/1393959894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8801020a0 0x7fb880108930 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb870009900 tx=0x7fb87002bc50 comp rx=0 tx=0).stop 2024-03-04T08:56:02.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.900+0000 7fb885823640 1 -- 172.21.15.71:0/1393959894 shutdown_connections 2024-03-04T08:56:02.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.900+0000 7fb885823640 1 --2- 172.21.15.71:0/1393959894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8801020a0 0x7fb880108930 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:02.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.900+0000 7fb885823640 1 --2- 172.21.15.71:0/1393959894 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8801016e0 0x7fb880101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:02.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.900+0000 7fb885823640 1 --2- 172.21.15.71:0/1393959894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8801004f0 0x7fb880100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:02.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.900+0000 7fb885823640 1 -- 172.21.15.71:0/1393959894 >> 172.21.15.71:0/1393959894 conn(0x7fb8800fba60 msgr2=0x7fb8800fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:02.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.900+0000 7fb885823640 1 -- 172.21.15.71:0/1393959894 shutdown_connections 2024-03-04T08:56:02.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.900+0000 7fb885823640 1 -- 172.21.15.71:0/1393959894 wait complete. 2024-03-04T08:56:02.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.900+0000 7fb885823640 1 Processor -- start 2024-03-04T08:56:02.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.901+0000 7fb885823640 1 -- start start 2024-03-04T08:56:02.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.901+0000 7fb885823640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8801004f0 0x7fb8800692a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:02.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.901+0000 7fb885823640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8801016e0 0x7fb88019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:02.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.901+0000 7fb885823640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8801020a0 0x7fb88019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:02.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.901+0000 7fb885823640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88010c280 con 0x7fb8801020a0 2024-03-04T08:56:02.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.901+0000 7fb885823640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88010c100 con 0x7fb8801016e0 2024-03-04T08:56:02.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.901+0000 7fb885823640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88010c400 con 0x7fb8801004f0 2024-03-04T08:56:02.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.901+0000 7fb87f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8801020a0 0x7fb88019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:02.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.901+0000 7fb87f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8801020a0 0x7fb88019edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48334/0 (socket says 172.21.15.71:48334) 2024-03-04T08:56:02.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.901+0000 7fb87f7fe640 1 -- 172.21.15.71:0/710814767 learned_addr learned my addr 172.21.15.71:0/710814767 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:02.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.901+0000 7fb87effd640 1 --2- 172.21.15.71:0/710814767 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8801004f0 0x7fb8800692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:02.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.901+0000 7fb87f7fe640 1 -- 172.21.15.71:0/710814767 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8801004f0 msgr2=0x7fb8800692a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:02.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.901+0000 7fb87f7fe640 1 --2- 172.21.15.71:0/710814767 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8801004f0 0x7fb8800692a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:02.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.901+0000 7fb87f7fe640 1 -- 172.21.15.71:0/710814767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8801016e0 msgr2=0x7fb88019c7b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-04T08:56:02.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.901+0000 7fb87f7fe640 1 --2- 172.21.15.71:0/710814767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8801016e0 0x7fb88019c7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:02.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.901+0000 7fb87f7fe640 1 -- 172.21.15.71:0/710814767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88019f6e0 con 0x7fb8801020a0 2024-03-04T08:56:02.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.901+0000 7fb87effd640 1 --2- 172.21.15.71:0/710814767 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8801004f0 0x7fb8800692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:56:02.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.901+0000 7fb87f7fe640 1 --2- 172.21.15.71:0/710814767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8801020a0 0x7fb88019edd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb870005ab0 tx=0x7fb870004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:02.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.901+0000 7fb884821640 1 -- 172.21.15.71:0/710814767 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb870042020 con 0x7fb8801020a0 2024-03-04T08:56:02.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.901+0000 7fb885823640 1 -- 172.21.15.71:0/710814767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88019f910 con 0x7fb8801020a0 2024-03-04T08:56:02.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.901+0000 7fb884821640 1 -- 172.21.15.71:0/710814767 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8700054e0 con 0x7fb8801020a0 2024-03-04T08:56:02.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.901+0000 7fb884821640 1 -- 172.21.15.71:0/710814767 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87003e6c0 con 0x7fb8801020a0 2024-03-04T08:56:02.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.901+0000 7fb885823640 1 -- 172.21.15.71:0/710814767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88019feb0 con 0x7fb8801020a0 2024-03-04T08:56:02.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.903+0000 7fb885823640 1 -- 172.21.15.71:0/710814767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8801077b0 con 0x7fb8801020a0 2024-03-04T08:56:02.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.904+0000 7fb884821640 1 -- 172.21.15.71:0/710814767 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fb870045020 con 0x7fb8801020a0 2024-03-04T08:56:02.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.904+0000 7fb884821640 1 --2- 172.21.15.71:0/710814767 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb8680a1d80 0x7fb8680a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:02.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.905+0000 7fb87effd640 1 --2- 172.21.15.71:0/710814767 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb8680a1d80 0x7fb8680a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:02.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.905+0000 7fb884821640 1 -- 172.21.15.71:0/710814767 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb8700eff40 con 0x7fb8801020a0 2024-03-04T08:56:02.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.905+0000 7fb87effd640 1 --2- 172.21.15.71:0/710814767 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb8680a1d80 0x7fb8680a4240 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb880101540 tx=0x7fb878008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:02.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:02.911+0000 7fb884821640 1 -- 172.21.15.71:0/710814767 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb87004c020 con 0x7fb8801020a0 2024-03-04T08:56:03.027 INFO:journalctl@ceph.mds.d.smithi071.stdout:Mar 04 08:56:02 smithi071 podman[73607]: 2024-03-04 08:56:02.777362721 +0000 UTC m=+0.108877962 container create 254998c03e910681b4e0a745eec1b547c2b764d4021a2a3ebb3d8ef6677dc048 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2) 2024-03-04T08:56:03.027 INFO:journalctl@ceph.mds.d.smithi071.stdout:Mar 04 08:56:02 smithi071 podman[73607]: 2024-03-04 08:56:02.686226304 +0000 UTC m=+0.017741552 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T08:56:03.027 INFO:journalctl@ceph.mds.d.smithi071.stdout:Mar 04 08:56:03 smithi071 podman[73607]: 2024-03-04 08:56:03.005156138 +0000 UTC m=+0.336671389 container init 254998c03e910681b4e0a745eec1b547c2b764d4021a2a3ebb3d8ef6677dc048 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T08:56:03.027 INFO:journalctl@ceph.mds.d.smithi071.stdout:Mar 04 08:56:03 smithi071 podman[73607]: 2024-03-04 08:56:03.008376222 +0000 UTC m=+0.339891468 container start 254998c03e910681b4e0a745eec1b547c2b764d4021a2a3ebb3d8ef6677dc048 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-d, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-04T08:56:03.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:03.129+0000 7fb885823640 1 -- 172.21.15.71:0/710814767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb88002d1a0 con 0x7fb8801020a0 2024-03-04T08:56:03.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:03.130+0000 7fb884821640 1 -- 172.21.15.71:0/710814767 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7fb8700bd810 con 0x7fb8801020a0 2024-03-04T08:56:03.126 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:03.126 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3975379807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3975379807},{"type":"v1","addr":"172.21.15.71:6835","nonce":3975379807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/178179528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":178179528},{"type":"v1","addr":"172.21.15.158:6833","nonce":178179528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2468877675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2468877675},{"type":"v1","addr":"172.21.15.150:6833","nonce":2468877675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-04T08:56:03.126 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 4 2024-03-04T08:56:03.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:03.132+0000 7fb885823640 1 -- 172.21.15.71:0/710814767 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb8680a1d80 msgr2=0x7fb8680a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:03.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:03.132+0000 7fb885823640 1 --2- 172.21.15.71:0/710814767 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb8680a1d80 0x7fb8680a4240 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb880101540 tx=0x7fb878008040 comp rx=0 tx=0).stop 2024-03-04T08:56:03.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:03.132+0000 7fb885823640 1 -- 172.21.15.71:0/710814767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8801020a0 msgr2=0x7fb88019edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:03.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:03.132+0000 7fb885823640 1 --2- 172.21.15.71:0/710814767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8801020a0 0x7fb88019edd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb870005ab0 tx=0x7fb870004270 comp rx=0 tx=0).stop 2024-03-04T08:56:03.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:03.132+0000 7fb885823640 1 -- 172.21.15.71:0/710814767 shutdown_connections 2024-03-04T08:56:03.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:03.132+0000 7fb885823640 1 --2- 172.21.15.71:0/710814767 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb8680a1d80 0x7fb8680a4240 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:03.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:03.132+0000 7fb885823640 1 --2- 172.21.15.71:0/710814767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8801020a0 0x7fb88019edd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:03.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:03.132+0000 7fb885823640 1 --2- 172.21.15.71:0/710814767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8801016e0 0x7fb88019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:03.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:03.132+0000 7fb885823640 1 --2- 172.21.15.71:0/710814767 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8801004f0 0x7fb8800692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:03.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:03.132+0000 7fb885823640 1 -- 172.21.15.71:0/710814767 >> 172.21.15.71:0/710814767 conn(0x7fb8800fba60 msgr2=0x7fb880104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:03.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:03.132+0000 7fb885823640 1 -- 172.21.15.71:0/710814767 shutdown_connections 2024-03-04T08:56:03.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:03.133+0000 7fb885823640 1 -- 172.21.15.71:0/710814767 wait complete. 2024-03-04T08:56:03.421 INFO:journalctl@ceph.mds.d.smithi071.stdout:Mar 04 08:56:03 smithi071 ceph-mds[73665]: -- 172.21.15.71:0/159213113 <== mon.0 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56008828da40 con 0x560088261000 2024-03-04T08:56:03.421 INFO:journalctl@ceph.mds.d.smithi071.stdout:Mar 04 08:56:03 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-d[73642]: starting mds.d at 2024-03-04T08:56:03.421 INFO:journalctl@ceph.mds.d.smithi071.stdout:Mar 04 08:56:03 smithi071 bash[73607]: 254998c03e910681b4e0a745eec1b547c2b764d4021a2a3ebb3d8ef6677dc048 2024-03-04T08:56:03.421 INFO:journalctl@ceph.mds.d.smithi071.stdout:Mar 04 08:56:03 smithi071 systemd[1]: Started Ceph mds.d for b3c85c40-da03-11ee-95c6-87774f69a715. 2024-03-04T08:56:04.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:03 smithi150 ceph-mon[36807]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:04.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:03 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/710814767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:04.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:03 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:04.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:03 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:04.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:03 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:04.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:03 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:04.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:03 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:04.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:03 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:04.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:03 smithi158 ceph-mon[37241]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:04.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:03 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/710814767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:04.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:03 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:04.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:03 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:04.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:03 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:04.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:03 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:04.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:03 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:04.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:03 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:04.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:03 smithi071 ceph-mon[32789]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:04.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:03 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/710814767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:04.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:03 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:04.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:03 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:04.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:03 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:04.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:03 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:04.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:03 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:04.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:03 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:04.848 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:04 smithi150 ceph-mon[36807]: Deploying daemon mds.e on smithi150 2024-03-04T08:56:04.848 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:04 smithi150 ceph-mon[36807]: mds.? [v2:172.21.15.71:6836/1962929347,v1:172.21.15.71:6837/1962929347] up:boot 2024-03-04T08:56:04.848 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:04 smithi150 ceph-mon[36807]: fsmap 4 up:standby 2024-03-04T08:56:04.848 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:04 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-04T08:56:05.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:04 smithi158 ceph-mon[37241]: Deploying daemon mds.e on smithi150 2024-03-04T08:56:05.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:04 smithi158 ceph-mon[37241]: mds.? [v2:172.21.15.71:6836/1962929347,v1:172.21.15.71:6837/1962929347] up:boot 2024-03-04T08:56:05.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:04 smithi158 ceph-mon[37241]: fsmap 4 up:standby 2024-03-04T08:56:05.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:04 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-04T08:56:05.141 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:56:05.170 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:04 smithi071 ceph-mon[32789]: Deploying daemon mds.e on smithi150 2024-03-04T08:56:05.170 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:04 smithi071 ceph-mon[32789]: mds.? [v2:172.21.15.71:6836/1962929347,v1:172.21.15.71:6837/1962929347] up:boot 2024-03-04T08:56:05.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:04 smithi071 ceph-mon[32789]: fsmap 4 up:standby 2024-03-04T08:56:05.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:04 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-04T08:56:05.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.259+0000 7f1dc8943640 1 -- 172.21.15.71:0/3463355103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dc4101700 msgr2=0x7f1dc4101b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:05.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.259+0000 7f1dc8943640 1 --2- 172.21.15.71:0/3463355103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dc4101700 0x7f1dc4101b80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009900 tx=0x7f1db402bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:05.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.259+0000 7f1dc8943640 1 -- 172.21.15.71:0/3463355103 shutdown_connections 2024-03-04T08:56:05.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.259+0000 7f1dc8943640 1 --2- 172.21.15.71:0/3463355103 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1dc41020c0 0x7f1dc4108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:05.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.259+0000 7f1dc8943640 1 --2- 172.21.15.71:0/3463355103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dc4101700 0x7f1dc4101b80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:05.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.259+0000 7f1dc8943640 1 --2- 172.21.15.71:0/3463355103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1dc4100510 0x7f1dc4100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:05.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.259+0000 7f1dc8943640 1 -- 172.21.15.71:0/3463355103 >> 172.21.15.71:0/3463355103 conn(0x7f1dc40fba80 msgr2=0x7f1dc40fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:05.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.259+0000 7f1dc8943640 1 -- 172.21.15.71:0/3463355103 shutdown_connections 2024-03-04T08:56:05.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.259+0000 7f1dc8943640 1 -- 172.21.15.71:0/3463355103 wait complete. 2024-03-04T08:56:05.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.260+0000 7f1dc8943640 1 Processor -- start 2024-03-04T08:56:05.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.260+0000 7f1dc8943640 1 -- start start 2024-03-04T08:56:05.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.260+0000 7f1dc8943640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dc4100510 0x7f1dc4197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:05.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.260+0000 7f1dc8943640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1dc4101700 0x7f1dc4197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:05.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.260+0000 7f1dc8943640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1dc41020c0 0x7f1dc419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:05.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.260+0000 7f1dc8943640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc410c1a0 con 0x7f1dc4100510 2024-03-04T08:56:05.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.260+0000 7f1dc8943640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc410c020 con 0x7f1dc4101700 2024-03-04T08:56:05.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.260+0000 7f1dc8943640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc410c320 con 0x7f1dc41020c0 2024-03-04T08:56:05.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.260+0000 7f1dc259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dc4100510 0x7f1dc4197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:05.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.260+0000 7f1dc259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dc4100510 0x7f1dc4197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48374/0 (socket says 172.21.15.71:48374) 2024-03-04T08:56:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.260+0000 7f1dc259c640 1 -- 172.21.15.71:0/4174509487 learned_addr learned my addr 172.21.15.71:0/4174509487 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.260+0000 7f1dc1d9b640 1 --2- 172.21.15.71:0/4174509487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1dc4101700 0x7f1dc4197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.260+0000 7f1dc2d9d640 1 --2- 172.21.15.71:0/4174509487 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1dc41020c0 0x7f1dc419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.260+0000 7f1dc259c640 1 -- 172.21.15.71:0/4174509487 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1dc41020c0 msgr2=0x7f1dc419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.260+0000 7f1dc259c640 1 --2- 172.21.15.71:0/4174509487 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1dc41020c0 0x7f1dc419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.260+0000 7f1dc259c640 1 -- 172.21.15.71:0/4174509487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1dc4101700 msgr2=0x7f1dc4197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.260+0000 7f1dc259c640 1 --2- 172.21.15.71:0/4174509487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1dc4101700 0x7f1dc4197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.260+0000 7f1dc259c640 1 -- 172.21.15.71:0/4174509487 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc419f500 con 0x7f1dc4100510 2024-03-04T08:56:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.260+0000 7f1dc259c640 1 --2- 172.21.15.71:0/4174509487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dc4100510 0x7f1dc4197950 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00e9c0 tx=0x7f1dac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.261+0000 7f1dbb7fe640 1 -- 172.21.15.71:0/4174509487 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dac00cd80 con 0x7f1dc4100510 2024-03-04T08:56:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.261+0000 7f1dbb7fe640 1 -- 172.21.15.71:0/4174509487 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1dac0035f0 con 0x7f1dc4100510 2024-03-04T08:56:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.261+0000 7f1dc8943640 1 -- 172.21.15.71:0/4174509487 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc419f7f0 con 0x7f1dc4100510 2024-03-04T08:56:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.261+0000 7f1dbb7fe640 1 -- 172.21.15.71:0/4174509487 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dac010690 con 0x7f1dc4100510 2024-03-04T08:56:05.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.261+0000 7f1dc8943640 1 -- 172.21.15.71:0/4174509487 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc419fd10 con 0x7f1dc4100510 2024-03-04T08:56:05.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.262+0000 7f1dc8943640 1 -- 172.21.15.71:0/4174509487 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dc419ffd0 con 0x7f1dc4100510 2024-03-04T08:56:05.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.263+0000 7f1dbb7fe640 1 -- 172.21.15.71:0/4174509487 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f1dac0037a0 con 0x7f1dc4100510 2024-03-04T08:56:05.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.263+0000 7f1dbb7fe640 1 --2- 172.21.15.71:0/4174509487 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f1db00a1d80 0x7f1db00a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:05.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.263+0000 7f1dbb7fe640 1 -- 172.21.15.71:0/4174509487 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1dac0cde40 con 0x7f1dc4100510 2024-03-04T08:56:05.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.263+0000 7f1dc1d9b640 1 --2- 172.21.15.71:0/4174509487 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f1db00a1d80 0x7f1db00a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:05.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.264+0000 7f1dc1d9b640 1 --2- 172.21.15.71:0/4174509487 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f1db00a1d80 0x7f1db00a4240 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1db402c7b0 tx=0x7f1db4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:05.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.267+0000 7f1dbb7fe640 1 -- 172.21.15.71:0/4174509487 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1dac09b7a0 con 0x7f1dc4100510 2024-03-04T08:56:05.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.479+0000 7f1dc8943640 1 -- 172.21.15.71:0/4174509487 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1dc41a01a0 con 0x7f1dc4100510 2024-03-04T08:56:05.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.479+0000 7f1dbb7fe640 1 -- 172.21.15.71:0/4174509487 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3701 (secure 0 0 0) 0x7f1dac09b0a0 con 0x7f1dc4100510 2024-03-04T08:56:05.479 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:05.480 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3975379807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3975379807},{"type":"v1","addr":"172.21.15.71:6835","nonce":3975379807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1962929347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1962929347},{"type":"v1","addr":"172.21.15.71:6837","nonce":1962929347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/178179528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":178179528},{"type":"v1","addr":"172.21.15.158:6833","nonce":178179528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2468877675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2468877675},{"type":"v1","addr":"172.21.15.150:6833","nonce":2468877675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-04T08:56:05.481 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 5 2024-03-04T08:56:05.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.481+0000 7f1dc8943640 1 -- 172.21.15.71:0/4174509487 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f1db00a1d80 msgr2=0x7f1db00a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:05.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.481+0000 7f1dc8943640 1 --2- 172.21.15.71:0/4174509487 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f1db00a1d80 0x7f1db00a4240 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1db402c7b0 tx=0x7f1db4005fd0 comp rx=0 tx=0).stop 2024-03-04T08:56:05.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.481+0000 7f1dc8943640 1 -- 172.21.15.71:0/4174509487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dc4100510 msgr2=0x7f1dc4197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:05.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.481+0000 7f1dc8943640 1 --2- 172.21.15.71:0/4174509487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dc4100510 0x7f1dc4197950 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00e9c0 tx=0x7f1dac00ee80 comp rx=0 tx=0).stop 2024-03-04T08:56:05.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.481+0000 7f1dc8943640 1 -- 172.21.15.71:0/4174509487 shutdown_connections 2024-03-04T08:56:05.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.481+0000 7f1dc8943640 1 --2- 172.21.15.71:0/4174509487 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f1db00a1d80 0x7f1db00a4240 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:05.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.481+0000 7f1dc8943640 1 --2- 172.21.15.71:0/4174509487 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1dc41020c0 0x7f1dc419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:05.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.481+0000 7f1dc8943640 1 --2- 172.21.15.71:0/4174509487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1dc4101700 0x7f1dc4197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:05.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.481+0000 7f1dc8943640 1 --2- 172.21.15.71:0/4174509487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dc4100510 0x7f1dc4197950 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:05.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.481+0000 7f1dc8943640 1 -- 172.21.15.71:0/4174509487 >> 172.21.15.71:0/4174509487 conn(0x7f1dc40fba80 msgr2=0x7f1dc40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:05.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.482+0000 7f1dc8943640 1 -- 172.21.15.71:0/4174509487 shutdown_connections 2024-03-04T08:56:05.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:05.482+0000 7f1dc8943640 1 -- 172.21.15.71:0/4174509487 wait complete. 2024-03-04T08:56:05.530 INFO:journalctl@ceph.mds.e.smithi150.stdout:Mar 04 08:56:05 smithi150 systemd[1]: Starting Ceph mds.e for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T08:56:05.787 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:05 smithi150 ceph-mon[36807]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:05.787 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:05 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/4174509487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:05.787 INFO:journalctl@ceph.mds.e.smithi150.stdout:Mar 04 08:56:05 smithi150 podman[66026]: 2024-03-04 08:56:05.444383241 +0000 UTC m=+0.018053189 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T08:56:05.787 INFO:journalctl@ceph.mds.e.smithi150.stdout:Mar 04 08:56:05 smithi150 podman[66026]: 2024-03-04 08:56:05.572078187 +0000 UTC m=+0.145748119 container create 5826b805502d889431a87d966f17a3e074f9857d786370f682ded0669e3cea59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-04T08:56:05.787 INFO:journalctl@ceph.mds.e.smithi150.stdout:Mar 04 08:56:05 smithi150 podman[66026]: 2024-03-04 08:56:05.79185219 +0000 UTC m=+0.365522153 container init 5826b805502d889431a87d966f17a3e074f9857d786370f682ded0669e3cea59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True) 2024-03-04T08:56:06.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:05 smithi158 ceph-mon[37241]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:06.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:05 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/4174509487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:06.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:05 smithi071 ceph-mon[32789]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:06.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:05 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/4174509487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:06.279 INFO:journalctl@ceph.mds.e.smithi150.stdout:Mar 04 08:56:05 smithi150 podman[66026]: 2024-03-04 08:56:05.79466967 +0000 UTC m=+0.368339611 container start 5826b805502d889431a87d966f17a3e074f9857d786370f682ded0669e3cea59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-e, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-04T08:56:06.280 INFO:journalctl@ceph.mds.e.smithi150.stdout:Mar 04 08:56:05 smithi150 ceph-mds[66045]: -- 172.21.15.150:0/4037131831 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560bc45a7a40 con 0x560bc457ac00 2024-03-04T08:56:06.280 INFO:journalctl@ceph.mds.e.smithi150.stdout:Mar 04 08:56:05 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-e[66041]: starting mds.e at 2024-03-04T08:56:06.280 INFO:journalctl@ceph.mds.e.smithi150.stdout:Mar 04 08:56:05 smithi150 bash[66026]: 5826b805502d889431a87d966f17a3e074f9857d786370f682ded0669e3cea59 2024-03-04T08:56:06.280 INFO:journalctl@ceph.mds.e.smithi150.stdout:Mar 04 08:56:05 smithi150 systemd[1]: Started Ceph mds.e for b3c85c40-da03-11ee-95c6-87774f69a715. 2024-03-04T08:56:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:06 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:06 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:06 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:06 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:06 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:06 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:06 smithi150 ceph-mon[36807]: Deploying daemon mds.f on smithi158 2024-03-04T08:56:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:06 smithi150 ceph-mon[36807]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:06 smithi150 ceph-mon[36807]: mds.? [v2:172.21.15.150:6834/1802416462,v1:172.21.15.150:6835/1802416462] up:boot 2024-03-04T08:56:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:06 smithi150 ceph-mon[36807]: fsmap 5 up:standby 2024-03-04T08:56:07.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:06 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-04T08:56:07.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:06 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:07.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:06 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:07.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:06 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:07.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:06 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:07.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:06 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:07.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:06 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:07.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:06 smithi158 ceph-mon[37241]: Deploying daemon mds.f on smithi158 2024-03-04T08:56:07.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:06 smithi158 ceph-mon[37241]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:07.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:06 smithi158 ceph-mon[37241]: mds.? [v2:172.21.15.150:6834/1802416462,v1:172.21.15.150:6835/1802416462] up:boot 2024-03-04T08:56:07.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:06 smithi158 ceph-mon[37241]: fsmap 5 up:standby 2024-03-04T08:56:07.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:06 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-04T08:56:07.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:06 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:07.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:06 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:06 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:06 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:06 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:06 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:06 smithi071 ceph-mon[32789]: Deploying daemon mds.f on smithi158 2024-03-04T08:56:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:06 smithi071 ceph-mon[32789]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:06 smithi071 ceph-mon[32789]: mds.? [v2:172.21.15.150:6834/1802416462,v1:172.21.15.150:6835/1802416462] up:boot 2024-03-04T08:56:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:06 smithi071 ceph-mon[32789]: fsmap 5 up:standby 2024-03-04T08:56:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:06 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-04T08:56:07.489 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:56:07.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.615+0000 7f3271ced640 1 -- 172.21.15.71:0/3307738734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f326c1049c0 msgr2=0x7f326c106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:07.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.615+0000 7f3271ced640 1 --2- 172.21.15.71:0/3307738734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f326c1049c0 0x7f326c106db0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3260009900 tx=0x7f326002bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:07.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.615+0000 7f3271ced640 1 -- 172.21.15.71:0/3307738734 shutdown_connections 2024-03-04T08:56:07.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.615+0000 7f3271ced640 1 --2- 172.21.15.71:0/3307738734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f326c1072f0 0x7f326c109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:07.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.615+0000 7f3271ced640 1 --2- 172.21.15.71:0/3307738734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f326c1049c0 0x7f326c106db0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:07.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.615+0000 7f3271ced640 1 --2- 172.21.15.71:0/3307738734 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f326c102090 0x7f326c104480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:07.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.615+0000 7f3271ced640 1 -- 172.21.15.71:0/3307738734 >> 172.21.15.71:0/3307738734 conn(0x7f326c0fba80 msgr2=0x7f326c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:07.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.615+0000 7f3271ced640 1 -- 172.21.15.71:0/3307738734 shutdown_connections 2024-03-04T08:56:07.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.615+0000 7f3271ced640 1 -- 172.21.15.71:0/3307738734 wait complete. 2024-03-04T08:56:07.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.616+0000 7f3271ced640 1 Processor -- start 2024-03-04T08:56:07.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.616+0000 7f3271ced640 1 -- start start 2024-03-04T08:56:07.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.616+0000 7f3271ced640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f326c102090 0x7f326c1978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:07.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.616+0000 7f3271ced640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f326c1049c0 0x7f326c197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:07.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.616+0000 7f3271ced640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f326c1072f0 0x7f326c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:07.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.616+0000 7f3271ced640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c10d270 con 0x7f326c102090 2024-03-04T08:56:07.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.616+0000 7f3271ced640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c10d0f0 con 0x7f326c1072f0 2024-03-04T08:56:07.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.616+0000 7f3271ced640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c10d3f0 con 0x7f326c1049c0 2024-03-04T08:56:07.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.616+0000 7f326b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f326c102090 0x7f326c1978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:07.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.616+0000 7f326b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f326c102090 0x7f326c1978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56232/0 (socket says 172.21.15.71:56232) 2024-03-04T08:56:07.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.616+0000 7f326b7fe640 1 -- 172.21.15.71:0/3857104705 learned_addr learned my addr 172.21.15.71:0/3857104705 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:07.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.617+0000 7f326affd640 1 --2- 172.21.15.71:0/3857104705 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f326c1049c0 0x7f326c197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:07.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.617+0000 7f326bfff640 1 --2- 172.21.15.71:0/3857104705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f326c1072f0 0x7f326c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:07.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.617+0000 7f326b7fe640 1 -- 172.21.15.71:0/3857104705 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f326c1049c0 msgr2=0x7f326c197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:07.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.617+0000 7f326b7fe640 1 --2- 172.21.15.71:0/3857104705 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f326c1049c0 0x7f326c197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:07.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.617+0000 7f326b7fe640 1 -- 172.21.15.71:0/3857104705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f326c1072f0 msgr2=0x7f326c19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:07.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.617+0000 7f326b7fe640 1 --2- 172.21.15.71:0/3857104705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f326c1072f0 0x7f326c19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:07.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.617+0000 7f326b7fe640 1 -- 172.21.15.71:0/3857104705 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326c19f4d0 con 0x7f326c102090 2024-03-04T08:56:07.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.617+0000 7f326bfff640 1 --2- 172.21.15.71:0/3857104705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f326c1072f0 0x7f326c19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:56:07.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.617+0000 7f326b7fe640 1 --2- 172.21.15.71:0/3857104705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f326c102090 0x7f326c1978b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f325c00d950 tx=0x7f325c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:07.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.617+0000 7f3268ff9640 1 -- 172.21.15.71:0/3857104705 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c017070 con 0x7f326c102090 2024-03-04T08:56:07.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.617+0000 7f3268ff9640 1 -- 172.21.15.71:0/3857104705 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f325c009c30 con 0x7f326c102090 2024-03-04T08:56:07.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.617+0000 7f3271ced640 1 -- 172.21.15.71:0/3857104705 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326c19f7c0 con 0x7f326c102090 2024-03-04T08:56:07.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.617+0000 7f3268ff9640 1 -- 172.21.15.71:0/3857104705 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c00f6b0 con 0x7f326c102090 2024-03-04T08:56:07.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.617+0000 7f3271ced640 1 -- 172.21.15.71:0/3857104705 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326c19fce0 con 0x7f326c102090 2024-03-04T08:56:07.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.618+0000 7f3271ced640 1 -- 172.21.15.71:0/3857104705 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f326c0fd570 con 0x7f326c102090 2024-03-04T08:56:07.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.619+0000 7f3268ff9640 1 -- 172.21.15.71:0/3857104705 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f325c0109d0 con 0x7f326c102090 2024-03-04T08:56:07.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.620+0000 7f3268ff9640 1 --2- 172.21.15.71:0/3857104705 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f32540a1d80 0x7f32540a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:07.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.620+0000 7f326affd640 1 --2- 172.21.15.71:0/3857104705 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f32540a1d80 0x7f32540a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:07.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.620+0000 7f3268ff9640 1 -- 172.21.15.71:0/3857104705 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f325c0cd5b0 con 0x7f326c102090 2024-03-04T08:56:07.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.620+0000 7f326affd640 1 --2- 172.21.15.71:0/3857104705 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f32540a1d80 0x7f32540a4240 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3260000c00 tx=0x7f3260005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:07.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.624+0000 7f3268ff9640 1 -- 172.21.15.71:0/3857104705 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f325c009de0 con 0x7f326c102090 2024-03-04T08:56:07.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.837+0000 7f3271ced640 1 -- 172.21.15.71:0/3857104705 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f326c02cff0 con 0x7f326c102090 2024-03-04T08:56:07.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.838+0000 7f3268ff9640 1 -- 172.21.15.71:0/3857104705 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4495 (secure 0 0 0) 0x7f325c09ae80 con 0x7f326c102090 2024-03-04T08:56:07.834 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:07.834 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3975379807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3975379807},{"type":"v1","addr":"172.21.15.71:6835","nonce":3975379807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1962929347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1962929347},{"type":"v1","addr":"172.21.15.71:6837","nonce":1962929347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/178179528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":178179528},{"type":"v1","addr":"172.21.15.158:6833","nonce":178179528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2468877675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2468877675},{"type":"v1","addr":"172.21.15.150:6833","nonce":2468877675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1802416462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1802416462},{"type":"v1","addr":"172.21.15.150:6835","nonce":1802416462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-04T08:56:07.835 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 6 2024-03-04T08:56:07.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.840+0000 7f3271ced640 1 -- 172.21.15.71:0/3857104705 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f32540a1d80 msgr2=0x7f32540a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:07.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.840+0000 7f3271ced640 1 --2- 172.21.15.71:0/3857104705 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f32540a1d80 0x7f32540a4240 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3260000c00 tx=0x7f3260005fd0 comp rx=0 tx=0).stop 2024-03-04T08:56:07.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.840+0000 7f3271ced640 1 -- 172.21.15.71:0/3857104705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f326c102090 msgr2=0x7f326c1978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:07.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.840+0000 7f3271ced640 1 --2- 172.21.15.71:0/3857104705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f326c102090 0x7f326c1978b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f325c00d950 tx=0x7f325c00de10 comp rx=0 tx=0).stop 2024-03-04T08:56:07.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.840+0000 7f3271ced640 1 -- 172.21.15.71:0/3857104705 shutdown_connections 2024-03-04T08:56:07.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.840+0000 7f3271ced640 1 --2- 172.21.15.71:0/3857104705 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f32540a1d80 0x7f32540a4240 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:07.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.840+0000 7f3271ced640 1 --2- 172.21.15.71:0/3857104705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f326c1072f0 0x7f326c19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:07.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.840+0000 7f3271ced640 1 --2- 172.21.15.71:0/3857104705 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f326c1049c0 0x7f326c197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:07.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.840+0000 7f3271ced640 1 --2- 172.21.15.71:0/3857104705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f326c102090 0x7f326c1978b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:07.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.840+0000 7f3271ced640 1 -- 172.21.15.71:0/3857104705 >> 172.21.15.71:0/3857104705 conn(0x7f326c0fba80 msgr2=0x7f326c1029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:07.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.840+0000 7f3271ced640 1 -- 172.21.15.71:0/3857104705 shutdown_connections 2024-03-04T08:56:07.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:07.840+0000 7f3271ced640 1 -- 172.21.15.71:0/3857104705 wait complete. 2024-03-04T08:56:08.042 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:07 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3857104705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:08.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:07 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3857104705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:08.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:07 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3857104705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:08.302 INFO:journalctl@ceph.mds.f.smithi158.stdout:Mar 04 08:56:08 smithi158 systemd[1]: Starting Ceph mds.f for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T08:56:08.632 INFO:journalctl@ceph.mds.f.smithi158.stdout:Mar 04 08:56:08 smithi158 podman[68023]: 2024-03-04 08:56:08.391621507 +0000 UTC m=+0.020105842 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T08:56:08.632 INFO:journalctl@ceph.mds.f.smithi158.stdout:Mar 04 08:56:08 smithi158 podman[68023]: 2024-03-04 08:56:08.520703478 +0000 UTC m=+0.149187799 container create 7faadf000d2256ea0a86a355c5c7a3d956f5d60d47db2f7de3ea72fb3365a3c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-f, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222) 2024-03-04T08:56:08.976 INFO:journalctl@ceph.mds.f.smithi158.stdout:Mar 04 08:56:08 smithi158 podman[68023]: 2024-03-04 08:56:08.757541372 +0000 UTC m=+0.386025692 container init 7faadf000d2256ea0a86a355c5c7a3d956f5d60d47db2f7de3ea72fb3365a3c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-f, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T08:56:08.977 INFO:journalctl@ceph.mds.f.smithi158.stdout:Mar 04 08:56:08 smithi158 podman[68023]: 2024-03-04 08:56:08.760163054 +0000 UTC m=+0.388647379 container start 7faadf000d2256ea0a86a355c5c7a3d956f5d60d47db2f7de3ea72fb3365a3c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-f, RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T08:56:08.977 INFO:journalctl@ceph.mds.f.smithi158.stdout:Mar 04 08:56:08 smithi158 ceph-mds[68052]: -- 172.21.15.158:0/972885834 <== mon.1 v2:172.21.15.158:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f4b59f9a40 con 0x55f4b59cc800 2024-03-04T08:56:08.977 INFO:journalctl@ceph.mds.f.smithi158.stdout:Mar 04 08:56:08 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-f[68046]: starting mds.f at 2024-03-04T08:56:08.977 INFO:journalctl@ceph.mds.f.smithi158.stdout:Mar 04 08:56:08 smithi158 bash[68023]: 7faadf000d2256ea0a86a355c5c7a3d956f5d60d47db2f7de3ea72fb3365a3c1 2024-03-04T08:56:08.977 INFO:journalctl@ceph.mds.f.smithi158.stdout:Mar 04 08:56:08 smithi158 systemd[1]: Started Ceph mds.f for b3c85c40-da03-11ee-95c6-87774f69a715. 2024-03-04T08:56:09.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:08 smithi150 ceph-mon[36807]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:09.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:08 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:09.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:08 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:09.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:08 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:09.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:08 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:09.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:08 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:09.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:08 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:09.290 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:08 smithi158 ceph-mon[37241]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:09.290 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:08 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:09.290 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:08 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:09.290 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:08 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:09.290 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:08 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:09.291 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:08 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:09.291 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:08 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:09.372 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:08 smithi071 ceph-mon[32789]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:09.373 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:08 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:09.373 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:08 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:09.373 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:08 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:09.373 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:08 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:09.373 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:08 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:09.373 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:08 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:09.849 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:56:09.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.958+0000 7f47569f7640 1 -- 172.21.15.71:0/3174848374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4750107370 msgr2=0x7f4750109760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:09.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.958+0000 7f47569f7640 1 --2- 172.21.15.71:0/3174848374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4750107370 0x7f4750109760 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4748009ca0 tx=0x7f474802c0f0 comp rx=0 tx=0).stop 2024-03-04T08:56:09.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.958+0000 7f47569f7640 1 -- 172.21.15.71:0/3174848374 shutdown_connections 2024-03-04T08:56:09.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.958+0000 7f47569f7640 1 --2- 172.21.15.71:0/3174848374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4750107370 0x7f4750109760 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:09.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.958+0000 7f47569f7640 1 --2- 172.21.15.71:0/3174848374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4750104a40 0x7f4750106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:09.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.958+0000 7f47569f7640 1 --2- 172.21.15.71:0/3174848374 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4750102110 0x7f4750104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:09.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.958+0000 7f47569f7640 1 -- 172.21.15.71:0/3174848374 >> 172.21.15.71:0/3174848374 conn(0x7f47500fba60 msgr2=0x7f47500fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:09.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.958+0000 7f47569f7640 1 -- 172.21.15.71:0/3174848374 shutdown_connections 2024-03-04T08:56:09.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.958+0000 7f47569f7640 1 -- 172.21.15.71:0/3174848374 wait complete. 2024-03-04T08:56:09.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.959+0000 7f47569f7640 1 Processor -- start 2024-03-04T08:56:09.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.959+0000 7f47569f7640 1 -- start start 2024-03-04T08:56:09.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.959+0000 7f47569f7640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4750102110 0x7f47501978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:09.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.959+0000 7f47569f7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4750104a40 0x7f4750197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:09.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.959+0000 7f47569f7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4750107370 0x7f475019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:09.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.959+0000 7f47569f7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475010d280 con 0x7f4750104a40 2024-03-04T08:56:09.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.959+0000 7f47569f7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475010d100 con 0x7f4750107370 2024-03-04T08:56:09.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.959+0000 7f47569f7640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475010d400 con 0x7f4750102110 2024-03-04T08:56:09.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.959+0000 7f474f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4750104a40 0x7f4750197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:09.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.959+0000 7f474f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4750104a40 0x7f4750197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56250/0 (socket says 172.21.15.71:56250) 2024-03-04T08:56:09.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.959+0000 7f474f7fe640 1 -- 172.21.15.71:0/2196908885 learned_addr learned my addr 172.21.15.71:0/2196908885 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:09.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.960+0000 7f474ffff640 1 --2- 172.21.15.71:0/2196908885 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4750102110 0x7f47501978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:09.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.960+0000 7f4754f94640 1 --2- 172.21.15.71:0/2196908885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4750107370 0x7f475019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:09.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.960+0000 7f474f7fe640 1 -- 172.21.15.71:0/2196908885 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4750102110 msgr2=0x7f47501978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:09.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.960+0000 7f474f7fe640 1 --2- 172.21.15.71:0/2196908885 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4750102110 0x7f47501978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:09.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.960+0000 7f474f7fe640 1 -- 172.21.15.71:0/2196908885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4750107370 msgr2=0x7f475019ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:09.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.960+0000 7f474f7fe640 1 --2- 172.21.15.71:0/2196908885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4750107370 0x7f475019ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:09.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.960+0000 7f474f7fe640 1 -- 172.21.15.71:0/2196908885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475019f4c0 con 0x7f4750104a40 2024-03-04T08:56:09.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.960+0000 7f474f7fe640 1 --2- 172.21.15.71:0/2196908885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4750104a40 0x7f4750197de0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f474400d770 tx=0x7f474400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:09.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.960+0000 7f474d7fa640 1 -- 172.21.15.71:0/2196908885 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4744017070 con 0x7f4750104a40 2024-03-04T08:56:09.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.960+0000 7f474d7fa640 1 -- 172.21.15.71:0/2196908885 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4744009c30 con 0x7f4750104a40 2024-03-04T08:56:09.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.960+0000 7f47569f7640 1 -- 172.21.15.71:0/2196908885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475019f7b0 con 0x7f4750104a40 2024-03-04T08:56:09.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.960+0000 7f474d7fa640 1 -- 172.21.15.71:0/2196908885 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474400f620 con 0x7f4750104a40 2024-03-04T08:56:09.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.960+0000 7f47569f7640 1 -- 172.21.15.71:0/2196908885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475019fd00 con 0x7f4750104a40 2024-03-04T08:56:09.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.961+0000 7f47569f7640 1 -- 172.21.15.71:0/2196908885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47500fd550 con 0x7f4750104a40 2024-03-04T08:56:09.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.962+0000 7f474d7fa640 1 -- 172.21.15.71:0/2196908885 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f474401a070 con 0x7f4750104a40 2024-03-04T08:56:09.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.963+0000 7f474d7fa640 1 --2- 172.21.15.71:0/2196908885 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f47380a1d80 0x7f47380a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:09.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.963+0000 7f474ffff640 1 --2- 172.21.15.71:0/2196908885 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f47380a1d80 0x7f47380a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:09.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.963+0000 7f474d7fa640 1 -- 172.21.15.71:0/2196908885 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f47440cd360 con 0x7f4750104a40 2024-03-04T08:56:09.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.963+0000 7f474ffff640 1 --2- 172.21.15.71:0/2196908885 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f47380a1d80 0x7f47380a4240 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4740009bd0 tx=0x7f4740005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:09.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:09.968+0000 7f474d7fa640 1 -- 172.21.15.71:0/2196908885 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4744009de0 con 0x7f4750104a40 2024-03-04T08:56:10.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:09 smithi071 ceph-mon[32789]: Deploying daemon mds.c on smithi158 2024-03-04T08:56:10.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:09 smithi071 ceph-mon[32789]: mds.? [v2:172.21.15.158:6834/1652094309,v1:172.21.15.158:6835/1652094309] up:boot 2024-03-04T08:56:10.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:09 smithi071 ceph-mon[32789]: fsmap 6 up:standby 2024-03-04T08:56:10.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:09 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-04T08:56:10.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:10.180+0000 7f47569f7640 1 -- 172.21.15.71:0/2196908885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f475002d240 con 0x7f4750104a40 2024-03-04T08:56:10.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:10.181+0000 7f474d7fa640 1 -- 172.21.15.71:0/2196908885 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5289 (secure 0 0 0) 0x7f474409a5e0 con 0x7f4750104a40 2024-03-04T08:56:10.177 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:10.177 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3975379807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3975379807},{"type":"v1","addr":"172.21.15.71:6835","nonce":3975379807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1962929347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1962929347},{"type":"v1","addr":"172.21.15.71:6837","nonce":1962929347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/178179528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":178179528},{"type":"v1","addr":"172.21.15.158:6833","nonce":178179528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2468877675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2468877675},{"type":"v1","addr":"172.21.15.150:6833","nonce":2468877675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1652094309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1652094309},{"type":"v1","addr":"172.21.15.158:6835","nonce":1652094309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1802416462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1802416462},{"type":"v1","addr":"172.21.15.150:6835","nonce":1802416462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-04T08:56:10.177 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 7 2024-03-04T08:56:10.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:10.183+0000 7f47569f7640 1 -- 172.21.15.71:0/2196908885 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f47380a1d80 msgr2=0x7f47380a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:10.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:10.183+0000 7f47569f7640 1 --2- 172.21.15.71:0/2196908885 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f47380a1d80 0x7f47380a4240 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4740009bd0 tx=0x7f4740005f90 comp rx=0 tx=0).stop 2024-03-04T08:56:10.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:10.183+0000 7f47569f7640 1 -- 172.21.15.71:0/2196908885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4750104a40 msgr2=0x7f4750197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:10.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:10.183+0000 7f47569f7640 1 --2- 172.21.15.71:0/2196908885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4750104a40 0x7f4750197de0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f474400d770 tx=0x7f474400dc30 comp rx=0 tx=0).stop 2024-03-04T08:56:10.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:10.183+0000 7f47569f7640 1 -- 172.21.15.71:0/2196908885 shutdown_connections 2024-03-04T08:56:10.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:10.183+0000 7f47569f7640 1 --2- 172.21.15.71:0/2196908885 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f47380a1d80 0x7f47380a4240 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:10.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:10.183+0000 7f47569f7640 1 --2- 172.21.15.71:0/2196908885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4750107370 0x7f475019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:10.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:10.183+0000 7f47569f7640 1 --2- 172.21.15.71:0/2196908885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4750104a40 0x7f4750197de0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:10.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:10.183+0000 7f47569f7640 1 --2- 172.21.15.71:0/2196908885 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4750102110 0x7f47501978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:10.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:10.183+0000 7f47569f7640 1 -- 172.21.15.71:0/2196908885 >> 172.21.15.71:0/2196908885 conn(0x7f47500fba60 msgr2=0x7f4750102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:10.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:10.183+0000 7f47569f7640 1 -- 172.21.15.71:0/2196908885 shutdown_connections 2024-03-04T08:56:10.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:10.183+0000 7f47569f7640 1 -- 172.21.15.71:0/2196908885 wait complete. 2024-03-04T08:56:10.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:09 smithi150 ceph-mon[36807]: Deploying daemon mds.c on smithi158 2024-03-04T08:56:10.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:09 smithi150 ceph-mon[36807]: mds.? [v2:172.21.15.158:6834/1652094309,v1:172.21.15.158:6835/1652094309] up:boot 2024-03-04T08:56:10.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:09 smithi150 ceph-mon[36807]: fsmap 6 up:standby 2024-03-04T08:56:10.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:09 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-04T08:56:10.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:09 smithi158 ceph-mon[37241]: Deploying daemon mds.c on smithi158 2024-03-04T08:56:10.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:09 smithi158 ceph-mon[37241]: mds.? [v2:172.21.15.158:6834/1652094309,v1:172.21.15.158:6835/1652094309] up:boot 2024-03-04T08:56:10.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:09 smithi158 ceph-mon[37241]: fsmap 6 up:standby 2024-03-04T08:56:10.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:09 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-04T08:56:11.273 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:10 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2196908885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:11.273 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:10 smithi158 ceph-mon[37241]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:11.274 INFO:journalctl@ceph.mds.c.smithi158.stdout:Mar 04 08:56:11 smithi158 systemd[1]: Starting Ceph mds.c for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T08:56:11.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:10 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2196908885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:11.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:10 smithi150 ceph-mon[36807]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:11.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:10 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2196908885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:11.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:10 smithi071 ceph-mon[32789]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:11.543 INFO:journalctl@ceph.mds.c.smithi158.stdout:Mar 04 08:56:11 smithi158 podman[68338]: 2024-03-04 08:56:11.181161417 +0000 UTC m=+0.018073676 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T08:56:11.543 INFO:journalctl@ceph.mds.c.smithi158.stdout:Mar 04 08:56:11 smithi158 podman[68338]: 2024-03-04 08:56:11.303011063 +0000 UTC m=+0.139923302 container create 78725097cb8497e692b468275028a4e7f6461cd8a60318ed8b3a1858f3a8541e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-c, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T08:56:11.544 INFO:journalctl@ceph.mds.c.smithi158.stdout:Mar 04 08:56:11 smithi158 podman[68338]: 2024-03-04 08:56:11.548166654 +0000 UTC m=+0.385078900 container init 78725097cb8497e692b468275028a4e7f6461cd8a60318ed8b3a1858f3a8541e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-04T08:56:11.802 INFO:journalctl@ceph.mds.c.smithi158.stdout:Mar 04 08:56:11 smithi158 podman[68338]: 2024-03-04 08:56:11.551339579 +0000 UTC m=+0.388251832 container start 78725097cb8497e692b468275028a4e7f6461cd8a60318ed8b3a1858f3a8541e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-1bc250c, maintainer=Guillaume Abrioux ) 2024-03-04T08:56:11.803 INFO:journalctl@ceph.mds.c.smithi158.stdout:Mar 04 08:56:11 smithi158 ceph-mds[68357]: -- 172.21.15.158:0/512439164 <== mon.1 v2:172.21.15.158:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e0cb073a40 con 0x55e0cb046800 2024-03-04T08:56:11.803 INFO:journalctl@ceph.mds.c.smithi158.stdout:Mar 04 08:56:11 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-c[68353]: starting mds.c at 2024-03-04T08:56:11.803 INFO:journalctl@ceph.mds.c.smithi158.stdout:Mar 04 08:56:11 smithi158 bash[68338]: 78725097cb8497e692b468275028a4e7f6461cd8a60318ed8b3a1858f3a8541e 2024-03-04T08:56:11.803 INFO:journalctl@ceph.mds.c.smithi158.stdout:Mar 04 08:56:11 smithi158 systemd[1]: Started Ceph mds.c for b3c85c40-da03-11ee-95c6-87774f69a715. 2024-03-04T08:56:12.190 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:56:12.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.308+0000 7f51218e1640 1 -- 172.21.15.71:0/2452308381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f511c1004f0 msgr2=0x7f511c100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:12.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.308+0000 7f51218e1640 1 --2- 172.21.15.71:0/2452308381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f511c1004f0 0x7f511c100910 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5110009900 tx=0x7f511002bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:12.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.308+0000 7f51218e1640 1 -- 172.21.15.71:0/2452308381 shutdown_connections 2024-03-04T08:56:12.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.308+0000 7f51218e1640 1 --2- 172.21.15.71:0/2452308381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f511c1020a0 0x7f511c108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:12.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.308+0000 7f51218e1640 1 --2- 172.21.15.71:0/2452308381 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f511c1016e0 0x7f511c101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:12.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.308+0000 7f51218e1640 1 --2- 172.21.15.71:0/2452308381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f511c1004f0 0x7f511c100910 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:12.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.308+0000 7f51218e1640 1 -- 172.21.15.71:0/2452308381 >> 172.21.15.71:0/2452308381 conn(0x7f511c0fba80 msgr2=0x7f511c0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:12.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.308+0000 7f51218e1640 1 -- 172.21.15.71:0/2452308381 shutdown_connections 2024-03-04T08:56:12.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.308+0000 7f51218e1640 1 -- 172.21.15.71:0/2452308381 wait complete. 2024-03-04T08:56:12.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.308+0000 7f51218e1640 1 Processor -- start 2024-03-04T08:56:12.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.308+0000 7f51218e1640 1 -- start start 2024-03-04T08:56:12.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.309+0000 7f51218e1640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f511c1004f0 0x7f511c06a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:12.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.309+0000 7f51218e1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f511c1016e0 0x7f511c19c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:12.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.309+0000 7f51218e1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f511c1020a0 0x7f511c19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:12.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.309+0000 7f51218e1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c10c180 con 0x7f511c1016e0 2024-03-04T08:56:12.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.309+0000 7f51218e1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c10c000 con 0x7f511c1020a0 2024-03-04T08:56:12.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.309+0000 7f51218e1640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c10c300 con 0x7f511c1004f0 2024-03-04T08:56:12.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.309+0000 7f511a7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f511c1016e0 0x7f511c19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:12.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.309+0000 7f511a7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f511c1016e0 0x7f511c19c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56272/0 (socket says 172.21.15.71:56272) 2024-03-04T08:56:12.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.309+0000 7f511a7fc640 1 -- 172.21.15.71:0/2222371739 learned_addr learned my addr 172.21.15.71:0/2222371739 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:12.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.309+0000 7f511b7fe640 1 --2- 172.21.15.71:0/2222371739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f511c1020a0 0x7f511c19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:12.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.309+0000 7f511affd640 1 --2- 172.21.15.71:0/2222371739 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f511c1004f0 0x7f511c06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:12.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.309+0000 7f511a7fc640 1 -- 172.21.15.71:0/2222371739 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f511c1004f0 msgr2=0x7f511c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:12.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.309+0000 7f511a7fc640 1 --2- 172.21.15.71:0/2222371739 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f511c1004f0 0x7f511c06a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:12.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.309+0000 7f511a7fc640 1 -- 172.21.15.71:0/2222371739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f511c1020a0 msgr2=0x7f511c19ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:12.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.309+0000 7f511a7fc640 1 --2- 172.21.15.71:0/2222371739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f511c1020a0 0x7f511c19ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:12.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.309+0000 7f511a7fc640 1 -- 172.21.15.71:0/2222371739 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511c19f580 con 0x7f511c1016e0 2024-03-04T08:56:12.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.309+0000 7f511a7fc640 1 --2- 172.21.15.71:0/2222371739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f511c1016e0 0x7f511c19c650 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f510c00e9f0 tx=0x7f510c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:12.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.309+0000 7f51208df640 1 -- 172.21.15.71:0/2222371739 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c00cd80 con 0x7f511c1016e0 2024-03-04T08:56:12.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.309+0000 7f51208df640 1 -- 172.21.15.71:0/2222371739 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f510c005500 con 0x7f511c1016e0 2024-03-04T08:56:12.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.309+0000 7f51218e1640 1 -- 172.21.15.71:0/2222371739 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f511c19f870 con 0x7f511c1016e0 2024-03-04T08:56:12.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.309+0000 7f51208df640 1 -- 172.21.15.71:0/2222371739 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c010690 con 0x7f511c1016e0 2024-03-04T08:56:12.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.309+0000 7f51218e1640 1 -- 172.21.15.71:0/2222371739 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511c19fd70 con 0x7f511c1016e0 2024-03-04T08:56:12.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.310+0000 7f51218e1640 1 -- 172.21.15.71:0/2222371739 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f511c107890 con 0x7f511c1016e0 2024-03-04T08:56:12.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.311+0000 7f51208df640 1 -- 172.21.15.71:0/2222371739 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f510c0046c0 con 0x7f511c1016e0 2024-03-04T08:56:12.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.312+0000 7f51208df640 1 --2- 172.21.15.71:0/2222371739 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f51040a1d80 0x7f51040a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:12.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.312+0000 7f511affd640 1 --2- 172.21.15.71:0/2222371739 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f51040a1d80 0x7f51040a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:12.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.312+0000 7f51208df640 1 -- 172.21.15.71:0/2222371739 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f510c0cdd00 con 0x7f511c1016e0 2024-03-04T08:56:12.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.312+0000 7f511affd640 1 --2- 172.21.15.71:0/2222371739 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f51040a1d80 0x7f51040a4240 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5110000c00 tx=0x7f5110005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:12.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.316+0000 7f51208df640 1 -- 172.21.15.71:0/2222371739 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f510c09b5d0 con 0x7f511c1016e0 2024-03-04T08:56:12.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.528+0000 7f51218e1640 1 -- 172.21.15.71:0/2222371739 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f511c02cff0 con 0x7f511c1016e0 2024-03-04T08:56:12.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.528+0000 7f51208df640 1 -- 172.21.15.71:0/2222371739 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6080 (secure 0 0 0) 0x7f510c09aed0 con 0x7f511c1016e0 2024-03-04T08:56:12.524 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:12.525 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3975379807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3975379807},{"type":"v1","addr":"172.21.15.71:6835","nonce":3975379807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1962929347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1962929347},{"type":"v1","addr":"172.21.15.71:6837","nonce":1962929347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/178179528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":178179528},{"type":"v1","addr":"172.21.15.158:6833","nonce":178179528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2468877675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2468877675},{"type":"v1","addr":"172.21.15.150:6833","nonce":2468877675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1652094309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1652094309},{"type":"v1","addr":"172.21.15.158:6835","nonce":1652094309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1802416462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1802416462},{"type":"v1","addr":"172.21.15.150:6835","nonce":1802416462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/617058380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":617058380},{"type":"v1","addr":"172.21.15.158:6837","nonce":617058380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-04T08:56:12.525 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 8 2024-03-04T08:56:12.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.530+0000 7f51218e1640 1 -- 172.21.15.71:0/2222371739 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f51040a1d80 msgr2=0x7f51040a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:12.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.530+0000 7f51218e1640 1 --2- 172.21.15.71:0/2222371739 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f51040a1d80 0x7f51040a4240 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5110000c00 tx=0x7f5110005fd0 comp rx=0 tx=0).stop 2024-03-04T08:56:12.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.530+0000 7f51218e1640 1 -- 172.21.15.71:0/2222371739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f511c1016e0 msgr2=0x7f511c19c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:12.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.530+0000 7f51218e1640 1 --2- 172.21.15.71:0/2222371739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f511c1016e0 0x7f511c19c650 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f510c00e9f0 tx=0x7f510c00edf0 comp rx=0 tx=0).stop 2024-03-04T08:56:12.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.530+0000 7f51218e1640 1 -- 172.21.15.71:0/2222371739 shutdown_connections 2024-03-04T08:56:12.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.530+0000 7f51218e1640 1 --2- 172.21.15.71:0/2222371739 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f51040a1d80 0x7f51040a4240 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:12.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.530+0000 7f51218e1640 1 --2- 172.21.15.71:0/2222371739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f511c1020a0 0x7f511c19ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:12.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.530+0000 7f51218e1640 1 --2- 172.21.15.71:0/2222371739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f511c1016e0 0x7f511c19c650 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:12.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.530+0000 7f51218e1640 1 --2- 172.21.15.71:0/2222371739 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f511c1004f0 0x7f511c06a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:12.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.530+0000 7f51218e1640 1 -- 172.21.15.71:0/2222371739 >> 172.21.15.71:0/2222371739 conn(0x7f511c0fba80 msgr2=0x7f511c104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:12.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.530+0000 7f51218e1640 1 -- 172.21.15.71:0/2222371739 shutdown_connections 2024-03-04T08:56:12.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:12.530+0000 7f51218e1640 1 -- 172.21.15.71:0/2222371739 wait complete. 2024-03-04T08:56:12.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:12 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:12.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:12 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:12.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:12 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:12.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:12 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:12.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:12 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:12.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:12 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:12.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:12 smithi071 ceph-mon[32789]: Deploying daemon mds.h on smithi150 2024-03-04T08:56:12.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:12 smithi071 ceph-mon[32789]: mds.? [v2:172.21.15.158:6836/617058380,v1:172.21.15.158:6837/617058380] up:boot 2024-03-04T08:56:12.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:12 smithi071 ceph-mon[32789]: fsmap 7 up:standby 2024-03-04T08:56:12.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:12 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-04T08:56:12.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:12 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2222371739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:13.020 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:12 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:13.020 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:12 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:13.020 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:12 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:13.020 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:12 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:13.021 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:12 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:13.021 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:12 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:13.021 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:12 smithi150 ceph-mon[36807]: Deploying daemon mds.h on smithi150 2024-03-04T08:56:13.021 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:12 smithi150 ceph-mon[36807]: mds.? [v2:172.21.15.158:6836/617058380,v1:172.21.15.158:6837/617058380] up:boot 2024-03-04T08:56:13.021 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:12 smithi150 ceph-mon[36807]: fsmap 7 up:standby 2024-03-04T08:56:13.021 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:12 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-04T08:56:13.021 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:12 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2222371739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:13.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:12 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:13.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:12 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:13.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:12 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:13.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:12 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:13.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:12 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:13.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:12 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:13.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:12 smithi158 ceph-mon[37241]: Deploying daemon mds.h on smithi150 2024-03-04T08:56:13.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:12 smithi158 ceph-mon[37241]: mds.? [v2:172.21.15.158:6836/617058380,v1:172.21.15.158:6837/617058380] up:boot 2024-03-04T08:56:13.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:12 smithi158 ceph-mon[37241]: fsmap 7 up:standby 2024-03-04T08:56:13.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:12 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-04T08:56:13.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:12 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2222371739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:14.007 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:13 smithi150 ceph-mon[36807]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:14.007 INFO:journalctl@ceph.mds.h.smithi150.stdout:Mar 04 08:56:13 smithi150 systemd[1]: Starting Ceph mds.h for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T08:56:14.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:13 smithi158 ceph-mon[37241]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:14.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:13 smithi071 ceph-mon[32789]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:14.267 INFO:journalctl@ceph.mds.h.smithi150.stdout:Mar 04 08:56:14 smithi150 podman[66332]: 2024-03-04 08:56:13.914649818 +0000 UTC m=+0.017898081 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T08:56:14.267 INFO:journalctl@ceph.mds.h.smithi150.stdout:Mar 04 08:56:14 smithi150 podman[66332]: 2024-03-04 08:56:14.027470369 +0000 UTC m=+0.130718627 container create 284af29d0072c7fc1735166e1080b86dd43a35fe2ebf17038693741749914b57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-h, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T08:56:14.267 INFO:journalctl@ceph.mds.h.smithi150.stdout:Mar 04 08:56:14 smithi150 podman[66332]: 2024-03-04 08:56:14.272035678 +0000 UTC m=+0.375283946 container init 284af29d0072c7fc1735166e1080b86dd43a35fe2ebf17038693741749914b57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T08:56:14.529 INFO:journalctl@ceph.mds.h.smithi150.stdout:Mar 04 08:56:14 smithi150 podman[66332]: 2024-03-04 08:56:14.274890805 +0000 UTC m=+0.378139083 container start 284af29d0072c7fc1735166e1080b86dd43a35fe2ebf17038693741749914b57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-h, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-04T08:56:14.530 INFO:journalctl@ceph.mds.h.smithi150.stdout:Mar 04 08:56:14 smithi150 ceph-mds[66351]: -- 172.21.15.150:0/2645935181 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e58e1a3a40 con 0x55e58e177000 2024-03-04T08:56:14.530 INFO:journalctl@ceph.mds.h.smithi150.stdout:Mar 04 08:56:14 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-h[66347]: starting mds.h at 2024-03-04T08:56:14.530 INFO:journalctl@ceph.mds.h.smithi150.stdout:Mar 04 08:56:14 smithi150 bash[66332]: 284af29d0072c7fc1735166e1080b86dd43a35fe2ebf17038693741749914b57 2024-03-04T08:56:14.530 INFO:journalctl@ceph.mds.h.smithi150.stdout:Mar 04 08:56:14 smithi150 systemd[1]: Started Ceph mds.h for b3c85c40-da03-11ee-95c6-87774f69a715. 2024-03-04T08:56:14.538 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:56:14.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.653+0000 7f09d1202640 1 -- 172.21.15.71:0/1151052369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09cc1027e0 msgr2=0x7f09cc102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:14.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.653+0000 7f09d1202640 1 --2- 172.21.15.71:0/1151052369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09cc1027e0 0x7f09cc102c60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009900 tx=0x7f09b402bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:14.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.654+0000 7f09d1202640 1 -- 172.21.15.71:0/1151052369 shutdown_connections 2024-03-04T08:56:14.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.654+0000 7f09d1202640 1 --2- 172.21.15.71:0/1151052369 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09cc1031a0 0x7f09cc109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:14.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.654+0000 7f09d1202640 1 --2- 172.21.15.71:0/1151052369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09cc1027e0 0x7f09cc102c60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:14.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.654+0000 7f09d1202640 1 --2- 172.21.15.71:0/1151052369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09cc1015f0 0x7f09cc101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:14.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.654+0000 7f09d1202640 1 -- 172.21.15.71:0/1151052369 >> 172.21.15.71:0/1151052369 conn(0x7f09cc0fcb80 msgr2=0x7f09cc0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:14.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.654+0000 7f09d1202640 1 -- 172.21.15.71:0/1151052369 shutdown_connections 2024-03-04T08:56:14.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.654+0000 7f09d1202640 1 -- 172.21.15.71:0/1151052369 wait complete. 2024-03-04T08:56:14.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.654+0000 7f09d1202640 1 Processor -- start 2024-03-04T08:56:14.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.654+0000 7f09d1202640 1 -- start start 2024-03-04T08:56:14.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.654+0000 7f09d1202640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09cc1015f0 0x7f09cc197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:14.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.654+0000 7f09d1202640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09cc1027e0 0x7f09cc197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:14.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.654+0000 7f09d1202640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09cc1031a0 0x7f09cc19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:14.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.654+0000 7f09d1202640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc10d250 con 0x7f09cc1031a0 2024-03-04T08:56:14.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.654+0000 7f09d1202640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc10d0d0 con 0x7f09cc1015f0 2024-03-04T08:56:14.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.654+0000 7f09d1202640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc10d3d0 con 0x7f09cc1027e0 2024-03-04T08:56:14.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.655+0000 7f09cb59e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09cc1031a0 0x7f09cc19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:14.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.655+0000 7f09cb59e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09cc1031a0 0x7f09cc19ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56294/0 (socket says 172.21.15.71:56294) 2024-03-04T08:56:14.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.655+0000 7f09cb59e640 1 -- 172.21.15.71:0/2976403657 learned_addr learned my addr 172.21.15.71:0/2976403657 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:14.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.655+0000 7f09c3fff640 1 --2- 172.21.15.71:0/2976403657 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09cc1027e0 0x7f09cc197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:14.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.655+0000 7f09cad9d640 1 --2- 172.21.15.71:0/2976403657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09cc1015f0 0x7f09cc197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:14.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.655+0000 7f09cb59e640 1 -- 172.21.15.71:0/2976403657 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09cc1027e0 msgr2=0x7f09cc197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:14.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.655+0000 7f09cb59e640 1 --2- 172.21.15.71:0/2976403657 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09cc1027e0 0x7f09cc197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:14.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.655+0000 7f09cb59e640 1 -- 172.21.15.71:0/2976403657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09cc1015f0 msgr2=0x7f09cc197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:14.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.655+0000 7f09cb59e640 1 --2- 172.21.15.71:0/2976403657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09cc1015f0 0x7f09cc197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:14.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.655+0000 7f09cb59e640 1 -- 172.21.15.71:0/2976403657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09cc19f4b0 con 0x7f09cc1031a0 2024-03-04T08:56:14.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.655+0000 7f09cb59e640 1 --2- 172.21.15.71:0/2976403657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09cc1031a0 0x7f09cc19ee40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009fd0 tx=0x7f09c400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:14.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.655+0000 7f09c8d99640 1 -- 172.21.15.71:0/2976403657 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c4017070 con 0x7f09cc1031a0 2024-03-04T08:56:14.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.655+0000 7f09c8d99640 1 -- 172.21.15.71:0/2976403657 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09c4003e50 con 0x7f09cc1031a0 2024-03-04T08:56:14.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.655+0000 7f09c8d99640 1 -- 172.21.15.71:0/2976403657 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c40106c0 con 0x7f09cc1031a0 2024-03-04T08:56:14.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.655+0000 7f09d1202640 1 -- 172.21.15.71:0/2976403657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09cc19f7a0 con 0x7f09cc1031a0 2024-03-04T08:56:14.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.655+0000 7f09d1202640 1 -- 172.21.15.71:0/2976403657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09cc19fc70 con 0x7f09cc1031a0 2024-03-04T08:56:14.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.656+0000 7f09d1202640 1 -- 172.21.15.71:0/2976403657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09cc1088b0 con 0x7f09cc1031a0 2024-03-04T08:56:14.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.657+0000 7f09c8d99640 1 -- 172.21.15.71:0/2976403657 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f09c4003970 con 0x7f09cc1031a0 2024-03-04T08:56:14.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.658+0000 7f09c8d99640 1 --2- 172.21.15.71:0/2976403657 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f09b80a1d80 0x7f09b80a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:14.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.658+0000 7f09cad9d640 1 --2- 172.21.15.71:0/2976403657 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f09b80a1d80 0x7f09b80a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:14.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.658+0000 7f09c8d99640 1 -- 172.21.15.71:0/2976403657 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f09c40cd6e0 con 0x7f09cc1031a0 2024-03-04T08:56:14.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.659+0000 7f09cad9d640 1 --2- 172.21.15.71:0/2976403657 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f09b80a1d80 0x7f09b80a4240 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f09bc0058c0 tx=0x7f09bc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:14.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.663+0000 7f09c8d99640 1 -- 172.21.15.71:0/2976403657 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f09c409afb0 con 0x7f09cc1031a0 2024-03-04T08:56:14.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.874+0000 7f09d1202640 1 -- 172.21.15.71:0/2976403657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09cc0621d0 con 0x7f09cc1031a0 2024-03-04T08:56:14.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.875+0000 7f09c8d99640 1 -- 172.21.15.71:0/2976403657 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6871 (secure 0 0 0) 0x7f09c409a8b0 con 0x7f09cc1031a0 2024-03-04T08:56:14.873 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:14.873 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3975379807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3975379807},{"type":"v1","addr":"172.21.15.71:6835","nonce":3975379807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1962929347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1962929347},{"type":"v1","addr":"172.21.15.71:6837","nonce":1962929347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/178179528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":178179528},{"type":"v1","addr":"172.21.15.158:6833","nonce":178179528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2468877675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2468877675},{"type":"v1","addr":"172.21.15.150:6833","nonce":2468877675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1652094309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1652094309},{"type":"v1","addr":"172.21.15.158:6835","nonce":1652094309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1802416462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1802416462},{"type":"v1","addr":"172.21.15.150:6835","nonce":1802416462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/617058380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":617058380},{"type":"v1","addr":"172.21.15.158:6837","nonce":617058380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/913465600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":913465600},{"type":"v1","addr":"172.21.15.150:6837","nonce":913465600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-04T08:56:14.874 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 9 2024-03-04T08:56:14.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.877+0000 7f09d1202640 1 -- 172.21.15.71:0/2976403657 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f09b80a1d80 msgr2=0x7f09b80a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:14.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.877+0000 7f09d1202640 1 --2- 172.21.15.71:0/2976403657 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f09b80a1d80 0x7f09b80a4240 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f09bc0058c0 tx=0x7f09bc005850 comp rx=0 tx=0).stop 2024-03-04T08:56:14.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.877+0000 7f09d1202640 1 -- 172.21.15.71:0/2976403657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09cc1031a0 msgr2=0x7f09cc19ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:14.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.877+0000 7f09d1202640 1 --2- 172.21.15.71:0/2976403657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09cc1031a0 0x7f09cc19ee40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009fd0 tx=0x7f09c400b260 comp rx=0 tx=0).stop 2024-03-04T08:56:14.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.877+0000 7f09d1202640 1 -- 172.21.15.71:0/2976403657 shutdown_connections 2024-03-04T08:56:14.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.877+0000 7f09d1202640 1 --2- 172.21.15.71:0/2976403657 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f09b80a1d80 0x7f09b80a4240 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:14.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.877+0000 7f09d1202640 1 --2- 172.21.15.71:0/2976403657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09cc1031a0 0x7f09cc19ee40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:14.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.877+0000 7f09d1202640 1 --2- 172.21.15.71:0/2976403657 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f09cc1027e0 0x7f09cc197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:14.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.877+0000 7f09d1202640 1 --2- 172.21.15.71:0/2976403657 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09cc1015f0 0x7f09cc197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:14.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.877+0000 7f09d1202640 1 -- 172.21.15.71:0/2976403657 >> 172.21.15.71:0/2976403657 conn(0x7f09cc0fcb80 msgr2=0x7f09cc105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:14.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.877+0000 7f09d1202640 1 -- 172.21.15.71:0/2976403657 shutdown_connections 2024-03-04T08:56:14.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:14.877+0000 7f09d1202640 1 -- 172.21.15.71:0/2976403657 wait complete. 2024-03-04T08:56:15.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:15 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:15.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:15 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:15.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:15 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:15.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:15 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:15.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:15 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:15.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:15 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:15.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:15 smithi071 ceph-mon[32789]: Deploying daemon mds.i on smithi158 2024-03-04T08:56:15.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:15 smithi071 ceph-mon[32789]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:15.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:15 smithi071 ceph-mon[32789]: mds.? [v2:172.21.15.150:6836/913465600,v1:172.21.15.150:6837/913465600] up:boot 2024-03-04T08:56:15.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:15 smithi071 ceph-mon[32789]: fsmap 8 up:standby 2024-03-04T08:56:15.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:15 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-04T08:56:15.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:15 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2976403657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:15.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:15 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:15.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:15 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:15.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:15 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:15.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:15 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:15.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:15 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:15.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:15 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:15.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:15 smithi150 ceph-mon[36807]: Deploying daemon mds.i on smithi158 2024-03-04T08:56:15.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:15 smithi150 ceph-mon[36807]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:15.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:15 smithi150 ceph-mon[36807]: mds.? [v2:172.21.15.150:6836/913465600,v1:172.21.15.150:6837/913465600] up:boot 2024-03-04T08:56:15.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:15 smithi150 ceph-mon[36807]: fsmap 8 up:standby 2024-03-04T08:56:15.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:15 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-04T08:56:15.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:15 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2976403657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:15.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:15.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:15.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:15.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:15.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:15.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:15.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:15 smithi158 ceph-mon[37241]: Deploying daemon mds.i on smithi158 2024-03-04T08:56:15.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:15 smithi158 ceph-mon[37241]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:15.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:15 smithi158 ceph-mon[37241]: mds.? [v2:172.21.15.150:6836/913465600,v1:172.21.15.150:6837/913465600] up:boot 2024-03-04T08:56:15.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:15 smithi158 ceph-mon[37241]: fsmap 8 up:standby 2024-03-04T08:56:15.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:15 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-04T08:56:15.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:15 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2976403657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:16.884 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:56:16.967 INFO:journalctl@ceph.mds.i.smithi158.stdout:Mar 04 08:56:16 smithi158 systemd[1]: Starting Ceph mds.i for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T08:56:17.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.003+0000 7fd96f70c640 1 -- 172.21.15.71:0/3628970685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9681020a0 msgr2=0x7fd968108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:17.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.003+0000 7fd96f70c640 1 --2- 172.21.15.71:0/3628970685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9681020a0 0x7fd968108930 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd960009900 tx=0x7fd96002bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:17.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.003+0000 7fd96f70c640 1 -- 172.21.15.71:0/3628970685 shutdown_connections 2024-03-04T08:56:17.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.003+0000 7fd96f70c640 1 --2- 172.21.15.71:0/3628970685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9681020a0 0x7fd968108930 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:17.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.003+0000 7fd96f70c640 1 --2- 172.21.15.71:0/3628970685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9681016e0 0x7fd968101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:17.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.003+0000 7fd96f70c640 1 --2- 172.21.15.71:0/3628970685 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd9681004f0 0x7fd968100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:17.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.003+0000 7fd96f70c640 1 -- 172.21.15.71:0/3628970685 >> 172.21.15.71:0/3628970685 conn(0x7fd9680fba60 msgr2=0x7fd9680fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:17.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.003+0000 7fd96f70c640 1 -- 172.21.15.71:0/3628970685 shutdown_connections 2024-03-04T08:56:17.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.003+0000 7fd96f70c640 1 -- 172.21.15.71:0/3628970685 wait complete. 2024-03-04T08:56:17.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.003+0000 7fd96f70c640 1 Processor -- start 2024-03-04T08:56:17.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.003+0000 7fd96f70c640 1 -- start start 2024-03-04T08:56:17.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.003+0000 7fd96f70c640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd9681004f0 0x7fd9681978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:17.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.003+0000 7fd96f70c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9681016e0 0x7fd968197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:17.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.003+0000 7fd96f70c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9681020a0 0x7fd96819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:17.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.003+0000 7fd96f70c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96810c180 con 0x7fd9681020a0 2024-03-04T08:56:17.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.003+0000 7fd96f70c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96810c000 con 0x7fd9681016e0 2024-03-04T08:56:17.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.003+0000 7fd96f70c640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96810c300 con 0x7fd9681004f0 2024-03-04T08:56:17.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.004+0000 7fd96dca9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9681020a0 0x7fd96819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:17.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.004+0000 7fd96dca9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9681020a0 0x7fd96819eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55762/0 (socket says 172.21.15.71:55762) 2024-03-04T08:56:17.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.004+0000 7fd96d4a8640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd9681004f0 0x7fd9681978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:17.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.004+0000 7fd96dca9640 1 -- 172.21.15.71:0/1403024396 learned_addr learned my addr 172.21.15.71:0/1403024396 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:17.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.004+0000 7fd96cca7640 1 --2- 172.21.15.71:0/1403024396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9681016e0 0x7fd968197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:17.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.004+0000 7fd96dca9640 1 -- 172.21.15.71:0/1403024396 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd9681004f0 msgr2=0x7fd9681978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:17.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.004+0000 7fd96dca9640 1 --2- 172.21.15.71:0/1403024396 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd9681004f0 0x7fd9681978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:17.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.004+0000 7fd96dca9640 1 -- 172.21.15.71:0/1403024396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9681016e0 msgr2=0x7fd968197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:17.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.004+0000 7fd96dca9640 1 --2- 172.21.15.71:0/1403024396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9681016e0 0x7fd968197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:17.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.004+0000 7fd96dca9640 1 -- 172.21.15.71:0/1403024396 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96819f510 con 0x7fd9681020a0 2024-03-04T08:56:17.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.004+0000 7fd96d4a8640 1 --2- 172.21.15.71:0/1403024396 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd9681004f0 0x7fd9681978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:56:17.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.004+0000 7fd96dca9640 1 --2- 172.21.15.71:0/1403024396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9681020a0 0x7fd96819eea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd96002c7b0 tx=0x7fd960005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:17.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.004+0000 7fd95e7fc640 1 -- 172.21.15.71:0/1403024396 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd960041070 con 0x7fd9681020a0 2024-03-04T08:56:17.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.004+0000 7fd96f70c640 1 -- 172.21.15.71:0/1403024396 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd96819f7a0 con 0x7fd9681020a0 2024-03-04T08:56:17.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.004+0000 7fd95e7fc640 1 -- 172.21.15.71:0/1403024396 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9600054e0 con 0x7fd9681020a0 2024-03-04T08:56:17.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.004+0000 7fd95e7fc640 1 -- 172.21.15.71:0/1403024396 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd960034440 con 0x7fd9681020a0 2024-03-04T08:56:17.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.004+0000 7fd96f70c640 1 -- 172.21.15.71:0/1403024396 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd96819fce0 con 0x7fd9681020a0 2024-03-04T08:56:17.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.005+0000 7fd96f70c640 1 -- 172.21.15.71:0/1403024396 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd96819ffa0 con 0x7fd9681020a0 2024-03-04T08:56:17.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.006+0000 7fd95e7fc640 1 -- 172.21.15.71:0/1403024396 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fd960044070 con 0x7fd9681020a0 2024-03-04T08:56:17.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.007+0000 7fd95e7fc640 1 --2- 172.21.15.71:0/1403024396 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd9540a1d80 0x7fd9540a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:17.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.007+0000 7fd96d4a8640 1 --2- 172.21.15.71:0/1403024396 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd9540a1d80 0x7fd9540a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:17.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.007+0000 7fd95e7fc640 1 -- 172.21.15.71:0/1403024396 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd9600ef8d0 con 0x7fd9681020a0 2024-03-04T08:56:17.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.007+0000 7fd96d4a8640 1 --2- 172.21.15.71:0/1403024396 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd9540a1d80 0x7fd9540a4240 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd968101540 tx=0x7fd964009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:17.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.010+0000 7fd95e7fc640 1 -- 172.21.15.71:0/1403024396 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd9600bcb50 con 0x7fd9681020a0 2024-03-04T08:56:17.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.223+0000 7fd96f70c640 1 -- 172.21.15.71:0/1403024396 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9681a0170 con 0x7fd9681020a0 2024-03-04T08:56:17.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.223+0000 7fd95e7fc640 1 -- 172.21.15.71:0/1403024396 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6871 (secure 0 0 0) 0x7fd9600c0bf0 con 0x7fd9681020a0 2024-03-04T08:56:17.220 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:17.220 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3975379807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3975379807},{"type":"v1","addr":"172.21.15.71:6835","nonce":3975379807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1962929347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1962929347},{"type":"v1","addr":"172.21.15.71:6837","nonce":1962929347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/178179528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":178179528},{"type":"v1","addr":"172.21.15.158:6833","nonce":178179528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2468877675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2468877675},{"type":"v1","addr":"172.21.15.150:6833","nonce":2468877675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1652094309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1652094309},{"type":"v1","addr":"172.21.15.158:6835","nonce":1652094309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1802416462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1802416462},{"type":"v1","addr":"172.21.15.150:6835","nonce":1802416462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/617058380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":617058380},{"type":"v1","addr":"172.21.15.158:6837","nonce":617058380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/913465600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":913465600},{"type":"v1","addr":"172.21.15.150:6837","nonce":913465600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-04T08:56:17.220 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 9 2024-03-04T08:56:17.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.225+0000 7fd96f70c640 1 -- 172.21.15.71:0/1403024396 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd9540a1d80 msgr2=0x7fd9540a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:17.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.225+0000 7fd96f70c640 1 --2- 172.21.15.71:0/1403024396 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd9540a1d80 0x7fd9540a4240 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd968101540 tx=0x7fd964009500 comp rx=0 tx=0).stop 2024-03-04T08:56:17.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.225+0000 7fd96f70c640 1 -- 172.21.15.71:0/1403024396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9681020a0 msgr2=0x7fd96819eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:17.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.225+0000 7fd96f70c640 1 --2- 172.21.15.71:0/1403024396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9681020a0 0x7fd96819eea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd96002c7b0 tx=0x7fd960005dd0 comp rx=0 tx=0).stop 2024-03-04T08:56:17.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.225+0000 7fd96f70c640 1 -- 172.21.15.71:0/1403024396 shutdown_connections 2024-03-04T08:56:17.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.225+0000 7fd96f70c640 1 --2- 172.21.15.71:0/1403024396 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd9540a1d80 0x7fd9540a4240 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:17.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.225+0000 7fd96f70c640 1 --2- 172.21.15.71:0/1403024396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9681020a0 0x7fd96819eea0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:17.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.225+0000 7fd96f70c640 1 --2- 172.21.15.71:0/1403024396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9681016e0 0x7fd968197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:17.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.225+0000 7fd96f70c640 1 --2- 172.21.15.71:0/1403024396 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd9681004f0 0x7fd9681978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:17.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.226+0000 7fd96f70c640 1 -- 172.21.15.71:0/1403024396 >> 172.21.15.71:0/1403024396 conn(0x7fd9680fba60 msgr2=0x7fd9680fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:17.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.226+0000 7fd96f70c640 1 -- 172.21.15.71:0/1403024396 shutdown_connections 2024-03-04T08:56:17.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:17.226+0000 7fd96f70c640 1 -- 172.21.15.71:0/1403024396 wait complete. 2024-03-04T08:56:17.261 INFO:journalctl@ceph.mds.i.smithi158.stdout:Mar 04 08:56:16 smithi158 podman[68651]: 2024-03-04 08:56:16.874636543 +0000 UTC m=+0.018009964 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T08:56:17.261 INFO:journalctl@ceph.mds.i.smithi158.stdout:Mar 04 08:56:17 smithi158 podman[68651]: 2024-03-04 08:56:17.004645699 +0000 UTC m=+0.148019100 container create c72f4745cb05f296f87564a9ecd7547f632457cf485a72e466aa2cc1068b43a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-04T08:56:17.261 INFO:journalctl@ceph.mds.i.smithi158.stdout:Mar 04 08:56:17 smithi158 podman[68651]: 2024-03-04 08:56:17.265824107 +0000 UTC m=+0.409197507 container init c72f4745cb05f296f87564a9ecd7547f632457cf485a72e466aa2cc1068b43a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-i, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-04T08:56:17.552 INFO:journalctl@ceph.mds.i.smithi158.stdout:Mar 04 08:56:17 smithi158 podman[68651]: 2024-03-04 08:56:17.268376198 +0000 UTC m=+0.411749606 container start c72f4745cb05f296f87564a9ecd7547f632457cf485a72e466aa2cc1068b43a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-i, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222) 2024-03-04T08:56:17.552 INFO:journalctl@ceph.mds.i.smithi158.stdout:Mar 04 08:56:17 smithi158 ceph-mds[68670]: -- 172.21.15.158:0/761081115 <== mon.1 v2:172.21.15.158:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555eb5b07a40 con 0x555eb5adb000 2024-03-04T08:56:17.552 INFO:journalctl@ceph.mds.i.smithi158.stdout:Mar 04 08:56:17 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-i[68666]: starting mds.i at 2024-03-04T08:56:17.552 INFO:journalctl@ceph.mds.i.smithi158.stdout:Mar 04 08:56:17 smithi158 bash[68651]: c72f4745cb05f296f87564a9ecd7547f632457cf485a72e466aa2cc1068b43a6 2024-03-04T08:56:17.552 INFO:journalctl@ceph.mds.i.smithi158.stdout:Mar 04 08:56:17 smithi158 systemd[1]: Started Ceph mds.i for b3c85c40-da03-11ee-95c6-87774f69a715. 2024-03-04T08:56:18.012 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:17 smithi150 ceph-mon[36807]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:18.012 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:17 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1403024396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:18.012 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:17 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:18.013 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:17 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:18.013 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:17 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:18.013 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:17 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:18.013 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:17 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:18.013 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:17 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:18.049 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:17 smithi071 ceph-mon[32789]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:18.050 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:17 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1403024396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:18.050 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:17 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:18.050 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:17 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:18.050 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:17 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:18.050 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:17 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:18.050 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:17 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:18.050 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:17 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:18.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:17 smithi158 ceph-mon[37241]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:18.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:17 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1403024396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:18.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:17 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:18.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:17 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:18.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:17 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:18.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:17 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:18.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:17 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:18.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:17 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:19.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:18 smithi150 ceph-mon[36807]: Deploying daemon mds.g on smithi071 2024-03-04T08:56:19.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:18 smithi150 ceph-mon[36807]: mds.? [v2:172.21.15.158:6838/2921704804,v1:172.21.15.158:6839/2921704804] up:boot 2024-03-04T08:56:19.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:18 smithi150 ceph-mon[36807]: fsmap 9 up:standby 2024-03-04T08:56:19.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:18 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-04T08:56:19.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:18 smithi158 ceph-mon[37241]: Deploying daemon mds.g on smithi071 2024-03-04T08:56:19.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:18 smithi158 ceph-mon[37241]: mds.? [v2:172.21.15.158:6838/2921704804,v1:172.21.15.158:6839/2921704804] up:boot 2024-03-04T08:56:19.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:18 smithi158 ceph-mon[37241]: fsmap 9 up:standby 2024-03-04T08:56:19.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:18 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-04T08:56:19.099 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:18 smithi071 ceph-mon[32789]: Deploying daemon mds.g on smithi071 2024-03-04T08:56:19.099 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:18 smithi071 ceph-mon[32789]: mds.? [v2:172.21.15.158:6838/2921704804,v1:172.21.15.158:6839/2921704804] up:boot 2024-03-04T08:56:19.099 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:18 smithi071 ceph-mon[32789]: fsmap 9 up:standby 2024-03-04T08:56:19.099 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:18 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-04T08:56:19.233 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:56:19.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.360+0000 7fa850393640 1 -- 172.21.15.71:0/2799510445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8481072d0 msgr2=0x7fa848109720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:19.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.360+0000 7fa850393640 1 --2- 172.21.15.71:0/2799510445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8481072d0 0x7fa848109720 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa838009900 tx=0x7fa83802bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:19.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.360+0000 7fa850393640 1 -- 172.21.15.71:0/2799510445 shutdown_connections 2024-03-04T08:56:19.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.360+0000 7fa850393640 1 --2- 172.21.15.71:0/2799510445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8481072d0 0x7fa848109720 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:19.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.360+0000 7fa850393640 1 --2- 172.21.15.71:0/2799510445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8481049a0 0x7fa848106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:19.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.360+0000 7fa850393640 1 --2- 172.21.15.71:0/2799510445 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa848102070 0x7fa848104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:19.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.360+0000 7fa850393640 1 -- 172.21.15.71:0/2799510445 >> 172.21.15.71:0/2799510445 conn(0x7fa8480fba80 msgr2=0x7fa8480fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:19.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.360+0000 7fa850393640 1 -- 172.21.15.71:0/2799510445 shutdown_connections 2024-03-04T08:56:19.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.360+0000 7fa850393640 1 -- 172.21.15.71:0/2799510445 wait complete. 2024-03-04T08:56:19.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.361+0000 7fa850393640 1 Processor -- start 2024-03-04T08:56:19.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.361+0000 7fa850393640 1 -- start start 2024-03-04T08:56:19.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.361+0000 7fa850393640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa848102070 0x7fa848197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:19.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.362+0000 7fa850393640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8481049a0 0x7fa848197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:19.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.362+0000 7fa850393640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa8481072d0 0x7fa84819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:19.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.362+0000 7fa850393640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84810d270 con 0x7fa848102070 2024-03-04T08:56:19.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.362+0000 7fa850393640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84810d0f0 con 0x7fa8481049a0 2024-03-04T08:56:19.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.362+0000 7fa850393640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84810d3f0 con 0x7fa8481072d0 2024-03-04T08:56:19.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.362+0000 7fa84e12f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa848102070 0x7fa848197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:19.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.362+0000 7fa84e12f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa848102070 0x7fa848197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55772/0 (socket says 172.21.15.71:55772) 2024-03-04T08:56:19.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.362+0000 7fa84e12f640 1 -- 172.21.15.71:0/3635476562 learned_addr learned my addr 172.21.15.71:0/3635476562 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:19.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.362+0000 7fa84d92e640 1 --2- 172.21.15.71:0/3635476562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8481049a0 0x7fa848197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:19.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.362+0000 7fa84e930640 1 --2- 172.21.15.71:0/3635476562 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa8481072d0 0x7fa84819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:19.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.362+0000 7fa84e12f640 1 -- 172.21.15.71:0/3635476562 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa8481072d0 msgr2=0x7fa84819ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:19.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.362+0000 7fa84e12f640 1 --2- 172.21.15.71:0/3635476562 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa8481072d0 0x7fa84819ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:19.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.362+0000 7fa84e12f640 1 -- 172.21.15.71:0/3635476562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8481049a0 msgr2=0x7fa848197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:19.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.362+0000 7fa84e12f640 1 --2- 172.21.15.71:0/3635476562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8481049a0 0x7fa848197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:19.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.362+0000 7fa84e12f640 1 -- 172.21.15.71:0/3635476562 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84819f4e0 con 0x7fa848102070 2024-03-04T08:56:19.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.362+0000 7fa84d92e640 1 --2- 172.21.15.71:0/3635476562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8481049a0 0x7fa848197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:56:19.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.362+0000 7fa84e12f640 1 --2- 172.21.15.71:0/3635476562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa848102070 0x7fa848197910 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa84400d950 tx=0x7fa84400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:19.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.362+0000 7fa83f7fe640 1 -- 172.21.15.71:0/3635476562 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa844017030 con 0x7fa848102070 2024-03-04T08:56:19.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.362+0000 7fa850393640 1 -- 172.21.15.71:0/3635476562 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84819f7d0 con 0x7fa848102070 2024-03-04T08:56:19.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.362+0000 7fa83f7fe640 1 -- 172.21.15.71:0/3635476562 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa844004020 con 0x7fa848102070 2024-03-04T08:56:19.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.362+0000 7fa83f7fe640 1 -- 172.21.15.71:0/3635476562 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8440047a0 con 0x7fa848102070 2024-03-04T08:56:19.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.362+0000 7fa850393640 1 -- 172.21.15.71:0/3635476562 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84819fd20 con 0x7fa848102070 2024-03-04T08:56:19.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.363+0000 7fa850393640 1 -- 172.21.15.71:0/3635476562 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8480fd570 con 0x7fa848102070 2024-03-04T08:56:19.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.364+0000 7fa83f7fe640 1 -- 172.21.15.71:0/3635476562 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fa8440041d0 con 0x7fa848102070 2024-03-04T08:56:19.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.365+0000 7fa83f7fe640 1 --2- 172.21.15.71:0/3635476562 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa8340a1d80 0x7fa8340a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:19.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.365+0000 7fa84d92e640 1 --2- 172.21.15.71:0/3635476562 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa8340a1d80 0x7fa8340a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:19.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.365+0000 7fa83f7fe640 1 -- 172.21.15.71:0/3635476562 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa8440cdad0 con 0x7fa848102070 2024-03-04T08:56:19.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.365+0000 7fa84d92e640 1 --2- 172.21.15.71:0/3635476562 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa8340a1d80 0x7fa8340a4240 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa848198dd0 tx=0x7fa840009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:19.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.373+0000 7fa83f7fe640 1 -- 172.21.15.71:0/3635476562 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa84409b3a0 con 0x7fa848102070 2024-03-04T08:56:19.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.605+0000 7fa850393640 1 -- 172.21.15.71:0/3635476562 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa84802d2a0 con 0x7fa848102070 2024-03-04T08:56:19.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.606+0000 7fa83f7fe640 1 -- 172.21.15.71:0/3635476562 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7667 (secure 0 0 0) 0x7fa84409aca0 con 0x7fa848102070 2024-03-04T08:56:19.602 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:19.602 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3975379807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3975379807},{"type":"v1","addr":"172.21.15.71:6835","nonce":3975379807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1962929347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1962929347},{"type":"v1","addr":"172.21.15.71:6837","nonce":1962929347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/178179528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":178179528},{"type":"v1","addr":"172.21.15.158:6833","nonce":178179528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2468877675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2468877675},{"type":"v1","addr":"172.21.15.150:6833","nonce":2468877675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1652094309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1652094309},{"type":"v1","addr":"172.21.15.158:6835","nonce":1652094309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1802416462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1802416462},{"type":"v1","addr":"172.21.15.150:6835","nonce":1802416462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/617058380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":617058380},{"type":"v1","addr":"172.21.15.158:6837","nonce":617058380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/913465600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":913465600},{"type":"v1","addr":"172.21.15.150:6837","nonce":913465600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6839/2921704804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6838","nonce":2921704804},{"type":"v1","addr":"172.21.15.158:6839","nonce":2921704804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-04T08:56:19.602 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 10 2024-03-04T08:56:19.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.608+0000 7fa850393640 1 -- 172.21.15.71:0/3635476562 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa8340a1d80 msgr2=0x7fa8340a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:19.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.608+0000 7fa850393640 1 --2- 172.21.15.71:0/3635476562 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa8340a1d80 0x7fa8340a4240 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa848198dd0 tx=0x7fa840009500 comp rx=0 tx=0).stop 2024-03-04T08:56:19.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.608+0000 7fa850393640 1 -- 172.21.15.71:0/3635476562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa848102070 msgr2=0x7fa848197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:19.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.608+0000 7fa850393640 1 --2- 172.21.15.71:0/3635476562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa848102070 0x7fa848197910 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa84400d950 tx=0x7fa84400de10 comp rx=0 tx=0).stop 2024-03-04T08:56:19.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.608+0000 7fa850393640 1 -- 172.21.15.71:0/3635476562 shutdown_connections 2024-03-04T08:56:19.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.608+0000 7fa850393640 1 --2- 172.21.15.71:0/3635476562 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa8340a1d80 0x7fa8340a4240 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:19.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.608+0000 7fa850393640 1 --2- 172.21.15.71:0/3635476562 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa8481072d0 0x7fa84819ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:19.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.608+0000 7fa850393640 1 --2- 172.21.15.71:0/3635476562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8481049a0 0x7fa848197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:19.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.608+0000 7fa850393640 1 --2- 172.21.15.71:0/3635476562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa848102070 0x7fa848197910 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:19.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.608+0000 7fa850393640 1 -- 172.21.15.71:0/3635476562 >> 172.21.15.71:0/3635476562 conn(0x7fa8480fba80 msgr2=0x7fa8481029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:19.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.608+0000 7fa850393640 1 -- 172.21.15.71:0/3635476562 shutdown_connections 2024-03-04T08:56:19.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:19.608+0000 7fa850393640 1 -- 172.21.15.71:0/3635476562 wait complete. 2024-03-04T08:56:19.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:19 smithi071 ceph-mon[32789]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:19.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:19 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3635476562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:20.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:19 smithi150 ceph-mon[36807]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:20.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:19 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3635476562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:20.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:19 smithi158 ceph-mon[37241]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:20.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:19 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3635476562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:20.671 INFO:journalctl@ceph.mds.g.smithi071.stdout:Mar 04 08:56:20 smithi071 systemd[1]: Started Ceph mds.g for b3c85c40-da03-11ee-95c6-87774f69a715. 2024-03-04T08:56:21.616 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:56:21.642 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:21 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:21.642 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:21 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:21.643 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:21 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:21.643 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:21 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:21.643 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:21 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:21.643 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:21 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:21.643 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:21 smithi071 ceph-mon[32789]: Deploying daemon mds.a on smithi071 2024-03-04T08:56:21.643 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:21 smithi071 ceph-mon[32789]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:21.643 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:21 smithi071 ceph-mon[32789]: mds.? [v2:172.21.15.71:6838/1459252176,v1:172.21.15.71:6839/1459252176] up:boot 2024-03-04T08:56:21.643 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:21 smithi071 ceph-mon[32789]: fsmap 10 up:standby 2024-03-04T08:56:21.643 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:21 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-04T08:56:21.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.727+0000 7f33066a4640 1 -- 172.21.15.71:0/1168079836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3300101620 msgr2=0x7f3300101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:21.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.727+0000 7f33066a4640 1 --2- 172.21.15.71:0/1168079836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3300101620 0x7f3300101a40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009900 tx=0x7f32f002bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:21.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.727+0000 7f33066a4640 1 -- 172.21.15.71:0/1168079836 shutdown_connections 2024-03-04T08:56:21.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.727+0000 7f33066a4640 1 --2- 172.21.15.71:0/1168079836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33001031d0 0x7f3300109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:21.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.727+0000 7f33066a4640 1 --2- 172.21.15.71:0/1168079836 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3300102810 0x7f3300102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:21.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.727+0000 7f33066a4640 1 --2- 172.21.15.71:0/1168079836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3300101620 0x7f3300101a40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:21.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.727+0000 7f33066a4640 1 -- 172.21.15.71:0/1168079836 >> 172.21.15.71:0/1168079836 conn(0x7f33000fcb70 msgr2=0x7f33000fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:21.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.727+0000 7f33066a4640 1 -- 172.21.15.71:0/1168079836 shutdown_connections 2024-03-04T08:56:21.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.727+0000 7f33066a4640 1 -- 172.21.15.71:0/1168079836 wait complete. 2024-03-04T08:56:21.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.728+0000 7f33066a4640 1 Processor -- start 2024-03-04T08:56:21.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.728+0000 7f33066a4640 1 -- start start 2024-03-04T08:56:21.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.728+0000 7f33066a4640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3300101620 0x7f33001978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:21.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.728+0000 7f33066a4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3300102810 0x7f3300197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:21.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.728+0000 7f33066a4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33001031d0 0x7f330019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:21.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.728+0000 7f33066a4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330010d2a0 con 0x7f33001031d0 2024-03-04T08:56:21.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.728+0000 7f33066a4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330010d120 con 0x7f3300102810 2024-03-04T08:56:21.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.728+0000 7f33066a4640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330010d420 con 0x7f3300101620 2024-03-04T08:56:21.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.728+0000 7f3304c41640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33001031d0 0x7f330019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:21.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.728+0000 7f3304c41640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33001031d0 0x7f330019eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55812/0 (socket says 172.21.15.71:55812) 2024-03-04T08:56:21.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.728+0000 7f3304c41640 1 -- 172.21.15.71:0/2307892632 learned_addr learned my addr 172.21.15.71:0/2307892632 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:21.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.728+0000 7f32ff7fe640 1 --2- 172.21.15.71:0/2307892632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3300102810 0x7f3300197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:21.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.728+0000 7f32fffff640 1 --2- 172.21.15.71:0/2307892632 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3300101620 0x7f33001978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:21.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.728+0000 7f3304c41640 1 -- 172.21.15.71:0/2307892632 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3300101620 msgr2=0x7f33001978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:21.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.728+0000 7f3304c41640 1 --2- 172.21.15.71:0/2307892632 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3300101620 0x7f33001978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:21.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.728+0000 7f3304c41640 1 -- 172.21.15.71:0/2307892632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3300102810 msgr2=0x7f3300197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:21.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.728+0000 7f3304c41640 1 --2- 172.21.15.71:0/2307892632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3300102810 0x7f3300197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:21.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.728+0000 7f3304c41640 1 -- 172.21.15.71:0/2307892632 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330019f510 con 0x7f33001031d0 2024-03-04T08:56:21.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.728+0000 7f32fffff640 1 --2- 172.21.15.71:0/2307892632 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3300101620 0x7f33001978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:56:21.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.729+0000 7f3304c41640 1 --2- 172.21.15.71:0/2307892632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33001031d0 0x7f330019eea0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f32f800cce0 tx=0x7f32f800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:21.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.729+0000 7f32fd7fa640 1 -- 172.21.15.71:0/2307892632 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f800f450 con 0x7f33001031d0 2024-03-04T08:56:21.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.729+0000 7f33066a4640 1 -- 172.21.15.71:0/2307892632 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f330019f800 con 0x7f33001031d0 2024-03-04T08:56:21.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.729+0000 7f32fd7fa640 1 -- 172.21.15.71:0/2307892632 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32f800fad0 con 0x7f33001031d0 2024-03-04T08:56:21.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.729+0000 7f32fd7fa640 1 -- 172.21.15.71:0/2307892632 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f8018c10 con 0x7f33001031d0 2024-03-04T08:56:21.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.729+0000 7f33066a4640 1 -- 172.21.15.71:0/2307892632 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f330019fd00 con 0x7f33001031d0 2024-03-04T08:56:21.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.730+0000 7f33066a4640 1 -- 172.21.15.71:0/2307892632 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3300108940 con 0x7f33001031d0 2024-03-04T08:56:21.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.731+0000 7f32fd7fa640 1 -- 172.21.15.71:0/2307892632 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f32f800f5f0 con 0x7f33001031d0 2024-03-04T08:56:21.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.732+0000 7f32fd7fa640 1 --2- 172.21.15.71:0/2307892632 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f32e80a1d80 0x7f32e80a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:21.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.732+0000 7f32fffff640 1 --2- 172.21.15.71:0/2307892632 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f32e80a1d80 0x7f32e80a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:21.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.732+0000 7f32fd7fa640 1 -- 172.21.15.71:0/2307892632 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f32f80ce130 con 0x7f33001031d0 2024-03-04T08:56:21.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.732+0000 7f32fffff640 1 --2- 172.21.15.71:0/2307892632 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f32e80a1d80 0x7f32e80a4240 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f32f0036040 tx=0x7f32f0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:21.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.735+0000 7f32fd7fa640 1 -- 172.21.15.71:0/2307892632 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f32f8014030 con 0x7f33001031d0 2024-03-04T08:56:21.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:21 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:21.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:21 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:21.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:21 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:21.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:21 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:21.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:21 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:21.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:21 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:21.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:21 smithi150 ceph-mon[36807]: Deploying daemon mds.a on smithi071 2024-03-04T08:56:21.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:21 smithi150 ceph-mon[36807]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:21.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:21 smithi150 ceph-mon[36807]: mds.? [v2:172.21.15.71:6838/1459252176,v1:172.21.15.71:6839/1459252176] up:boot 2024-03-04T08:56:21.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:21 smithi150 ceph-mon[36807]: fsmap 10 up:standby 2024-03-04T08:56:21.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:21 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-04T08:56:21.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:21 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:21.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:21 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:21.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:21 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:21.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:21 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:21.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:21 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:21.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:21 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:21.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:21 smithi158 ceph-mon[37241]: Deploying daemon mds.a on smithi071 2024-03-04T08:56:21.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:21 smithi158 ceph-mon[37241]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:21.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:21 smithi158 ceph-mon[37241]: mds.? [v2:172.21.15.71:6838/1459252176,v1:172.21.15.71:6839/1459252176] up:boot 2024-03-04T08:56:21.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:21 smithi158 ceph-mon[37241]: fsmap 10 up:standby 2024-03-04T08:56:21.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:21 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-04T08:56:21.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.958+0000 7f33066a4640 1 -- 172.21.15.71:0/2307892632 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f330002cff0 con 0x7f33001031d0 2024-03-04T08:56:21.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.959+0000 7f32fd7fa640 1 -- 172.21.15.71:0/2307892632 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8459 (secure 0 0 0) 0x7f32f809ba90 con 0x7f33001031d0 2024-03-04T08:56:21.956 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:21.956 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3975379807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3975379807},{"type":"v1","addr":"172.21.15.71:6835","nonce":3975379807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1962929347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1962929347},{"type":"v1","addr":"172.21.15.71:6837","nonce":1962929347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/1459252176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1459252176},{"type":"v1","addr":"172.21.15.71:6839","nonce":1459252176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/178179528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":178179528},{"type":"v1","addr":"172.21.15.158:6833","nonce":178179528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2468877675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2468877675},{"type":"v1","addr":"172.21.15.150:6833","nonce":2468877675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1652094309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1652094309},{"type":"v1","addr":"172.21.15.158:6835","nonce":1652094309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1802416462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1802416462},{"type":"v1","addr":"172.21.15.150:6835","nonce":1802416462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/617058380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":617058380},{"type":"v1","addr":"172.21.15.158:6837","nonce":617058380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/913465600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":913465600},{"type":"v1","addr":"172.21.15.150:6837","nonce":913465600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6839/2921704804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6838","nonce":2921704804},{"type":"v1","addr":"172.21.15.158:6839","nonce":2921704804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-04T08:56:21.956 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 11 2024-03-04T08:56:21.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.961+0000 7f33066a4640 1 -- 172.21.15.71:0/2307892632 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f32e80a1d80 msgr2=0x7f32e80a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:21.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.961+0000 7f33066a4640 1 --2- 172.21.15.71:0/2307892632 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f32e80a1d80 0x7f32e80a4240 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f32f0036040 tx=0x7f32f0000bc0 comp rx=0 tx=0).stop 2024-03-04T08:56:21.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.961+0000 7f33066a4640 1 -- 172.21.15.71:0/2307892632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33001031d0 msgr2=0x7f330019eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:21.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.961+0000 7f33066a4640 1 --2- 172.21.15.71:0/2307892632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33001031d0 0x7f330019eea0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f32f800cce0 tx=0x7f32f800a620 comp rx=0 tx=0).stop 2024-03-04T08:56:21.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.961+0000 7f33066a4640 1 -- 172.21.15.71:0/2307892632 shutdown_connections 2024-03-04T08:56:21.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.961+0000 7f33066a4640 1 --2- 172.21.15.71:0/2307892632 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f32e80a1d80 0x7f32e80a4240 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:21.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.961+0000 7f33066a4640 1 --2- 172.21.15.71:0/2307892632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33001031d0 0x7f330019eea0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:21.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.961+0000 7f33066a4640 1 --2- 172.21.15.71:0/2307892632 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3300102810 0x7f3300197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:21.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.961+0000 7f33066a4640 1 --2- 172.21.15.71:0/2307892632 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3300101620 0x7f33001978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:21.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.961+0000 7f33066a4640 1 -- 172.21.15.71:0/2307892632 >> 172.21.15.71:0/2307892632 conn(0x7f33000fcb70 msgr2=0x7f3300105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:21.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.962+0000 7f33066a4640 1 -- 172.21.15.71:0/2307892632 shutdown_connections 2024-03-04T08:56:21.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:21.962+0000 7f33066a4640 1 -- 172.21.15.71:0/2307892632 wait complete. 2024-03-04T08:56:22.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:22 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2307892632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:22.672 INFO:journalctl@ceph.mds.a.smithi071.stdout:Mar 04 08:56:22 smithi071 systemd[1]: Starting Ceph mds.a for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T08:56:22.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:22 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2307892632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:22.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:22 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2307892632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:23.123 INFO:journalctl@ceph.mds.a.smithi071.stdout:Mar 04 08:56:22 smithi071 podman[74622]: 2024-03-04 08:56:22.816262781 +0000 UTC m=+0.111841321 container create 086e350c76f92aa721e2dfc16e56bf41107d7513bcd76f1080b684b246a4b095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222) 2024-03-04T08:56:23.123 INFO:journalctl@ceph.mds.a.smithi071.stdout:Mar 04 08:56:22 smithi071 podman[74622]: 2024-03-04 08:56:22.722150506 +0000 UTC m=+0.017729017 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T08:56:23.123 INFO:journalctl@ceph.mds.a.smithi071.stdout:Mar 04 08:56:23 smithi071 podman[74622]: 2024-03-04 08:56:23.02703764 +0000 UTC m=+0.322616143 container init 086e350c76f92aa721e2dfc16e56bf41107d7513bcd76f1080b684b246a4b095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-a, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c, maintainer=Guillaume Abrioux ) 2024-03-04T08:56:23.123 INFO:journalctl@ceph.mds.a.smithi071.stdout:Mar 04 08:56:23 smithi071 podman[74622]: 2024-03-04 08:56:23.029588966 +0000 UTC m=+0.325167469 container start 086e350c76f92aa721e2dfc16e56bf41107d7513bcd76f1080b684b246a4b095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-a, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-04T08:56:23.123 INFO:journalctl@ceph.mds.a.smithi071.stdout:Mar 04 08:56:23 smithi071 ceph-mds[74641]: -- 172.21.15.71:0/215921849 <== mon.0 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5624f4f01a40 con 0x5624f4ed5000 2024-03-04T08:56:23.123 INFO:journalctl@ceph.mds.a.smithi071.stdout:Mar 04 08:56:23 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-a[74637]: starting mds.a at 2024-03-04T08:56:23.123 INFO:journalctl@ceph.mds.a.smithi071.stdout:Mar 04 08:56:23 smithi071 bash[74622]: 086e350c76f92aa721e2dfc16e56bf41107d7513bcd76f1080b684b246a4b095 2024-03-04T08:56:23.421 INFO:journalctl@ceph.mds.a.smithi071.stdout:Mar 04 08:56:23 smithi071 systemd[1]: Started Ceph mds.a for b3c85c40-da03-11ee-95c6-87774f69a715. 2024-03-04T08:56:23.726 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:23 smithi158 ceph-mon[37241]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:23.727 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:23 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:23.727 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:23 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:23.727 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:23 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:23.727 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:23 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:23.727 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:23 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:23.727 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:23 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:23.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:23 smithi150 ceph-mon[36807]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:23.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:23 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:23.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:23 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:23.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:23 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:23.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:23 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:23.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:23 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:23.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:23 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:23.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:23 smithi071 ceph-mon[32789]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:23.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:23 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:23.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:23 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:23.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:23 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:23.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:23 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T08:56:23.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:23 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T08:56:23.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:23 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:23.969 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:56:24.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.086+0000 7f813b793640 1 -- 172.21.15.71:0/1613919246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8134100e70 msgr2=0x7f81341012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:24.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.086+0000 7f813b793640 1 --2- 172.21.15.71:0/1613919246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8134100e70 0x7f81341012f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8124009900 tx=0x7f812402bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:24.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.086+0000 7f813b793640 1 -- 172.21.15.71:0/1613919246 shutdown_connections 2024-03-04T08:56:24.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.086+0000 7f813b793640 1 --2- 172.21.15.71:0/1613919246 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8134101830 0x7f81341080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:24.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.086+0000 7f813b793640 1 --2- 172.21.15.71:0/1613919246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8134100e70 0x7f81341012f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:24.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.086+0000 7f813b793640 1 --2- 172.21.15.71:0/1613919246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81340ffc80 0x7f81341000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:24.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.086+0000 7f813b793640 1 -- 172.21.15.71:0/1613919246 >> 172.21.15.71:0/1613919246 conn(0x7f81340fb1f0 msgr2=0x7f81340fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:24.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.086+0000 7f813b793640 1 -- 172.21.15.71:0/1613919246 shutdown_connections 2024-03-04T08:56:24.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.086+0000 7f813b793640 1 -- 172.21.15.71:0/1613919246 wait complete. 2024-03-04T08:56:24.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.087+0000 7f813b793640 1 Processor -- start 2024-03-04T08:56:24.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.087+0000 7f813b793640 1 -- start start 2024-03-04T08:56:24.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.087+0000 7f813b793640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f81340ffc80 0x7f8134196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:24.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.087+0000 7f813b793640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8134100e70 0x7f8134196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:24.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.087+0000 7f813b793640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8134101830 0x7f813419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:24.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.087+0000 7f813b793640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813410b900 con 0x7f8134101830 2024-03-04T08:56:24.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.087+0000 7f813b793640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813410b780 con 0x7f8134100e70 2024-03-04T08:56:24.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.087+0000 7f813b793640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813410ba80 con 0x7f81340ffc80 2024-03-04T08:56:24.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.087+0000 7f8139d30640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8134101830 0x7f813419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:24.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.087+0000 7f8139d30640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8134101830 0x7f813419ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55848/0 (socket says 172.21.15.71:55848) 2024-03-04T08:56:24.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.087+0000 7f8139d30640 1 -- 172.21.15.71:0/627324171 learned_addr learned my addr 172.21.15.71:0/627324171 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:24.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.087+0000 7f8138d2e640 1 --2- 172.21.15.71:0/627324171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8134100e70 0x7f8134196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:24.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.087+0000 7f813952f640 1 --2- 172.21.15.71:0/627324171 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f81340ffc80 0x7f8134196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:24.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.088+0000 7f8139d30640 1 -- 172.21.15.71:0/627324171 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f81340ffc80 msgr2=0x7f8134196850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:24.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.088+0000 7f8139d30640 1 --2- 172.21.15.71:0/627324171 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f81340ffc80 0x7f8134196850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:24.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.088+0000 7f8139d30640 1 -- 172.21.15.71:0/627324171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8134100e70 msgr2=0x7f8134196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:24.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.088+0000 7f8139d30640 1 --2- 172.21.15.71:0/627324171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8134100e70 0x7f8134196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:24.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.088+0000 7f8139d30640 1 -- 172.21.15.71:0/627324171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813419e420 con 0x7f8134101830 2024-03-04T08:56:24.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.088+0000 7f8139d30640 1 --2- 172.21.15.71:0/627324171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8134101830 0x7f813419ddb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f813000eaa0 tx=0x7f813000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:24.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.088+0000 7f812a7fc640 1 -- 172.21.15.71:0/627324171 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813000cdb0 con 0x7f8134101830 2024-03-04T08:56:24.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.088+0000 7f813b793640 1 -- 172.21.15.71:0/627324171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f813419e710 con 0x7f8134101830 2024-03-04T08:56:24.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.088+0000 7f812a7fc640 1 -- 172.21.15.71:0/627324171 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8130005020 con 0x7f8134101830 2024-03-04T08:56:24.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.088+0000 7f812a7fc640 1 -- 172.21.15.71:0/627324171 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81300116b0 con 0x7f8134101830 2024-03-04T08:56:24.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.088+0000 7f813b793640 1 -- 172.21.15.71:0/627324171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f813419ec30 con 0x7f8134101830 2024-03-04T08:56:24.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.089+0000 7f813b793640 1 -- 172.21.15.71:0/627324171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8134107760 con 0x7f8134101830 2024-03-04T08:56:24.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.090+0000 7f812a7fc640 1 -- 172.21.15.71:0/627324171 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f8130005560 con 0x7f8134101830 2024-03-04T08:56:24.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.091+0000 7f812a7fc640 1 --2- 172.21.15.71:0/627324171 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f81200a1d80 0x7f81200a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:24.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.091+0000 7f813952f640 1 --2- 172.21.15.71:0/627324171 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f81200a1d80 0x7f81200a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:24.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.091+0000 7f812a7fc640 1 -- 172.21.15.71:0/627324171 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f81300cda90 con 0x7f8134101830 2024-03-04T08:56:24.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.091+0000 7f813952f640 1 --2- 172.21.15.71:0/627324171 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f81200a1d80 0x7f81200a4240 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f812c005d00 tx=0x7f812c005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:24.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.095+0000 7f812a7fc640 1 -- 172.21.15.71:0/627324171 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8130016030 con 0x7f8134101830 2024-03-04T08:56:24.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.306+0000 7f813b793640 1 -- 172.21.15.71:0/627324171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f813402cff0 con 0x7f8134101830 2024-03-04T08:56:24.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.306+0000 7f812a7fc640 1 -- 172.21.15.71:0/627324171 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9251 (secure 0 0 0) 0x7f813009b360 con 0x7f8134101830 2024-03-04T08:56:24.302 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:24.302 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3975379807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3975379807},{"type":"v1","addr":"172.21.15.71:6835","nonce":3975379807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1962929347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1962929347},{"type":"v1","addr":"172.21.15.71:6837","nonce":1962929347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/1459252176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1459252176},{"type":"v1","addr":"172.21.15.71:6839","nonce":1459252176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6841/1083546600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":1083546600},{"type":"v1","addr":"172.21.15.71:6841","nonce":1083546600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/178179528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":178179528},{"type":"v1","addr":"172.21.15.158:6833","nonce":178179528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2468877675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2468877675},{"type":"v1","addr":"172.21.15.150:6833","nonce":2468877675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1652094309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1652094309},{"type":"v1","addr":"172.21.15.158:6835","nonce":1652094309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1802416462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1802416462},{"type":"v1","addr":"172.21.15.150:6835","nonce":1802416462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/617058380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":617058380},{"type":"v1","addr":"172.21.15.158:6837","nonce":617058380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/913465600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":913465600},{"type":"v1","addr":"172.21.15.150:6837","nonce":913465600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6839/2921704804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6838","nonce":2921704804},{"type":"v1","addr":"172.21.15.158:6839","nonce":2921704804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-04T08:56:24.303 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 12 2024-03-04T08:56:24.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.308+0000 7f813b793640 1 -- 172.21.15.71:0/627324171 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f81200a1d80 msgr2=0x7f81200a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:24.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.308+0000 7f813b793640 1 --2- 172.21.15.71:0/627324171 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f81200a1d80 0x7f81200a4240 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f812c005d00 tx=0x7f812c005c50 comp rx=0 tx=0).stop 2024-03-04T08:56:24.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.308+0000 7f813b793640 1 -- 172.21.15.71:0/627324171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8134101830 msgr2=0x7f813419ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:24.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.308+0000 7f813b793640 1 --2- 172.21.15.71:0/627324171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8134101830 0x7f813419ddb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f813000eaa0 tx=0x7f813000ef60 comp rx=0 tx=0).stop 2024-03-04T08:56:24.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.308+0000 7f813b793640 1 -- 172.21.15.71:0/627324171 shutdown_connections 2024-03-04T08:56:24.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.308+0000 7f813b793640 1 --2- 172.21.15.71:0/627324171 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f81200a1d80 0x7f81200a4240 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:24.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.308+0000 7f813b793640 1 --2- 172.21.15.71:0/627324171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8134101830 0x7f813419ddb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:24.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.308+0000 7f813b793640 1 --2- 172.21.15.71:0/627324171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8134100e70 0x7f8134196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:24.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.308+0000 7f813b793640 1 --2- 172.21.15.71:0/627324171 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f81340ffc80 0x7f8134196850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:24.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.308+0000 7f813b793640 1 -- 172.21.15.71:0/627324171 >> 172.21.15.71:0/627324171 conn(0x7f81340fb1f0 msgr2=0x7f8134104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:24.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.308+0000 7f813b793640 1 -- 172.21.15.71:0/627324171 shutdown_connections 2024-03-04T08:56:24.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:24.308+0000 7f813b793640 1 -- 172.21.15.71:0/627324171 wait complete. 2024-03-04T08:56:24.441 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:24 smithi150 ceph-mon[36807]: Deploying daemon mds.b on smithi150 2024-03-04T08:56:24.441 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:24 smithi150 ceph-mon[36807]: mds.? [v2:172.21.15.71:6840/1083546600,v1:172.21.15.71:6841/1083546600] up:boot 2024-03-04T08:56:24.441 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:24 smithi150 ceph-mon[36807]: fsmap 11 up:standby 2024-03-04T08:56:24.441 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:24 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-04T08:56:24.441 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:24 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/627324171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:24.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:24 smithi071 ceph-mon[32789]: Deploying daemon mds.b on smithi150 2024-03-04T08:56:24.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:24 smithi071 ceph-mon[32789]: mds.? [v2:172.21.15.71:6840/1083546600,v1:172.21.15.71:6841/1083546600] up:boot 2024-03-04T08:56:24.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:24 smithi071 ceph-mon[32789]: fsmap 11 up:standby 2024-03-04T08:56:24.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:24 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-04T08:56:24.672 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:24 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/627324171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:24.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:24 smithi158 ceph-mon[37241]: Deploying daemon mds.b on smithi150 2024-03-04T08:56:24.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:24 smithi158 ceph-mon[37241]: mds.? [v2:172.21.15.71:6840/1083546600,v1:172.21.15.71:6841/1083546600] up:boot 2024-03-04T08:56:24.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:24 smithi158 ceph-mon[37241]: fsmap 11 up:standby 2024-03-04T08:56:24.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:24 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-04T08:56:24.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:24 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/627324171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:25.447 INFO:journalctl@ceph.mds.b.smithi150.stdout:Mar 04 08:56:25 smithi150 systemd[1]: Starting Ceph mds.b for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T08:56:25.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:25 smithi150 ceph-mon[36807]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:25.780 INFO:journalctl@ceph.mds.b.smithi150.stdout:Mar 04 08:56:25 smithi150 podman[66645]: 2024-03-04 08:56:25.452086689 +0000 UTC m=+0.018161582 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T08:56:25.780 INFO:journalctl@ceph.mds.b.smithi150.stdout:Mar 04 08:56:25 smithi150 podman[66645]: 2024-03-04 08:56:25.558552874 +0000 UTC m=+0.124627750 container create 20c8cbae09406d97eda8d39ca9dd38a00549afa34d617cec85ccab308b4ae88c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T08:56:25.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:25 smithi158 ceph-mon[37241]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:25.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:25 smithi071 ceph-mon[32789]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:26.280 INFO:journalctl@ceph.mds.b.smithi150.stdout:Mar 04 08:56:25 smithi150 podman[66645]: 2024-03-04 08:56:25.818091128 +0000 UTC m=+0.384166008 container init 20c8cbae09406d97eda8d39ca9dd38a00549afa34d617cec85ccab308b4ae88c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-04T08:56:26.280 INFO:journalctl@ceph.mds.b.smithi150.stdout:Mar 04 08:56:25 smithi150 podman[66645]: 2024-03-04 08:56:25.82082086 +0000 UTC m=+0.386895729 container start 20c8cbae09406d97eda8d39ca9dd38a00549afa34d617cec85ccab308b4ae88c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-b, RELEASE=wip-lusov-qdb-1bc250c, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-04T08:56:26.280 INFO:journalctl@ceph.mds.b.smithi150.stdout:Mar 04 08:56:25 smithi150 ceph-mds[66664]: -- 172.21.15.150:0/3247055728 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56416ee2ba40 con 0x56416edfec00 2024-03-04T08:56:26.280 INFO:journalctl@ceph.mds.b.smithi150.stdout:Mar 04 08:56:25 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-b[66660]: starting mds.b at 2024-03-04T08:56:26.280 INFO:journalctl@ceph.mds.b.smithi150.stdout:Mar 04 08:56:25 smithi150 bash[66645]: 20c8cbae09406d97eda8d39ca9dd38a00549afa34d617cec85ccab308b4ae88c 2024-03-04T08:56:26.280 INFO:journalctl@ceph.mds.b.smithi150.stdout:Mar 04 08:56:25 smithi150 systemd[1]: Started Ceph mds.b for b3c85c40-da03-11ee-95c6-87774f69a715. 2024-03-04T08:56:26.316 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:56:26.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.425+0000 7fc9d000e640 1 -- 172.21.15.71:0/1673254730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9c81049a0 msgr2=0x7fc9c8106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:26.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.425+0000 7fc9d000e640 1 --2- 172.21.15.71:0/1673254730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9c81049a0 0x7fc9c8106d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009900 tx=0x7fc9c402bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:26.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.425+0000 7fc9d000e640 1 -- 172.21.15.71:0/1673254730 shutdown_connections 2024-03-04T08:56:26.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.425+0000 7fc9d000e640 1 --2- 172.21.15.71:0/1673254730 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc9c81072d0 0x7fc9c8109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:26.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.425+0000 7fc9d000e640 1 --2- 172.21.15.71:0/1673254730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9c81049a0 0x7fc9c8106d90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:26.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.425+0000 7fc9d000e640 1 --2- 172.21.15.71:0/1673254730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9c8102070 0x7fc9c8104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:26.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.425+0000 7fc9d000e640 1 -- 172.21.15.71:0/1673254730 >> 172.21.15.71:0/1673254730 conn(0x7fc9c80fba60 msgr2=0x7fc9c80fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:26.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.425+0000 7fc9d000e640 1 -- 172.21.15.71:0/1673254730 shutdown_connections 2024-03-04T08:56:26.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.425+0000 7fc9d000e640 1 -- 172.21.15.71:0/1673254730 wait complete. 2024-03-04T08:56:26.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.426+0000 7fc9d000e640 1 Processor -- start 2024-03-04T08:56:26.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.426+0000 7fc9d000e640 1 -- start start 2024-03-04T08:56:26.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.426+0000 7fc9d000e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9c8102070 0x7fc9c8197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:26.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.426+0000 7fc9d000e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9c81049a0 0x7fc9c8197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:26.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.426+0000 7fc9d000e640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc9c81072d0 0x7fc9c819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:26.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.426+0000 7fc9d000e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c810d260 con 0x7fc9c81049a0 2024-03-04T08:56:26.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.426+0000 7fc9d000e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c810d0e0 con 0x7fc9c8102070 2024-03-04T08:56:26.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.426+0000 7fc9d000e640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c810d3e0 con 0x7fc9c81072d0 2024-03-04T08:56:26.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.426+0000 7fc9cd5a9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9c81049a0 0x7fc9c8197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:26.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.426+0000 7fc9cd5a9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9c81049a0 0x7fc9c8197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47066/0 (socket says 172.21.15.71:47066) 2024-03-04T08:56:26.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.426+0000 7fc9cd5a9640 1 -- 172.21.15.71:0/1712701574 learned_addr learned my addr 172.21.15.71:0/1712701574 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:26.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.426+0000 7fc9cddaa640 1 --2- 172.21.15.71:0/1712701574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9c8102070 0x7fc9c8197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:26.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.426+0000 7fc9ce5ab640 1 --2- 172.21.15.71:0/1712701574 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc9c81072d0 0x7fc9c819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:26.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.426+0000 7fc9cd5a9640 1 -- 172.21.15.71:0/1712701574 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc9c81072d0 msgr2=0x7fc9c819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:26.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.426+0000 7fc9cd5a9640 1 --2- 172.21.15.71:0/1712701574 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc9c81072d0 0x7fc9c819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:26.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.426+0000 7fc9cd5a9640 1 -- 172.21.15.71:0/1712701574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9c8102070 msgr2=0x7fc9c8197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:26.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.426+0000 7fc9cd5a9640 1 --2- 172.21.15.71:0/1712701574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9c8102070 0x7fc9c8197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:26.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.426+0000 7fc9cd5a9640 1 -- 172.21.15.71:0/1712701574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c819f550 con 0x7fc9c81049a0 2024-03-04T08:56:26.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.426+0000 7fc9cd5a9640 1 --2- 172.21.15.71:0/1712701574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9c81049a0 0x7fc9c8197e70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4000c00 tx=0x7fc9c402c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:26.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.427+0000 7fc9beffd640 1 -- 172.21.15.71:0/1712701574 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4041070 con 0x7fc9c81049a0 2024-03-04T08:56:26.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.427+0000 7fc9beffd640 1 -- 172.21.15.71:0/1712701574 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc9c4004450 con 0x7fc9c81049a0 2024-03-04T08:56:26.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.427+0000 7fc9d000e640 1 -- 172.21.15.71:0/1712701574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c819f7e0 con 0x7fc9c81049a0 2024-03-04T08:56:26.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.427+0000 7fc9beffd640 1 -- 172.21.15.71:0/1712701574 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4034440 con 0x7fc9c81049a0 2024-03-04T08:56:26.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.427+0000 7fc9d000e640 1 -- 172.21.15.71:0/1712701574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c819fd20 con 0x7fc9c81049a0 2024-03-04T08:56:26.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.428+0000 7fc9d000e640 1 -- 172.21.15.71:0/1712701574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c80fd550 con 0x7fc9c81049a0 2024-03-04T08:56:26.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.429+0000 7fc9beffd640 1 -- 172.21.15.71:0/1712701574 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fc9c4044070 con 0x7fc9c81049a0 2024-03-04T08:56:26.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.429+0000 7fc9beffd640 1 --2- 172.21.15.71:0/1712701574 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc9b40a1e00 0x7fc9b40a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:26.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.429+0000 7fc9cddaa640 1 --2- 172.21.15.71:0/1712701574 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc9b40a1e00 0x7fc9b40a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:26.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.429+0000 7fc9beffd640 1 -- 172.21.15.71:0/1712701574 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc9c40ef330 con 0x7fc9c81049a0 2024-03-04T08:56:26.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.430+0000 7fc9cddaa640 1 --2- 172.21.15.71:0/1712701574 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc9b40a1e00 0x7fc9b40a42c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009bd0 tx=0x7fc9b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:26.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.434+0000 7fc9beffd640 1 -- 172.21.15.71:0/1712701574 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc9c40bc640 con 0x7fc9c81049a0 2024-03-04T08:56:26.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.678+0000 7fc9d000e640 1 -- 172.21.15.71:0/1712701574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9c802cff0 con 0x7fc9c81049a0 2024-03-04T08:56:26.675 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:26.675 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3975379807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3975379807},{"type":"v1","addr":"172.21.15.71:6835","nonce":3975379807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1962929347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1962929347},{"type":"v1","addr":"172.21.15.71:6837","nonce":1962929347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/1459252176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1459252176},{"type":"v1","addr":"172.21.15.71:6839","nonce":1459252176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6841/1083546600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":1083546600},{"type":"v1","addr":"172.21.15.71:6841","nonce":1083546600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/178179528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":178179528},{"type":"v1","addr":"172.21.15.158:6833","nonce":178179528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2468877675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2468877675},{"type":"v1","addr":"172.21.15.150:6833","nonce":2468877675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1652094309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1652094309},{"type":"v1","addr":"172.21.15.158:6835","nonce":1652094309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1802416462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1802416462},{"type":"v1","addr":"172.21.15.150:6835","nonce":1802416462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/617058380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":617058380},{"type":"v1","addr":"172.21.15.158:6837","nonce":617058380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/913465600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":913465600},{"type":"v1","addr":"172.21.15.150:6837","nonce":913465600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6839/2921704804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6838","nonce":2921704804},{"type":"v1","addr":"172.21.15.158:6839","nonce":2921704804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/2639278083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2639278083},{"type":"v1","addr":"172.21.15.150:6839","nonce":2639278083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-03-04T08:56:26.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.679+0000 7fc9beffd640 1 -- 172.21.15.71:0/1712701574 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10046 (secure 0 0 0) 0x7fc9c40b8030 con 0x7fc9c81049a0 2024-03-04T08:56:26.675 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 13 2024-03-04T08:56:26.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.681+0000 7fc9d000e640 1 -- 172.21.15.71:0/1712701574 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc9b40a1e00 msgr2=0x7fc9b40a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:26.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.681+0000 7fc9d000e640 1 --2- 172.21.15.71:0/1712701574 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc9b40a1e00 0x7fc9b40a42c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009bd0 tx=0x7fc9b8005f90 comp rx=0 tx=0).stop 2024-03-04T08:56:26.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.681+0000 7fc9d000e640 1 -- 172.21.15.71:0/1712701574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9c81049a0 msgr2=0x7fc9c8197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:26.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.681+0000 7fc9d000e640 1 --2- 172.21.15.71:0/1712701574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9c81049a0 0x7fc9c8197e70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4000c00 tx=0x7fc9c402c640 comp rx=0 tx=0).stop 2024-03-04T08:56:26.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.681+0000 7fc9d000e640 1 -- 172.21.15.71:0/1712701574 shutdown_connections 2024-03-04T08:56:26.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.681+0000 7fc9d000e640 1 --2- 172.21.15.71:0/1712701574 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc9b40a1e00 0x7fc9b40a42c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:26.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.681+0000 7fc9d000e640 1 --2- 172.21.15.71:0/1712701574 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc9c81072d0 0x7fc9c819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:26.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.681+0000 7fc9d000e640 1 --2- 172.21.15.71:0/1712701574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9c81049a0 0x7fc9c8197e70 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:26.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.681+0000 7fc9d000e640 1 --2- 172.21.15.71:0/1712701574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc9c8102070 0x7fc9c8197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:26.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.681+0000 7fc9d000e640 1 -- 172.21.15.71:0/1712701574 >> 172.21.15.71:0/1712701574 conn(0x7fc9c80fba60 msgr2=0x7fc9c8102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:26.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.681+0000 7fc9d000e640 1 -- 172.21.15.71:0/1712701574 shutdown_connections 2024-03-04T08:56:26.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.681+0000 7fc9d000e640 1 -- 172.21.15.71:0/1712701574 wait complete. 2024-03-04T08:56:26.689 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T08:56:26.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.809+0000 7faaee21c640 1 -- 172.21.15.71:0/3941826884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faae8107350 msgr2=0x7faae8109740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:26.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.809+0000 7faaee21c640 1 --2- 172.21.15.71:0/3941826884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faae8107350 0x7faae8109740 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faae0009ca0 tx=0x7faae002c0f0 comp rx=0 tx=0).stop 2024-03-04T08:56:26.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.809+0000 7faaee21c640 1 -- 172.21.15.71:0/3941826884 shutdown_connections 2024-03-04T08:56:26.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.809+0000 7faaee21c640 1 --2- 172.21.15.71:0/3941826884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faae8107350 0x7faae8109740 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:26.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.809+0000 7faaee21c640 1 --2- 172.21.15.71:0/3941826884 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faae8104a20 0x7faae8106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:26.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.809+0000 7faaee21c640 1 --2- 172.21.15.71:0/3941826884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faae81020f0 0x7faae81044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:26.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.809+0000 7faaee21c640 1 -- 172.21.15.71:0/3941826884 >> 172.21.15.71:0/3941826884 conn(0x7faae80fba60 msgr2=0x7faae80fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:26.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.809+0000 7faaee21c640 1 -- 172.21.15.71:0/3941826884 shutdown_connections 2024-03-04T08:56:26.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.809+0000 7faaee21c640 1 -- 172.21.15.71:0/3941826884 wait complete. 2024-03-04T08:56:26.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.809+0000 7faaee21c640 1 Processor -- start 2024-03-04T08:56:26.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.810+0000 7faaee21c640 1 -- start start 2024-03-04T08:56:26.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.810+0000 7faaee21c640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faae81020f0 0x7faae81978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:26.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.810+0000 7faaee21c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faae8104a20 0x7faae8197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:26.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.810+0000 7faaee21c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faae8107350 0x7faae819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:26.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.810+0000 7faaee21c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae810d280 con 0x7faae8104a20 2024-03-04T08:56:26.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.810+0000 7faaee21c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae810d100 con 0x7faae8107350 2024-03-04T08:56:26.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.810+0000 7faaee21c640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae810d400 con 0x7faae81020f0 2024-03-04T08:56:26.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.810+0000 7faae6ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faae8104a20 0x7faae8197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:26.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.810+0000 7faae6ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faae8104a20 0x7faae8197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47086/0 (socket says 172.21.15.71:47086) 2024-03-04T08:56:26.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.810+0000 7faae6ffd640 1 -- 172.21.15.71:0/880909222 learned_addr learned my addr 172.21.15.71:0/880909222 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:26.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.810+0000 7faae7fff640 1 --2- 172.21.15.71:0/880909222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faae8107350 0x7faae819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:26.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.810+0000 7faae77fe640 1 --2- 172.21.15.71:0/880909222 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faae81020f0 0x7faae81978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:26.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.810+0000 7faae6ffd640 1 -- 172.21.15.71:0/880909222 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faae81020f0 msgr2=0x7faae81978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:26.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.810+0000 7faae6ffd640 1 --2- 172.21.15.71:0/880909222 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faae81020f0 0x7faae81978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:26.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.810+0000 7faae6ffd640 1 -- 172.21.15.71:0/880909222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faae8107350 msgr2=0x7faae819ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:26.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.810+0000 7faae6ffd640 1 --2- 172.21.15.71:0/880909222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faae8107350 0x7faae819ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:26.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.810+0000 7faae6ffd640 1 -- 172.21.15.71:0/880909222 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae819f4e0 con 0x7faae8104a20 2024-03-04T08:56:26.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.811+0000 7faae6ffd640 1 --2- 172.21.15.71:0/880909222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faae8104a20 0x7faae8197e00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7faadc00e9f0 tx=0x7faadc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:26.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.811+0000 7faae4ff9640 1 -- 172.21.15.71:0/880909222 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc00ccc0 con 0x7faae8104a20 2024-03-04T08:56:26.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.811+0000 7faaee21c640 1 -- 172.21.15.71:0/880909222 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae819f7d0 con 0x7faae8104a20 2024-03-04T08:56:26.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.811+0000 7faae4ff9640 1 -- 172.21.15.71:0/880909222 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faadc005500 con 0x7faae8104a20 2024-03-04T08:56:26.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.811+0000 7faae4ff9640 1 -- 172.21.15.71:0/880909222 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc010690 con 0x7faae8104a20 2024-03-04T08:56:26.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.811+0000 7faaee21c640 1 -- 172.21.15.71:0/880909222 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae819fcd0 con 0x7faae8104a20 2024-03-04T08:56:26.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.812+0000 7faaee21c640 1 -- 172.21.15.71:0/880909222 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae80fd550 con 0x7faae8104a20 2024-03-04T08:56:26.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.813+0000 7faae4ff9640 1 -- 172.21.15.71:0/880909222 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7faadc0046c0 con 0x7faae8104a20 2024-03-04T08:56:26.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.814+0000 7faae4ff9640 1 --2- 172.21.15.71:0/880909222 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7faad00a1d80 0x7faad00a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:26.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.814+0000 7faae77fe640 1 --2- 172.21.15.71:0/880909222 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7faad00a1d80 0x7faad00a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:26.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.814+0000 7faae4ff9640 1 -- 172.21.15.71:0/880909222 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faadc0cdc90 con 0x7faae8104a20 2024-03-04T08:56:26.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.815+0000 7faae77fe640 1 --2- 172.21.15.71:0/880909222 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7faad00a1d80 0x7faad00a4240 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faad800bbe0 tx=0x7faad800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:26.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:26.822+0000 7faae4ff9640 1 -- 172.21.15.71:0/880909222 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7faadc09b560 con 0x7faae8104a20 2024-03-04T08:56:27.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.010+0000 7faaee21c640 1 -- 172.21.15.71:0/880909222 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faae802d240 con 0x7faae8104a20 2024-03-04T08:56:27.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.011+0000 7faae4ff9640 1 -- 172.21.15.71:0/880909222 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+18481 (secure 0 0 0) 0x7faadc09ae60 con 0x7faae8104a20 2024-03-04T08:56:27.007 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:27.008 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":83,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","created":"2024-03-04T08:47:28.546102+0000","modified":"2024-03-04T08:55:31.753527+0000","last_up_change":"2024-03-04T08:55:30.744961+0000","last_in_change":"2024-03-04T08:55:11.570990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T08:51:14.933487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7e43b47d-8888-4631-bc96-8a96fcf8cd46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6803","nonce":1173641530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6805","nonce":1173641530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6809","nonce":1173641530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6807","nonce":1173641530}]},"public_addr":"172.21.15.71:6803/1173641530","cluster_addr":"172.21.15.71:6805/1173641530","heartbeat_back_addr":"172.21.15.71:6809/1173641530","heartbeat_front_addr":"172.21.15.71:6807/1173641530","state":["exists","up"]},{"osd":1,"uuid":"1209f214-f2e2-4209-bf69-4528a43a2123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6801","nonce":3383948110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6803","nonce":3383948110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6807","nonce":3383948110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6805","nonce":3383948110}]},"public_addr":"172.21.15.150:6801/3383948110","cluster_addr":"172.21.15.150:6803/3383948110","heartbeat_back_addr":"172.21.15.150:6807/3383948110","heartbeat_front_addr":"172.21.15.150:6805/3383948110","state":["exists","up"]},{"osd":2,"uuid":"b88c3082-93a9-4c9b-9221-a7105a44c91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6801","nonce":586944144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6803","nonce":586944144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6807","nonce":586944144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6805","nonce":586944144}]},"public_addr":"172.21.15.158:6801/586944144","cluster_addr":"172.21.15.158:6803/586944144","heartbeat_back_addr":"172.21.15.158:6807/586944144","heartbeat_front_addr":"172.21.15.158:6805/586944144","state":["exists","up"]},{"osd":3,"uuid":"e4fc7723-fd73-40b2-b9d0-7f9bda282ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6811","nonce":1311903262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6813","nonce":1311903262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6817","nonce":1311903262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6815","nonce":1311903262}]},"public_addr":"172.21.15.71:6811/1311903262","cluster_addr":"172.21.15.71:6813/1311903262","heartbeat_back_addr":"172.21.15.71:6817/1311903262","heartbeat_front_addr":"172.21.15.71:6815/1311903262","state":["exists","up"]},{"osd":4,"uuid":"e7eb72ac-f066-4ccd-b2bf-22a6a26daf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6809","nonce":2300837573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6811","nonce":2300837573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6815","nonce":2300837573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6813","nonce":2300837573}]},"public_addr":"172.21.15.150:6809/2300837573","cluster_addr":"172.21.15.150:6811/2300837573","heartbeat_back_addr":"172.21.15.150:6815/2300837573","heartbeat_front_addr":"172.21.15.150:6813/2300837573","state":["exists","up"]},{"osd":5,"uuid":"9b295840-e104-4a66-9cc4-7020627c1765","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6809","nonce":1922694045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6811","nonce":1922694045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6815","nonce":1922694045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6813","nonce":1922694045}]},"public_addr":"172.21.15.158:6809/1922694045","cluster_addr":"172.21.15.158:6811/1922694045","heartbeat_back_addr":"172.21.15.158:6815/1922694045","heartbeat_front_addr":"172.21.15.158:6813/1922694045","state":["exists","up"]},{"osd":6,"uuid":"6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6819","nonce":3848842707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6821","nonce":3848842707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6825","nonce":3848842707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6823","nonce":3848842707}]},"public_addr":"172.21.15.71:6819/3848842707","cluster_addr":"172.21.15.71:6821/3848842707","heartbeat_back_addr":"172.21.15.71:6825/3848842707","heartbeat_front_addr":"172.21.15.71:6823/3848842707","state":["exists","up"]},{"osd":7,"uuid":"6160a40a-d30f-4569-949c-0afb1bb3d133","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6817","nonce":1328330305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6819","nonce":1328330305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6823","nonce":1328330305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6821","nonce":1328330305}]},"public_addr":"172.21.15.150:6817/1328330305","cluster_addr":"172.21.15.150:6819/1328330305","heartbeat_back_addr":"172.21.15.150:6823/1328330305","heartbeat_front_addr":"172.21.15.150:6821/1328330305","state":["exists","up"]},{"osd":8,"uuid":"95ed2341-84a4-465c-9948-ac667680ce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6817","nonce":2195304577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6819","nonce":2195304577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6823","nonce":2195304577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6821","nonce":2195304577}]},"public_addr":"172.21.15.158:6817/2195304577","cluster_addr":"172.21.15.158:6819/2195304577","heartbeat_back_addr":"172.21.15.158:6823/2195304577","heartbeat_front_addr":"172.21.15.158:6821/2195304577","state":["exists","up"]},{"osd":9,"uuid":"088f6419-63a0-4768-afae-dbb8a5f5a565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6827","nonce":1913779347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6829","nonce":1913779347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6833","nonce":1913779347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6831","nonce":1913779347}]},"public_addr":"172.21.15.71:6827/1913779347","cluster_addr":"172.21.15.71:6829/1913779347","heartbeat_back_addr":"172.21.15.71:6833/1913779347","heartbeat_front_addr":"172.21.15.71:6831/1913779347","state":["exists","up"]},{"osd":10,"uuid":"18c85e24-068d-4f1e-a3b8-f7915b363234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6825","nonce":3295420996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6827","nonce":3295420996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6831","nonce":3295420996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6829","nonce":3295420996}]},"public_addr":"172.21.15.150:6825/3295420996","cluster_addr":"172.21.15.150:6827/3295420996","heartbeat_back_addr":"172.21.15.150:6831/3295420996","heartbeat_front_addr":"172.21.15.150:6829/3295420996","state":["exists","up"]},{"osd":11,"uuid":"4f6b3ddf-c482-4740-b1fd-de836e047c2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6825","nonce":3332083865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6827","nonce":3332083865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6831","nonce":3332083865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6829","nonce":3332083865}]},"public_addr":"172.21.15.158:6825/3332083865","cluster_addr":"172.21.15.158:6827/3332083865","heartbeat_back_addr":"172.21.15.158:6831/3332083865","heartbeat_front_addr":"172.21.15.158:6829/3332083865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:05.997832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:29.314235+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:08.819465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:41.194275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:06.032479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:38.814498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:13.067960+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:38.568479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:03.416713+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:38.349479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:04.096945+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:29.425250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1287712358":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/3770452860":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/718743094":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/1904717532":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:6800/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6800/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/2862184313":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/3621754630":"2024-03-05T08:48:22.593931+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T08:56:27.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.013+0000 7faaee21c640 1 -- 172.21.15.71:0/880909222 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7faad00a1d80 msgr2=0x7faad00a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:27.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.013+0000 7faaee21c640 1 --2- 172.21.15.71:0/880909222 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7faad00a1d80 0x7faad00a4240 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faad800bbe0 tx=0x7faad800b390 comp rx=0 tx=0).stop 2024-03-04T08:56:27.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.013+0000 7faaee21c640 1 -- 172.21.15.71:0/880909222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faae8104a20 msgr2=0x7faae8197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:27.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.013+0000 7faaee21c640 1 --2- 172.21.15.71:0/880909222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faae8104a20 0x7faae8197e00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7faadc00e9f0 tx=0x7faadc00eeb0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.013+0000 7faaee21c640 1 -- 172.21.15.71:0/880909222 shutdown_connections 2024-03-04T08:56:27.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.013+0000 7faaee21c640 1 --2- 172.21.15.71:0/880909222 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7faad00a1d80 0x7faad00a4240 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.013+0000 7faaee21c640 1 --2- 172.21.15.71:0/880909222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faae8107350 0x7faae819ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.013+0000 7faaee21c640 1 --2- 172.21.15.71:0/880909222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faae8104a20 0x7faae8197e00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.013+0000 7faaee21c640 1 --2- 172.21.15.71:0/880909222 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faae81020f0 0x7faae81978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.013+0000 7faaee21c640 1 -- 172.21.15.71:0/880909222 >> 172.21.15.71:0/880909222 conn(0x7faae80fba60 msgr2=0x7faae8102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:27.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.013+0000 7faaee21c640 1 -- 172.21.15.71:0/880909222 shutdown_connections 2024-03-04T08:56:27.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.013+0000 7faaee21c640 1 -- 172.21.15.71:0/880909222 wait complete. 2024-03-04T08:56:27.021 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T08:51:14.933487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-04T08:56:27.021 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T08:56:27.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.134+0000 7feb780aa640 1 -- 172.21.15.71:0/62421970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb70102080 msgr2=0x7feb70108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:27.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.134+0000 7feb780aa640 1 --2- 172.21.15.71:0/62421970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb70102080 0x7feb70108910 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7feb68009ca0 tx=0x7feb6802c0f0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.134+0000 7feb780aa640 1 -- 172.21.15.71:0/62421970 shutdown_connections 2024-03-04T08:56:27.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.134+0000 7feb780aa640 1 --2- 172.21.15.71:0/62421970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb70102080 0x7feb70108910 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.134+0000 7feb780aa640 1 --2- 172.21.15.71:0/62421970 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feb701016c0 0x7feb70101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.134+0000 7feb780aa640 1 --2- 172.21.15.71:0/62421970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb701004d0 0x7feb701008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.134+0000 7feb780aa640 1 -- 172.21.15.71:0/62421970 >> 172.21.15.71:0/62421970 conn(0x7feb700fba60 msgr2=0x7feb700fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:27.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.134+0000 7feb780aa640 1 -- 172.21.15.71:0/62421970 shutdown_connections 2024-03-04T08:56:27.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.134+0000 7feb780aa640 1 -- 172.21.15.71:0/62421970 wait complete. 2024-03-04T08:56:27.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.134+0000 7feb780aa640 1 Processor -- start 2024-03-04T08:56:27.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.134+0000 7feb780aa640 1 -- start start 2024-03-04T08:56:27.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.135+0000 7feb780aa640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb701004d0 0x7feb701978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:27.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.135+0000 7feb780aa640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feb701016c0 0x7feb70197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:27.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.135+0000 7feb780aa640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb70102080 0x7feb7019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:27.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.135+0000 7feb780aa640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7010c160 con 0x7feb701004d0 2024-03-04T08:56:27.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.135+0000 7feb780aa640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7010bfe0 con 0x7feb70102080 2024-03-04T08:56:27.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.135+0000 7feb780aa640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7010c2e0 con 0x7feb701016c0 2024-03-04T08:56:27.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.135+0000 7feb75e46640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb701004d0 0x7feb701978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:27.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.135+0000 7feb75e46640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb701004d0 0x7feb701978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47106/0 (socket says 172.21.15.71:47106) 2024-03-04T08:56:27.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.135+0000 7feb75e46640 1 -- 172.21.15.71:0/521168640 learned_addr learned my addr 172.21.15.71:0/521168640 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:27.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.135+0000 7feb76647640 1 --2- 172.21.15.71:0/521168640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb70102080 0x7feb7019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:27.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.135+0000 7feb75645640 1 --2- 172.21.15.71:0/521168640 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feb701016c0 0x7feb70197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:27.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.135+0000 7feb75e46640 1 -- 172.21.15.71:0/521168640 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feb701016c0 msgr2=0x7feb70197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:27.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.135+0000 7feb75e46640 1 --2- 172.21.15.71:0/521168640 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feb701016c0 0x7feb70197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.135+0000 7feb75e46640 1 -- 172.21.15.71:0/521168640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb70102080 msgr2=0x7feb7019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:27.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.135+0000 7feb75e46640 1 --2- 172.21.15.71:0/521168640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb70102080 0x7feb7019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.135+0000 7feb75e46640 1 -- 172.21.15.71:0/521168640 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7019f4d0 con 0x7feb701004d0 2024-03-04T08:56:27.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.135+0000 7feb75e46640 1 --2- 172.21.15.71:0/521168640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb701004d0 0x7feb701978b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7feb6000d950 tx=0x7feb6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:27.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.135+0000 7feb66ffd640 1 -- 172.21.15.71:0/521168640 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb60017020 con 0x7feb701004d0 2024-03-04T08:56:27.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.135+0000 7feb780aa640 1 -- 172.21.15.71:0/521168640 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb7019f7c0 con 0x7feb701004d0 2024-03-04T08:56:27.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.135+0000 7feb66ffd640 1 -- 172.21.15.71:0/521168640 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb60009c30 con 0x7feb701004d0 2024-03-04T08:56:27.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.135+0000 7feb66ffd640 1 -- 172.21.15.71:0/521168640 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6000f740 con 0x7feb701004d0 2024-03-04T08:56:27.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.135+0000 7feb780aa640 1 -- 172.21.15.71:0/521168640 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7019fd60 con 0x7feb701004d0 2024-03-04T08:56:27.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.136+0000 7feb780aa640 1 -- 172.21.15.71:0/521168640 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb7019ffc0 con 0x7feb701004d0 2024-03-04T08:56:27.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.137+0000 7feb66ffd640 1 -- 172.21.15.71:0/521168640 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7feb6001a020 con 0x7feb701004d0 2024-03-04T08:56:27.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.138+0000 7feb66ffd640 1 --2- 172.21.15.71:0/521168640 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7feb5c0a1d80 0x7feb5c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:27.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.138+0000 7feb75645640 1 --2- 172.21.15.71:0/521168640 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7feb5c0a1d80 0x7feb5c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:27.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.138+0000 7feb66ffd640 1 -- 172.21.15.71:0/521168640 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7feb600cd320 con 0x7feb701004d0 2024-03-04T08:56:27.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.138+0000 7feb75645640 1 --2- 172.21.15.71:0/521168640 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7feb5c0a1d80 0x7feb5c0a4240 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7feb6c005fd0 tx=0x7feb6c00b510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:27.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.142+0000 7feb66ffd640 1 -- 172.21.15.71:0/521168640 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7feb60021070 con 0x7feb701004d0 2024-03-04T08:56:27.235 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:26 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:27.236 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:26 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:27.236 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:26 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:27.236 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:26 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:27.236 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:26 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:27.236 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:26 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:56:27.236 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:26 smithi071 ceph-mon[32789]: mds.? [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] up:boot 2024-03-04T08:56:27.236 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:26 smithi071 ceph-mon[32789]: fsmap 12 up:standby 2024-03-04T08:56:27.236 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:26 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-04T08:56:27.236 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:26 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1712701574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:27.236 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:26 smithi071 ceph-mon[32789]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:27.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:26 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:27.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:26 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:27.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:26 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:27.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:26 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:27.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:26 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:27.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:26 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:56:27.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:26 smithi150 ceph-mon[36807]: mds.? [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] up:boot 2024-03-04T08:56:27.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:26 smithi150 ceph-mon[36807]: fsmap 12 up:standby 2024-03-04T08:56:27.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:26 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-04T08:56:27.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:26 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1712701574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:27.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:26 smithi150 ceph-mon[36807]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:27.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:26 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:27.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:26 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:27.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:26 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:27.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:26 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:27.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:26 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:27.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:26 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:56:27.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:26 smithi158 ceph-mon[37241]: mds.? [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] up:boot 2024-03-04T08:56:27.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:26 smithi158 ceph-mon[37241]: fsmap 12 up:standby 2024-03-04T08:56:27.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:26 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-04T08:56:27.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:26 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1712701574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:27.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:26 smithi158 ceph-mon[37241]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:27.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.330+0000 7feb780aa640 1 -- 172.21.15.71:0/521168640 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb701a0190 con 0x7feb701004d0 2024-03-04T08:56:27.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.330+0000 7feb66ffd640 1 -- 172.21.15.71:0/521168640 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7feb60009de0 con 0x7feb701004d0 2024-03-04T08:56:27.327 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-03-04T08:56:27.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.332+0000 7feb780aa640 1 -- 172.21.15.71:0/521168640 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7feb5c0a1d80 msgr2=0x7feb5c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:27.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.332+0000 7feb780aa640 1 --2- 172.21.15.71:0/521168640 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7feb5c0a1d80 0x7feb5c0a4240 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7feb6c005fd0 tx=0x7feb6c00b510 comp rx=0 tx=0).stop 2024-03-04T08:56:27.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.332+0000 7feb780aa640 1 -- 172.21.15.71:0/521168640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb701004d0 msgr2=0x7feb701978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:27.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.332+0000 7feb780aa640 1 --2- 172.21.15.71:0/521168640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb701004d0 0x7feb701978b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7feb6000d950 tx=0x7feb6000de10 comp rx=0 tx=0).stop 2024-03-04T08:56:27.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.332+0000 7feb780aa640 1 -- 172.21.15.71:0/521168640 shutdown_connections 2024-03-04T08:56:27.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.332+0000 7feb780aa640 1 --2- 172.21.15.71:0/521168640 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7feb5c0a1d80 0x7feb5c0a4240 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.332+0000 7feb780aa640 1 --2- 172.21.15.71:0/521168640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb70102080 0x7feb7019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.332+0000 7feb780aa640 1 --2- 172.21.15.71:0/521168640 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feb701016c0 0x7feb70197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.332+0000 7feb780aa640 1 --2- 172.21.15.71:0/521168640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb701004d0 0x7feb701978b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.332+0000 7feb780aa640 1 -- 172.21.15.71:0/521168640 >> 172.21.15.71:0/521168640 conn(0x7feb700fba60 msgr2=0x7feb70068730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:27.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.333+0000 7feb780aa640 1 -- 172.21.15.71:0/521168640 shutdown_connections 2024-03-04T08:56:27.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.333+0000 7feb780aa640 1 -- 172.21.15.71:0/521168640 wait complete. 2024-03-04T08:56:27.340 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-04T08:56:27.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.447+0000 7f898f281640 1 -- 172.21.15.71:0/3973721041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89881016e0 msgr2=0x7f8988101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:27.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.447+0000 7f898f281640 1 --2- 172.21.15.71:0/3973721041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89881016e0 0x7f8988101b60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8978009900 tx=0x7f897802bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:27.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.447+0000 7f898f281640 1 -- 172.21.15.71:0/3973721041 shutdown_connections 2024-03-04T08:56:27.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.447+0000 7f898f281640 1 --2- 172.21.15.71:0/3973721041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89881020a0 0x7f8988108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.447+0000 7f898f281640 1 --2- 172.21.15.71:0/3973721041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89881016e0 0x7f8988101b60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.447+0000 7f898f281640 1 --2- 172.21.15.71:0/3973721041 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89881004f0 0x7f8988100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.447+0000 7f898f281640 1 -- 172.21.15.71:0/3973721041 >> 172.21.15.71:0/3973721041 conn(0x7f89880fba80 msgr2=0x7f89880fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:27.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.447+0000 7f898f281640 1 -- 172.21.15.71:0/3973721041 shutdown_connections 2024-03-04T08:56:27.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.447+0000 7f898f281640 1 -- 172.21.15.71:0/3973721041 wait complete. 2024-03-04T08:56:27.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.447+0000 7f898f281640 1 Processor -- start 2024-03-04T08:56:27.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.447+0000 7f898f281640 1 -- start start 2024-03-04T08:56:27.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.447+0000 7f898f281640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89881004f0 0x7f8988199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:27.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.447+0000 7f898f281640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89881016e0 0x7f8988199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:27.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.447+0000 7f898f281640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89881020a0 0x7f89881a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:27.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.448+0000 7f898f281640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898810c180 con 0x7f89881016e0 2024-03-04T08:56:27.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.448+0000 7f898f281640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898810c000 con 0x7f89881004f0 2024-03-04T08:56:27.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.448+0000 7f898f281640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898810c300 con 0x7f89881020a0 2024-03-04T08:56:27.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.448+0000 7f898c81c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89881016e0 0x7f8988199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:27.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.448+0000 7f898c81c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89881016e0 0x7f8988199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47132/0 (socket says 172.21.15.71:47132) 2024-03-04T08:56:27.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.448+0000 7f898c81c640 1 -- 172.21.15.71:0/2595772118 learned_addr learned my addr 172.21.15.71:0/2595772118 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:27.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.448+0000 7f898d01d640 1 --2- 172.21.15.71:0/2595772118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89881004f0 0x7f8988199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:27.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.448+0000 7f898d81e640 1 --2- 172.21.15.71:0/2595772118 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89881020a0 0x7f89881a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:27.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.448+0000 7f898c81c640 1 -- 172.21.15.71:0/2595772118 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89881020a0 msgr2=0x7f89881a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:27.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.448+0000 7f898c81c640 1 --2- 172.21.15.71:0/2595772118 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89881020a0 0x7f89881a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.448+0000 7f898c81c640 1 -- 172.21.15.71:0/2595772118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89881004f0 msgr2=0x7f8988199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:27.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.448+0000 7f898c81c640 1 --2- 172.21.15.71:0/2595772118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89881004f0 0x7f8988199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.448+0000 7f898c81c640 1 -- 172.21.15.71:0/2595772118 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89881a16d0 con 0x7f89881016e0 2024-03-04T08:56:27.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.448+0000 7f898d81e640 1 --2- 172.21.15.71:0/2595772118 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89881020a0 0x7f89881a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:56:27.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.448+0000 7f898c81c640 1 --2- 172.21.15.71:0/2595772118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89881016e0 0x7f8988199ff0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8978009230 tx=0x7f8978005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:27.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.448+0000 7f897e7fc640 1 -- 172.21.15.71:0/2595772118 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8978041070 con 0x7f89881016e0 2024-03-04T08:56:27.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.448+0000 7f897e7fc640 1 -- 172.21.15.71:0/2595772118 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f89780054e0 con 0x7f89881016e0 2024-03-04T08:56:27.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.448+0000 7f897e7fc640 1 -- 172.21.15.71:0/2595772118 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8978034440 con 0x7f89881016e0 2024-03-04T08:56:27.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.448+0000 7f898f281640 1 -- 172.21.15.71:0/2595772118 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89881a1960 con 0x7f89881016e0 2024-03-04T08:56:27.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.448+0000 7f898f281640 1 -- 172.21.15.71:0/2595772118 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89881a1e00 con 0x7f89881016e0 2024-03-04T08:56:27.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.449+0000 7f898f281640 1 -- 172.21.15.71:0/2595772118 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89881a2030 con 0x7f89881016e0 2024-03-04T08:56:27.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.450+0000 7f897e7fc640 1 -- 172.21.15.71:0/2595772118 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f8978044070 con 0x7f89881016e0 2024-03-04T08:56:27.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.451+0000 7f897e7fc640 1 --2- 172.21.15.71:0/2595772118 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f89740a1e00 0x7f89740a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:27.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.451+0000 7f898d01d640 1 --2- 172.21.15.71:0/2595772118 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f89740a1e00 0x7f89740a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:27.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.451+0000 7f897e7fc640 1 -- 172.21.15.71:0/2595772118 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f89780ef570 con 0x7f89881016e0 2024-03-04T08:56:27.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.451+0000 7f898d01d640 1 --2- 172.21.15.71:0/2595772118 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f89740a1e00 0x7f89740a42c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8984005fd0 tx=0x7f898400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:27.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.455+0000 7f897e7fc640 1 -- 172.21.15.71:0/2595772118 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f89780bce40 con 0x7f89881016e0 2024-03-04T08:56:27.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.637+0000 7f898f281640 1 -- 172.21.15.71:0/2595772118 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f89881a2320 con 0x7f89881016e0 2024-03-04T08:56:27.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.638+0000 7f897e7fc640 1 -- 172.21.15.71:0/2595772118 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) v1 ==== 84+0+65 (secure 0 0 0) 0x7f89780bc740 con 0x7f89881016e0 2024-03-04T08:56:27.634 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:27.634 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-03-04T08:56:27.634 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T08:56:27.634 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2024-03-04T08:56:27.634 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2024-03-04T08:56:27.634 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T08:56:27.634 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-03-04T08:56:27.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.640+0000 7f898f281640 1 -- 172.21.15.71:0/2595772118 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f89740a1e00 msgr2=0x7f89740a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:27.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.640+0000 7f898f281640 1 --2- 172.21.15.71:0/2595772118 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f89740a1e00 0x7f89740a42c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8984005fd0 tx=0x7f898400b390 comp rx=0 tx=0).stop 2024-03-04T08:56:27.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.640+0000 7f898f281640 1 -- 172.21.15.71:0/2595772118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89881016e0 msgr2=0x7f8988199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:27.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.640+0000 7f898f281640 1 --2- 172.21.15.71:0/2595772118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89881016e0 0x7f8988199ff0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8978009230 tx=0x7f8978005dd0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.640+0000 7f898f281640 1 -- 172.21.15.71:0/2595772118 shutdown_connections 2024-03-04T08:56:27.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.640+0000 7f898f281640 1 --2- 172.21.15.71:0/2595772118 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f89740a1e00 0x7f89740a42c0 secure :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8984005fd0 tx=0x7f898400b390 comp rx=0 tx=0).stop 2024-03-04T08:56:27.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.640+0000 7f898f281640 1 --2- 172.21.15.71:0/2595772118 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f89881020a0 0x7f89881a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.640+0000 7f898f281640 1 --2- 172.21.15.71:0/2595772118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89881016e0 0x7f8988199ff0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.640+0000 7f898f281640 1 --2- 172.21.15.71:0/2595772118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89881004f0 0x7f8988199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.640+0000 7f898f281640 1 -- 172.21.15.71:0/2595772118 >> 172.21.15.71:0/2595772118 conn(0x7f89880fba80 msgr2=0x7f89880fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:27.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.640+0000 7f898f281640 1 -- 172.21.15.71:0/2595772118 shutdown_connections 2024-03-04T08:56:27.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.640+0000 7f898f281640 1 -- 172.21.15.71:0/2595772118 wait complete. 2024-03-04T08:56:27.647 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-04T08:56:27.648 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-04T08:56:27.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.770+0000 7f38936c9640 1 -- 172.21.15.71:0/1653891644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f388c103190 msgr2=0x7f388c109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:27.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.770+0000 7f38936c9640 1 --2- 172.21.15.71:0/1653891644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f388c103190 0x7f388c109a20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3884009ca0 tx=0x7f388402c0f0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.770+0000 7f38936c9640 1 -- 172.21.15.71:0/1653891644 shutdown_connections 2024-03-04T08:56:27.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.770+0000 7f38936c9640 1 --2- 172.21.15.71:0/1653891644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f388c103190 0x7f388c109a20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.770+0000 7f38936c9640 1 --2- 172.21.15.71:0/1653891644 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f388c1027d0 0x7f388c102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.770+0000 7f38936c9640 1 --2- 172.21.15.71:0/1653891644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f388c1015e0 0x7f388c101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.770+0000 7f38936c9640 1 -- 172.21.15.71:0/1653891644 >> 172.21.15.71:0/1653891644 conn(0x7f388c0fcb30 msgr2=0x7f388c0fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:27.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.771+0000 7f38936c9640 1 -- 172.21.15.71:0/1653891644 shutdown_connections 2024-03-04T08:56:27.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.771+0000 7f38936c9640 1 -- 172.21.15.71:0/1653891644 wait complete. 2024-03-04T08:56:27.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.771+0000 7f38936c9640 1 Processor -- start 2024-03-04T08:56:27.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.771+0000 7f38936c9640 1 -- start start 2024-03-04T08:56:27.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.771+0000 7f38936c9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f388c1015e0 0x7f388c075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:27.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.771+0000 7f38936c9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f388c1027d0 0x7f388c0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:27.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.771+0000 7f38936c9640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f388c103190 0x7f388c073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:27.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.771+0000 7f38936c9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c10d240 con 0x7f388c1027d0 2024-03-04T08:56:27.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.771+0000 7f38936c9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c10d0c0 con 0x7f388c1015e0 2024-03-04T08:56:27.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.771+0000 7f38936c9640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c10d3c0 con 0x7f388c103190 2024-03-04T08:56:27.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.771+0000 7f3890c64640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f388c1027d0 0x7f388c0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:27.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.771+0000 7f3890c64640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f388c1027d0 0x7f388c0739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47138/0 (socket says 172.21.15.71:47138) 2024-03-04T08:56:27.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.771+0000 7f3890c64640 1 -- 172.21.15.71:0/3638287280 learned_addr learned my addr 172.21.15.71:0/3638287280 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:27.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.771+0000 7f3891465640 1 --2- 172.21.15.71:0/3638287280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f388c1015e0 0x7f388c075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:27.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.771+0000 7f3891c66640 1 --2- 172.21.15.71:0/3638287280 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f388c103190 0x7f388c073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:27.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.772+0000 7f3890c64640 1 -- 172.21.15.71:0/3638287280 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f388c103190 msgr2=0x7f388c073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:27.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.772+0000 7f3890c64640 1 --2- 172.21.15.71:0/3638287280 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f388c103190 0x7f388c073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.772+0000 7f3890c64640 1 -- 172.21.15.71:0/3638287280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f388c1015e0 msgr2=0x7f388c075380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:27.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.772+0000 7f3890c64640 1 --2- 172.21.15.71:0/3638287280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f388c1015e0 0x7f388c075380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:27.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.772+0000 7f3890c64640 1 -- 172.21.15.71:0/3638287280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c074790 con 0x7f388c1027d0 2024-03-04T08:56:27.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.772+0000 7f3890c64640 1 --2- 172.21.15.71:0/3638287280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f388c1027d0 0x7f388c0739c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f387c00d950 tx=0x7f387c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:27.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.772+0000 7f38827fc640 1 -- 172.21.15.71:0/3638287280 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c017030 con 0x7f388c1027d0 2024-03-04T08:56:27.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.772+0000 7f38827fc640 1 -- 172.21.15.71:0/3638287280 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f387c004020 con 0x7f388c1027d0 2024-03-04T08:56:27.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.772+0000 7f38936c9640 1 -- 172.21.15.71:0/3638287280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f388c1a19a0 con 0x7f388c1027d0 2024-03-04T08:56:27.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.772+0000 7f38827fc640 1 -- 172.21.15.71:0/3638287280 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c0047a0 con 0x7f388c1027d0 2024-03-04T08:56:27.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.772+0000 7f38936c9640 1 -- 172.21.15.71:0/3638287280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f388c1a1ef0 con 0x7f388c1027d0 2024-03-04T08:56:27.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.773+0000 7f38936c9640 1 -- 172.21.15.71:0/3638287280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f388c108900 con 0x7f388c1027d0 2024-03-04T08:56:27.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.774+0000 7f38827fc640 1 -- 172.21.15.71:0/3638287280 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f387c0041d0 con 0x7f388c1027d0 2024-03-04T08:56:27.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.774+0000 7f38827fc640 1 --2- 172.21.15.71:0/3638287280 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f38780a1d80 0x7f38780a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:27.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.775+0000 7f3891465640 1 --2- 172.21.15.71:0/3638287280 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f38780a1d80 0x7f38780a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:27.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.775+0000 7f38827fc640 1 -- 172.21.15.71:0/3638287280 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f387c0cd2b0 con 0x7f388c1027d0 2024-03-04T08:56:27.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.775+0000 7f3891465640 1 --2- 172.21.15.71:0/3638287280 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f38780a1d80 0x7f38780a4240 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f388c102630 tx=0x7f3888005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:27.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:27.780+0000 7f38827fc640 1 -- 172.21.15.71:0/3638287280 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f387c09ab80 con 0x7f388c1027d0 2024-03-04T08:56:28.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:28.018+0000 7f38936c9640 1 -- 172.21.15.71:0/3638287280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f388c02cff0 con 0x7f388c1027d0 2024-03-04T08:56:28.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:27 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/880909222' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:28.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:27 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/521168640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:56:28.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:27 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2595772118' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-04T08:56:28.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:27 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:28.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:27 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:28.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:27 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:28.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:27 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:28.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:27 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:28.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:27 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/880909222' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:28.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:27 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/521168640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:56:28.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:27 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2595772118' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-04T08:56:28.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:27 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:28.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:27 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:28.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:27 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:28.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:27 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:28.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:27 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:28.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:27 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/880909222' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:28.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:27 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/521168640' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:56:28.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:27 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2595772118' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-04T08:56:28.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:27 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:28.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:27 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:28.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:27 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:28.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:27 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:28.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:27 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:28.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:28.966+0000 7f38827fc640 1 -- 172.21.15.71:0/3638287280 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) v1 ==== 138+0+0 (secure 0 0 0) 0x7f387c09a480 con 0x7f388c1027d0 2024-03-04T08:56:28.962 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2024-03-04T08:56:28.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:28.969+0000 7f38936c9640 1 -- 172.21.15.71:0/3638287280 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f38780a1d80 msgr2=0x7f38780a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:28.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:28.969+0000 7f38936c9640 1 --2- 172.21.15.71:0/3638287280 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f38780a1d80 0x7f38780a4240 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f388c102630 tx=0x7f3888005f90 comp rx=0 tx=0).stop 2024-03-04T08:56:28.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:28.969+0000 7f38936c9640 1 -- 172.21.15.71:0/3638287280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f388c1027d0 msgr2=0x7f388c0739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:28.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:28.969+0000 7f38936c9640 1 --2- 172.21.15.71:0/3638287280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f388c1027d0 0x7f388c0739c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f387c00d950 tx=0x7f387c00de10 comp rx=0 tx=0).stop 2024-03-04T08:56:28.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:28.969+0000 7f38936c9640 1 -- 172.21.15.71:0/3638287280 shutdown_connections 2024-03-04T08:56:28.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:28.969+0000 7f38936c9640 1 --2- 172.21.15.71:0/3638287280 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f38780a1d80 0x7f38780a4240 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:28.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:28.969+0000 7f38936c9640 1 --2- 172.21.15.71:0/3638287280 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f388c103190 0x7f388c073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:28.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:28.969+0000 7f38936c9640 1 --2- 172.21.15.71:0/3638287280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f388c1027d0 0x7f388c0739c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:28.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:28.969+0000 7f38936c9640 1 --2- 172.21.15.71:0/3638287280 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f388c1015e0 0x7f388c075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:28.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:28.969+0000 7f38936c9640 1 -- 172.21.15.71:0/3638287280 >> 172.21.15.71:0/3638287280 conn(0x7f388c0fcb30 msgr2=0x7f388c105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:28.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:28.969+0000 7f38936c9640 1 -- 172.21.15.71:0/3638287280 shutdown_connections 2024-03-04T08:56:28.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:28.970+0000 7f38936c9640 1 -- 172.21.15.71:0/3638287280 wait complete. 2024-03-04T08:56:28.978 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-04T08:56:29.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.095+0000 7f3420b93640 1 -- 172.21.15.71:0/3280826997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f341c1049e0 msgr2=0x7f341c106dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:29.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.095+0000 7f3420b93640 1 --2- 172.21.15.71:0/3280826997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f341c1049e0 0x7f341c106dd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3404009900 tx=0x7f340402bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:29.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.095+0000 7f3420b93640 1 -- 172.21.15.71:0/3280826997 shutdown_connections 2024-03-04T08:56:29.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.095+0000 7f3420b93640 1 --2- 172.21.15.71:0/3280826997 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f341c107310 0x7f341c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:29.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.095+0000 7f3420b93640 1 --2- 172.21.15.71:0/3280826997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f341c1049e0 0x7f341c106dd0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:29.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.095+0000 7f3420b93640 1 --2- 172.21.15.71:0/3280826997 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f341c1020b0 0x7f341c1044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:29.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.095+0000 7f3420b93640 1 -- 172.21.15.71:0/3280826997 >> 172.21.15.71:0/3280826997 conn(0x7f341c0fba60 msgr2=0x7f341c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:29.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.095+0000 7f3420b93640 1 -- 172.21.15.71:0/3280826997 shutdown_connections 2024-03-04T08:56:29.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.095+0000 7f3420b93640 1 -- 172.21.15.71:0/3280826997 wait complete. 2024-03-04T08:56:29.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.096+0000 7f3420b93640 1 Processor -- start 2024-03-04T08:56:29.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.096+0000 7f3420b93640 1 -- start start 2024-03-04T08:56:29.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.096+0000 7f3420b93640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f341c1020b0 0x7f341c197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:29.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.096+0000 7f3420b93640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f341c1049e0 0x7f341c197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:29.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.096+0000 7f3420b93640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f341c107310 0x7f341c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:29.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.096+0000 7f3420b93640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c10d260 con 0x7f341c1020b0 2024-03-04T08:56:29.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.096+0000 7f3420b93640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c10d0e0 con 0x7f341c107310 2024-03-04T08:56:29.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.096+0000 7f3420b93640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c10d3e0 con 0x7f341c1049e0 2024-03-04T08:56:29.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.096+0000 7f341a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f341c1020b0 0x7f341c197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:29.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.096+0000 7f341a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f341c1020b0 0x7f341c197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47150/0 (socket says 172.21.15.71:47150) 2024-03-04T08:56:29.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.096+0000 7f341a59c640 1 -- 172.21.15.71:0/1241879249 learned_addr learned my addr 172.21.15.71:0/1241879249 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:29.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.096+0000 7f3413fff640 1 --2- 172.21.15.71:0/1241879249 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f341c1049e0 0x7f341c197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:29.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.096+0000 7f341ad9d640 1 --2- 172.21.15.71:0/1241879249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f341c107310 0x7f341c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:29.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.097+0000 7f341a59c640 1 -- 172.21.15.71:0/1241879249 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f341c1049e0 msgr2=0x7f341c197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:29.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.097+0000 7f341a59c640 1 --2- 172.21.15.71:0/1241879249 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f341c1049e0 0x7f341c197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:29.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.097+0000 7f341a59c640 1 -- 172.21.15.71:0/1241879249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f341c107310 msgr2=0x7f341c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:29.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.097+0000 7f341a59c640 1 --2- 172.21.15.71:0/1241879249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f341c107310 0x7f341c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:29.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.097+0000 7f341a59c640 1 -- 172.21.15.71:0/1241879249 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c19f590 con 0x7f341c1020b0 2024-03-04T08:56:29.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.097+0000 7f3413fff640 1 --2- 172.21.15.71:0/1241879249 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f341c1049e0 0x7f341c197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:56:29.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.097+0000 7f341a59c640 1 --2- 172.21.15.71:0/1241879249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f341c1020b0 0x7f341c197970 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f340c00e9f0 tx=0x7f340c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:29.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.097+0000 7f34137fe640 1 -- 172.21.15.71:0/1241879249 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c00cd80 con 0x7f341c1020b0 2024-03-04T08:56:29.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.097+0000 7f34137fe640 1 -- 172.21.15.71:0/1241879249 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f340c005500 con 0x7f341c1020b0 2024-03-04T08:56:29.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.097+0000 7f3420b93640 1 -- 172.21.15.71:0/1241879249 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341c19f880 con 0x7f341c1020b0 2024-03-04T08:56:29.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.097+0000 7f34137fe640 1 -- 172.21.15.71:0/1241879249 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c010690 con 0x7f341c1020b0 2024-03-04T08:56:29.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.097+0000 7f3420b93640 1 -- 172.21.15.71:0/1241879249 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341c19fd50 con 0x7f341c1020b0 2024-03-04T08:56:29.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.098+0000 7f3420b93640 1 -- 172.21.15.71:0/1241879249 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f341c0fd550 con 0x7f341c1020b0 2024-03-04T08:56:29.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.099+0000 7f34137fe640 1 -- 172.21.15.71:0/1241879249 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f340c0046c0 con 0x7f341c1020b0 2024-03-04T08:56:29.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.100+0000 7f34137fe640 1 --2- 172.21.15.71:0/1241879249 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f34080a1d80 0x7f34080a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:29.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.100+0000 7f3413fff640 1 --2- 172.21.15.71:0/1241879249 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f34080a1d80 0x7f34080a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:29.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.100+0000 7f34137fe640 1 -- 172.21.15.71:0/1241879249 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f340c0ce000 con 0x7f341c1020b0 2024-03-04T08:56:29.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.100+0000 7f3413fff640 1 --2- 172.21.15.71:0/1241879249 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f34080a1d80 0x7f34080a4240 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3404000c00 tx=0x7f3404005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.108+0000 7f34137fe640 1 -- 172.21.15.71:0/1241879249 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f340c053280 con 0x7f341c1020b0 2024-03-04T08:56:29.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:28 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3638287280' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-04T08:56:29.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:28 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:29.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:28 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:29.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:28 smithi150 ceph-mon[36807]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:29.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:28 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:29.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:28 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:56:29.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:28 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:29.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.291+0000 7f3420b93640 1 -- 172.21.15.71:0/1241879249 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f341c02cff0 con 0x7f341c1020b0 2024-03-04T08:56:29.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:28 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3638287280' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-04T08:56:29.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:28 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:29.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:28 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:29.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:28 smithi158 ceph-mon[37241]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:29.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:28 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:29.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:28 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:56:29.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:28 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:29.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:28 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3638287280' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-04T08:56:29.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:28 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:29.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:28 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:29.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:28 smithi071 ceph-mon[32789]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:29.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:28 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:29.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:28 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:56:29.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:28 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:29.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.968+0000 7f34137fe640 1 -- 172.21.15.71:0/1241879249 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v85) v1 ==== 171+0+0 (secure 0 0 0) 0x7f340c09b760 con 0x7f341c1020b0 2024-03-04T08:56:29.964 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2024-03-04T08:56:29.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.972+0000 7f3420b93640 1 -- 172.21.15.71:0/1241879249 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f34080a1d80 msgr2=0x7f34080a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:29.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.972+0000 7f3420b93640 1 --2- 172.21.15.71:0/1241879249 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f34080a1d80 0x7f34080a4240 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3404000c00 tx=0x7f3404005fd0 comp rx=0 tx=0).stop 2024-03-04T08:56:29.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.972+0000 7f3420b93640 1 -- 172.21.15.71:0/1241879249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f341c1020b0 msgr2=0x7f341c197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:29.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.972+0000 7f3420b93640 1 --2- 172.21.15.71:0/1241879249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f341c1020b0 0x7f341c197970 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f340c00e9f0 tx=0x7f340c00edf0 comp rx=0 tx=0).stop 2024-03-04T08:56:29.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.976+0000 7f3420b93640 1 -- 172.21.15.71:0/1241879249 shutdown_connections 2024-03-04T08:56:29.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.976+0000 7f3420b93640 1 --2- 172.21.15.71:0/1241879249 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f34080a1d80 0x7f34080a4240 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:29.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.976+0000 7f3420b93640 1 --2- 172.21.15.71:0/1241879249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f341c107310 0x7f341c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:29.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.976+0000 7f3420b93640 1 --2- 172.21.15.71:0/1241879249 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f341c1049e0 0x7f341c197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:29.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.976+0000 7f3420b93640 1 --2- 172.21.15.71:0/1241879249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f341c1020b0 0x7f341c197970 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:29.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.976+0000 7f3420b93640 1 -- 172.21.15.71:0/1241879249 >> 172.21.15.71:0/1241879249 conn(0x7f341c0fba60 msgr2=0x7f341c102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:29.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.976+0000 7f3420b93640 1 -- 172.21.15.71:0/1241879249 shutdown_connections 2024-03-04T08:56:29.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:29.976+0000 7f3420b93640 1 -- 172.21.15.71:0/1241879249 wait complete. 2024-03-04T08:56:29.999 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-04T08:56:30.086 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:29 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3638287280' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-04T08:56:30.086 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:29 smithi158 ceph-mon[37241]: osdmap e84: 12 total, 12 up, 12 in 2024-03-04T08:56:30.086 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:29 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:56:30.086 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:29 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1241879249' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-04T08:56:30.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.109+0000 7f86e2be2640 1 -- 172.21.15.71:0/3457838727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86dc102090 msgr2=0x7f86dc108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:30.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.109+0000 7f86e2be2640 1 --2- 172.21.15.71:0/3457838727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86dc102090 0x7f86dc108920 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009ca0 tx=0x7f86d402c0f0 comp rx=0 tx=0).stop 2024-03-04T08:56:30.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.109+0000 7f86e2be2640 1 -- 172.21.15.71:0/3457838727 shutdown_connections 2024-03-04T08:56:30.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.109+0000 7f86e2be2640 1 --2- 172.21.15.71:0/3457838727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86dc102090 0x7f86dc108920 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:30.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.109+0000 7f86e2be2640 1 --2- 172.21.15.71:0/3457838727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86dc1016d0 0x7f86dc101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:30.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.109+0000 7f86e2be2640 1 --2- 172.21.15.71:0/3457838727 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86dc1004e0 0x7f86dc100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:30.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.109+0000 7f86e2be2640 1 -- 172.21.15.71:0/3457838727 >> 172.21.15.71:0/3457838727 conn(0x7f86dc0fba30 msgr2=0x7f86dc0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:30.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.109+0000 7f86e2be2640 1 -- 172.21.15.71:0/3457838727 shutdown_connections 2024-03-04T08:56:30.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.109+0000 7f86e2be2640 1 -- 172.21.15.71:0/3457838727 wait complete. 2024-03-04T08:56:30.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.109+0000 7f86e2be2640 1 Processor -- start 2024-03-04T08:56:30.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.109+0000 7f86e2be2640 1 -- start start 2024-03-04T08:56:30.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.110+0000 7f86e2be2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86dc1004e0 0x7f86dc06a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:30.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.110+0000 7f86e2be2640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86dc1016d0 0x7f86dc19c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:30.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.110+0000 7f86e2be2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86dc102090 0x7f86dc19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:30.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.110+0000 7f86e2be2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc10c170 con 0x7f86dc102090 2024-03-04T08:56:30.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.110+0000 7f86e2be2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc10bff0 con 0x7f86dc1004e0 2024-03-04T08:56:30.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.110+0000 7f86e2be2640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc10c2f0 con 0x7f86dc1016d0 2024-03-04T08:56:30.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.110+0000 7f86e117f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86dc102090 0x7f86dc19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:30.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.110+0000 7f86e117f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86dc102090 0x7f86dc19ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47158/0 (socket says 172.21.15.71:47158) 2024-03-04T08:56:30.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.110+0000 7f86e117f640 1 -- 172.21.15.71:0/218270231 learned_addr learned my addr 172.21.15.71:0/218270231 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:30.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.110+0000 7f86e097e640 1 --2- 172.21.15.71:0/218270231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86dc1004e0 0x7f86dc06a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:30.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.110+0000 7f86dbfff640 1 --2- 172.21.15.71:0/218270231 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86dc1016d0 0x7f86dc19c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:30.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.110+0000 7f86e117f640 1 -- 172.21.15.71:0/218270231 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86dc1016d0 msgr2=0x7f86dc19c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:30.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.110+0000 7f86e117f640 1 --2- 172.21.15.71:0/218270231 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86dc1016d0 0x7f86dc19c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:30.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.110+0000 7f86e117f640 1 -- 172.21.15.71:0/218270231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86dc1004e0 msgr2=0x7f86dc06a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:30.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.110+0000 7f86e117f640 1 --2- 172.21.15.71:0/218270231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86dc1004e0 0x7f86dc06a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:30.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.110+0000 7f86e117f640 1 -- 172.21.15.71:0/218270231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc19f4a0 con 0x7f86dc102090 2024-03-04T08:56:30.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.110+0000 7f86e117f640 1 --2- 172.21.15.71:0/218270231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86dc102090 0x7f86dc19ebe0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f86d402c700 tx=0x7f86d4032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:30.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.110+0000 7f86d9ffb640 1 -- 172.21.15.71:0/218270231 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d4039040 con 0x7f86dc102090 2024-03-04T08:56:30.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.110+0000 7f86d9ffb640 1 -- 172.21.15.71:0/218270231 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86d4044070 con 0x7f86dc102090 2024-03-04T08:56:30.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.110+0000 7f86d9ffb640 1 -- 172.21.15.71:0/218270231 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d40042a0 con 0x7f86dc102090 2024-03-04T08:56:30.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.110+0000 7f86e2be2640 1 -- 172.21.15.71:0/218270231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86dc19f730 con 0x7f86dc102090 2024-03-04T08:56:30.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.111+0000 7f86e2be2640 1 -- 172.21.15.71:0/218270231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86dc19fbd0 con 0x7f86dc102090 2024-03-04T08:56:30.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.112+0000 7f86e2be2640 1 -- 172.21.15.71:0/218270231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86dc107800 con 0x7f86dc102090 2024-03-04T08:56:30.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.113+0000 7f86d9ffb640 1 -- 172.21.15.71:0/218270231 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f86d4003970 con 0x7f86dc102090 2024-03-04T08:56:30.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.113+0000 7f86d9ffb640 1 --2- 172.21.15.71:0/218270231 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f86cc0a1e00 0x7f86cc0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:30.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.113+0000 7f86e097e640 1 --2- 172.21.15.71:0/218270231 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f86cc0a1e00 0x7f86cc0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:30.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.113+0000 7f86d9ffb640 1 -- 172.21.15.71:0/218270231 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f86d40f0640 con 0x7f86dc102090 2024-03-04T08:56:30.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.114+0000 7f86e097e640 1 --2- 172.21.15.71:0/218270231 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f86cc0a1e00 0x7f86cc0a42c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f86d0005fd0 tx=0x7f86d000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:30.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.117+0000 7f86d9ffb640 1 -- 172.21.15.71:0/218270231 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f86d40bdcb0 con 0x7f86dc102090 2024-03-04T08:56:30.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:29 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3638287280' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-04T08:56:30.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:29 smithi071 ceph-mon[32789]: osdmap e84: 12 total, 12 up, 12 in 2024-03-04T08:56:30.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:29 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:56:30.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:29 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1241879249' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-04T08:56:30.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:29 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3638287280' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-04T08:56:30.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:29 smithi150 ceph-mon[36807]: osdmap e84: 12 total, 12 up, 12 in 2024-03-04T08:56:30.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:29 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:56:30.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:29 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1241879249' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-04T08:56:30.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.341+0000 7f86e2be2640 1 -- 172.21.15.71:0/218270231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f86dc0622e0 con 0x7f86dc102090 2024-03-04T08:56:30.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.991+0000 7f86d9ffb640 1 -- 172.21.15.71:0/218270231 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f86d40bd5b0 con 0x7f86dc102090 2024-03-04T08:56:30.987 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 2 and data pool 3 2024-03-04T08:56:30.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.993+0000 7f86e2be2640 1 -- 172.21.15.71:0/218270231 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f86cc0a1e00 msgr2=0x7f86cc0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:30.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.993+0000 7f86e2be2640 1 --2- 172.21.15.71:0/218270231 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f86cc0a1e00 0x7f86cc0a42c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f86d0005fd0 tx=0x7f86d000b390 comp rx=0 tx=0).stop 2024-03-04T08:56:30.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.993+0000 7f86e2be2640 1 -- 172.21.15.71:0/218270231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86dc102090 msgr2=0x7f86dc19ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:30.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.993+0000 7f86e2be2640 1 --2- 172.21.15.71:0/218270231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86dc102090 0x7f86dc19ebe0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f86d402c700 tx=0x7f86d4032f20 comp rx=0 tx=0).stop 2024-03-04T08:56:30.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.993+0000 7f86e2be2640 1 -- 172.21.15.71:0/218270231 shutdown_connections 2024-03-04T08:56:30.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.993+0000 7f86e2be2640 1 --2- 172.21.15.71:0/218270231 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f86cc0a1e00 0x7f86cc0a42c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:30.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.993+0000 7f86e2be2640 1 --2- 172.21.15.71:0/218270231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86dc102090 0x7f86dc19ebe0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:30.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.993+0000 7f86e2be2640 1 --2- 172.21.15.71:0/218270231 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86dc1016d0 0x7f86dc19c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:30.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.993+0000 7f86e2be2640 1 --2- 172.21.15.71:0/218270231 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86dc1004e0 0x7f86dc06a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:30.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.993+0000 7f86e2be2640 1 -- 172.21.15.71:0/218270231 >> 172.21.15.71:0/218270231 conn(0x7f86dc0fba30 msgr2=0x7f86dc1049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:30.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.993+0000 7f86e2be2640 1 -- 172.21.15.71:0/218270231 shutdown_connections 2024-03-04T08:56:30.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:30.993+0000 7f86e2be2640 1 -- 172.21.15.71:0/218270231 wait complete. 2024-03-04T08:56:31.002 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-04T08:56:31.002 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-04T08:56:31.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.113+0000 7f9496c0e640 1 -- 172.21.15.71:0/3312870815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94901015e0 msgr2=0x7f9490101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:31.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.113+0000 7f9496c0e640 1 --2- 172.21.15.71:0/3312870815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94901015e0 0x7f9490101a00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f948c009900 tx=0x7f948c02bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:31.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.113+0000 7f9496c0e640 1 -- 172.21.15.71:0/3312870815 shutdown_connections 2024-03-04T08:56:31.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.113+0000 7f9496c0e640 1 --2- 172.21.15.71:0/3312870815 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9490103190 0x7f9490109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:31.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.113+0000 7f9496c0e640 1 --2- 172.21.15.71:0/3312870815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94901027d0 0x7f9490102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:31.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.113+0000 7f9496c0e640 1 --2- 172.21.15.71:0/3312870815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94901015e0 0x7f9490101a00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:31.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.113+0000 7f9496c0e640 1 -- 172.21.15.71:0/3312870815 >> 172.21.15.71:0/3312870815 conn(0x7f94900fcb70 msgr2=0x7f94900fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:31.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.113+0000 7f9496c0e640 1 -- 172.21.15.71:0/3312870815 shutdown_connections 2024-03-04T08:56:31.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.113+0000 7f9496c0e640 1 -- 172.21.15.71:0/3312870815 wait complete. 2024-03-04T08:56:31.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.113+0000 7f9496c0e640 1 Processor -- start 2024-03-04T08:56:31.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.113+0000 7f9496c0e640 1 -- start start 2024-03-04T08:56:31.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.113+0000 7f9496c0e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94901015e0 0x7f94901978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:31.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.113+0000 7f9496c0e640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f94901027d0 0x7f9490197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:31.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.113+0000 7f9496c0e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9490103190 0x7f949019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:31.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.113+0000 7f9496c0e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949010d230 con 0x7f94901015e0 2024-03-04T08:56:31.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.113+0000 7f9496c0e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949010d0b0 con 0x7f9490103190 2024-03-04T08:56:31.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.113+0000 7f9496c0e640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949010d3b0 con 0x7f94901027d0 2024-03-04T08:56:31.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.114+0000 7f94949aa640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94901015e0 0x7f94901978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:31.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.114+0000 7f94949aa640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94901015e0 0x7f94901978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47186/0 (socket says 172.21.15.71:47186) 2024-03-04T08:56:31.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.114+0000 7f94949aa640 1 -- 172.21.15.71:0/3154464860 learned_addr learned my addr 172.21.15.71:0/3154464860 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:31.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.114+0000 7f9487fff640 1 --2- 172.21.15.71:0/3154464860 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f94901027d0 0x7f9490197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:31.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.114+0000 7f94951ab640 1 --2- 172.21.15.71:0/3154464860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9490103190 0x7f949019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:31.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.114+0000 7f94949aa640 1 -- 172.21.15.71:0/3154464860 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f94901027d0 msgr2=0x7f9490197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:31.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.114+0000 7f94949aa640 1 --2- 172.21.15.71:0/3154464860 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f94901027d0 0x7f9490197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:31.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.114+0000 7f94949aa640 1 -- 172.21.15.71:0/3154464860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9490103190 msgr2=0x7f949019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:31.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.114+0000 7f94949aa640 1 --2- 172.21.15.71:0/3154464860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9490103190 0x7f949019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:31.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.114+0000 7f94949aa640 1 -- 172.21.15.71:0/3154464860 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949019f4e0 con 0x7f94901015e0 2024-03-04T08:56:31.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.114+0000 7f94949aa640 1 --2- 172.21.15.71:0/3154464860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94901015e0 0x7f94901978c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f948c000c00 tx=0x7f948c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:31.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.114+0000 7f9485ffb640 1 -- 172.21.15.71:0/3154464860 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c041070 con 0x7f94901015e0 2024-03-04T08:56:31.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.114+0000 7f9485ffb640 1 -- 172.21.15.71:0/3154464860 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f948c0044d0 con 0x7f94901015e0 2024-03-04T08:56:31.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.114+0000 7f9496c0e640 1 -- 172.21.15.71:0/3154464860 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949019f770 con 0x7f94901015e0 2024-03-04T08:56:31.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.114+0000 7f9485ffb640 1 -- 172.21.15.71:0/3154464860 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c034440 con 0x7f94901015e0 2024-03-04T08:56:31.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.114+0000 7f9496c0e640 1 -- 172.21.15.71:0/3154464860 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949019fc60 con 0x7f94901015e0 2024-03-04T08:56:31.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.115+0000 7f9496c0e640 1 -- 172.21.15.71:0/3154464860 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94901089a0 con 0x7f94901015e0 2024-03-04T08:56:31.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.116+0000 7f9485ffb640 1 -- 172.21.15.71:0/3154464860 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f948c044070 con 0x7f94901015e0 2024-03-04T08:56:31.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.117+0000 7f9485ffb640 1 --2- 172.21.15.71:0/3154464860 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f94780a1d80 0x7f94780a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:31.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.117+0000 7f9487fff640 1 --2- 172.21.15.71:0/3154464860 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f94780a1d80 0x7f94780a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:31.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.117+0000 7f9485ffb640 1 -- 172.21.15.71:0/3154464860 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f948c0ef770 con 0x7f94901015e0 2024-03-04T08:56:31.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.117+0000 7f9487fff640 1 --2- 172.21.15.71:0/3154464860 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f94780a1d80 0x7f94780a4240 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9490198dd0 tx=0x7f9480005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:31.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.121+0000 7f9485ffb640 1 -- 172.21.15.71:0/3154464860 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f948c0bc6a0 con 0x7f94901015e0 2024-03-04T08:56:31.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:30 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1241879249' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-04T08:56:31.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:30 smithi150 ceph-mon[36807]: osdmap e85: 12 total, 12 up, 12 in 2024-03-04T08:56:31.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:30 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/218270231' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-04T08:56:31.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:30 smithi150 ceph-mon[36807]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-04T08:56:31.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:30 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:31.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:30 smithi150 ceph-mon[36807]: osdmap e86: 12 total, 12 up, 12 in 2024-03-04T08:56:31.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:30 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:31.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:30 smithi150 ceph-mon[36807]: pgmap v320: 97 pgs: 4 active+clean, 11 creating+peering, 82 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:31.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:30 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:31.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:30 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:31.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:30 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:31.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:30 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:31.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:30 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:31.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:30 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:56:31.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:30 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:31.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.304+0000 7f9496c0e640 1 -- 172.21.15.71:0/3154464860 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f949002cff0 con 0x7f94901015e0 2024-03-04T08:56:31.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:30 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1241879249' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-04T08:56:31.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:30 smithi158 ceph-mon[37241]: osdmap e85: 12 total, 12 up, 12 in 2024-03-04T08:56:31.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:30 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/218270231' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-04T08:56:31.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:30 smithi158 ceph-mon[37241]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-04T08:56:31.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:30 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:31.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:30 smithi158 ceph-mon[37241]: osdmap e86: 12 total, 12 up, 12 in 2024-03-04T08:56:31.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:30 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:31.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:30 smithi158 ceph-mon[37241]: pgmap v320: 97 pgs: 4 active+clean, 11 creating+peering, 82 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:31.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:30 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:31.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:30 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:31.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:30 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:31.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:30 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:31.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:30 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:31.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:30 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:56:31.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:30 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:31.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:30 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1241879249' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-04T08:56:31.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:30 smithi071 ceph-mon[32789]: osdmap e85: 12 total, 12 up, 12 in 2024-03-04T08:56:31.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:30 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/218270231' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-04T08:56:31.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:30 smithi071 ceph-mon[32789]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-04T08:56:31.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:30 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:31.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:30 smithi071 ceph-mon[32789]: osdmap e86: 12 total, 12 up, 12 in 2024-03-04T08:56:31.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:30 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:31.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:30 smithi071 ceph-mon[32789]: pgmap v320: 97 pgs: 4 active+clean, 11 creating+peering, 82 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:56:31.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:30 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:31.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:30 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:31.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:30 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:31.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:30 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:31.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:30 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:56:31.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:30 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:56:31.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:30 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:56:31.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.641+0000 7f9485ffb640 1 -- 172.21.15.71:0/3154464860 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v87) v1 ==== 157+0+0 (secure 0 0 0) 0x7f948c0c0740 con 0x7f94901015e0 2024-03-04T08:56:31.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.644+0000 7f9496c0e640 1 -- 172.21.15.71:0/3154464860 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f94780a1d80 msgr2=0x7f94780a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:31.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.644+0000 7f9496c0e640 1 --2- 172.21.15.71:0/3154464860 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f94780a1d80 0x7f94780a4240 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9490198dd0 tx=0x7f9480005f90 comp rx=0 tx=0).stop 2024-03-04T08:56:31.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.644+0000 7f9496c0e640 1 -- 172.21.15.71:0/3154464860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94901015e0 msgr2=0x7f94901978c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:31.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.644+0000 7f9496c0e640 1 --2- 172.21.15.71:0/3154464860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94901015e0 0x7f94901978c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f948c000c00 tx=0x7f948c005d50 comp rx=0 tx=0).stop 2024-03-04T08:56:31.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.644+0000 7f9496c0e640 1 -- 172.21.15.71:0/3154464860 shutdown_connections 2024-03-04T08:56:31.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.644+0000 7f9496c0e640 1 --2- 172.21.15.71:0/3154464860 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f94780a1d80 0x7f94780a4240 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:31.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.644+0000 7f9496c0e640 1 --2- 172.21.15.71:0/3154464860 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9490103190 0x7f949019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:31.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.644+0000 7f9496c0e640 1 --2- 172.21.15.71:0/3154464860 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f94901027d0 0x7f9490197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:31.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.644+0000 7f9496c0e640 1 --2- 172.21.15.71:0/3154464860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94901015e0 0x7f94901978c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:31.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.644+0000 7f9496c0e640 1 -- 172.21.15.71:0/3154464860 >> 172.21.15.71:0/3154464860 conn(0x7f94900fcb70 msgr2=0x7f9490105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:31.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.644+0000 7f9496c0e640 1 -- 172.21.15.71:0/3154464860 shutdown_connections 2024-03-04T08:56:31.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.644+0000 7f9496c0e640 1 -- 172.21.15.71:0/3154464860 wait complete. 2024-03-04T08:56:31.651 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-04T08:56:31.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.765+0000 7fee60da0640 1 -- 172.21.15.71:0/4288830607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee5c0ffe60 msgr2=0x7fee5c100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:31.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.765+0000 7fee60da0640 1 --2- 172.21.15.71:0/4288830607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee5c0ffe60 0x7fee5c100280 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009900 tx=0x7fee4c02bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:31.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.765+0000 7fee60da0640 1 -- 172.21.15.71:0/4288830607 shutdown_connections 2024-03-04T08:56:31.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.765+0000 7fee60da0640 1 --2- 172.21.15.71:0/4288830607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee5c101160 0x7fee5c1015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:31.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.765+0000 7fee60da0640 1 --2- 172.21.15.71:0/4288830607 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fee5c1007c0 0x7fee5c100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:31.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.765+0000 7fee60da0640 1 --2- 172.21.15.71:0/4288830607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee5c0ffe60 0x7fee5c100280 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:31.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.765+0000 7fee60da0640 1 -- 172.21.15.71:0/4288830607 >> 172.21.15.71:0/4288830607 conn(0x7fee5c0fba60 msgr2=0x7fee5c0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:31.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.765+0000 7fee60da0640 1 -- 172.21.15.71:0/4288830607 shutdown_connections 2024-03-04T08:56:31.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.765+0000 7fee60da0640 1 -- 172.21.15.71:0/4288830607 wait complete. 2024-03-04T08:56:31.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.766+0000 7fee60da0640 1 Processor -- start 2024-03-04T08:56:31.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.766+0000 7fee60da0640 1 -- start start 2024-03-04T08:56:31.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.766+0000 7fee60da0640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fee5c0ffe60 0x7fee5c199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:31.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.766+0000 7fee60da0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee5c1007c0 0x7fee5c199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:31.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.766+0000 7fee60da0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee5c101160 0x7fee5c1a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:31.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.766+0000 7fee60da0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c10f4b0 con 0x7fee5c101160 2024-03-04T08:56:31.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.766+0000 7fee60da0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c10f330 con 0x7fee5c1007c0 2024-03-04T08:56:31.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.766+0000 7fee60da0640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c10f630 con 0x7fee5c0ffe60 2024-03-04T08:56:31.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.766+0000 7fee5ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee5c101160 0x7fee5c1a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:31.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.766+0000 7fee5ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee5c101160 0x7fee5c1a1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47206/0 (socket says 172.21.15.71:47206) 2024-03-04T08:56:31.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.766+0000 7fee5ad9d640 1 -- 172.21.15.71:0/709735585 learned_addr learned my addr 172.21.15.71:0/709735585 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:31.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.766+0000 7fee5a59c640 1 --2- 172.21.15.71:0/709735585 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fee5c0ffe60 0x7fee5c199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:31.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.766+0000 7fee5ad9d640 1 -- 172.21.15.71:0/709735585 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fee5c0ffe60 msgr2=0x7fee5c199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:31.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.766+0000 7fee5ad9d640 1 --2- 172.21.15.71:0/709735585 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fee5c0ffe60 0x7fee5c199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:31.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.766+0000 7fee53fff640 1 --2- 172.21.15.71:0/709735585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee5c1007c0 0x7fee5c199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:31.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.766+0000 7fee5ad9d640 1 -- 172.21.15.71:0/709735585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee5c1007c0 msgr2=0x7fee5c199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:31.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.766+0000 7fee5ad9d640 1 --2- 172.21.15.71:0/709735585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee5c1007c0 0x7fee5c199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:31.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.766+0000 7fee5ad9d640 1 -- 172.21.15.71:0/709735585 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5c1a16c0 con 0x7fee5c101160 2024-03-04T08:56:31.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.766+0000 7fee5ad9d640 1 --2- 172.21.15.71:0/709735585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee5c101160 0x7fee5c1a1050 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fee5400cce0 tx=0x7fee5400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:31.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.767+0000 7fee537fe640 1 -- 172.21.15.71:0/709735585 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5400f450 con 0x7fee5c101160 2024-03-04T08:56:31.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.767+0000 7fee537fe640 1 -- 172.21.15.71:0/709735585 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee5400fad0 con 0x7fee5c101160 2024-03-04T08:56:31.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.767+0000 7fee537fe640 1 -- 172.21.15.71:0/709735585 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee54018c10 con 0x7fee5c101160 2024-03-04T08:56:31.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.767+0000 7fee60da0640 1 -- 172.21.15.71:0/709735585 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5c1a19b0 con 0x7fee5c101160 2024-03-04T08:56:31.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.767+0000 7fee60da0640 1 -- 172.21.15.71:0/709735585 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee5c1a1eb0 con 0x7fee5c101160 2024-03-04T08:56:31.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.768+0000 7fee60da0640 1 -- 172.21.15.71:0/709735585 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee5c10acd0 con 0x7fee5c101160 2024-03-04T08:56:31.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.769+0000 7fee537fe640 1 -- 172.21.15.71:0/709735585 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fee5400f5f0 con 0x7fee5c101160 2024-03-04T08:56:31.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.769+0000 7fee537fe640 1 --2- 172.21.15.71:0/709735585 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fee480a1d80 0x7fee480a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:31.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.769+0000 7fee5a59c640 1 --2- 172.21.15.71:0/709735585 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fee480a1d80 0x7fee480a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:31.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.769+0000 7fee537fe640 1 -- 172.21.15.71:0/709735585 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fee540ce5f0 con 0x7fee5c101160 2024-03-04T08:56:31.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.770+0000 7fee5a59c640 1 --2- 172.21.15.71:0/709735585 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fee480a1d80 0x7fee480a4240 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fee4c036040 tx=0x7fee4c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:31.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.773+0000 7fee537fe640 1 -- 172.21.15.71:0/709735585 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fee54014030 con 0x7fee5c101160 2024-03-04T08:56:31.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:31.950+0000 7fee60da0640 1 -- 172.21.15.71:0/709735585 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fee5c02cff0 con 0x7fee5c101160 2024-03-04T08:56:32.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:31 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/218270231' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-04T08:56:32.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:31 smithi150 ceph-mon[36807]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-04T08:56:32.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:31 smithi150 ceph-mon[36807]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-04T08:56:32.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:31 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3154464860' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T08:56:32.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:31 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3154464860' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-04T08:56:32.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:31 smithi150 ceph-mon[36807]: osdmap e87: 12 total, 12 up, 12 in 2024-03-04T08:56:32.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:31 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/709735585' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-04T08:56:32.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:31 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/218270231' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-04T08:56:32.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:31 smithi158 ceph-mon[37241]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-04T08:56:32.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:31 smithi158 ceph-mon[37241]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-04T08:56:32.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:31 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3154464860' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T08:56:32.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:31 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3154464860' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-04T08:56:32.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:31 smithi158 ceph-mon[37241]: osdmap e87: 12 total, 12 up, 12 in 2024-03-04T08:56:32.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:31 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/709735585' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-04T08:56:32.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:31 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/218270231' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-04T08:56:32.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:31 smithi071 ceph-mon[32789]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-04T08:56:32.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:31 smithi071 ceph-mon[32789]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-04T08:56:32.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:31 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3154464860' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T08:56:32.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:31 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3154464860' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-04T08:56:32.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:31 smithi071 ceph-mon[32789]: osdmap e87: 12 total, 12 up, 12 in 2024-03-04T08:56:32.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:31 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/709735585' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-04T08:56:33.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:33 smithi150 ceph-mon[36807]: mds.? [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] up:active 2024-03-04T08:56:33.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:33 smithi150 ceph-mon[36807]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T08:56:33.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:33 smithi150 ceph-mon[36807]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T08:56:33.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:33 smithi150 ceph-mon[36807]: pgmap v323: 97 pgs: 39 active+clean, 11 creating+peering, 47 unknown; 578 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 825 B/s wr, 1 op/s 2024-03-04T08:56:33.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:33 smithi150 ceph-mon[36807]: osdmap e88: 12 total, 12 up, 12 in 2024-03-04T08:56:33.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:33 smithi158 ceph-mon[37241]: mds.? [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] up:active 2024-03-04T08:56:33.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:33 smithi158 ceph-mon[37241]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T08:56:33.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:33 smithi158 ceph-mon[37241]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T08:56:33.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:33 smithi158 ceph-mon[37241]: pgmap v323: 97 pgs: 39 active+clean, 11 creating+peering, 47 unknown; 578 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 825 B/s wr, 1 op/s 2024-03-04T08:56:33.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:33 smithi158 ceph-mon[37241]: osdmap e88: 12 total, 12 up, 12 in 2024-03-04T08:56:33.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:33 smithi071 ceph-mon[32789]: mds.? [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] up:active 2024-03-04T08:56:33.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:33 smithi071 ceph-mon[32789]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T08:56:33.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:33 smithi071 ceph-mon[32789]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T08:56:33.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:33 smithi071 ceph-mon[32789]: pgmap v323: 97 pgs: 39 active+clean, 11 creating+peering, 47 unknown; 578 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 825 B/s wr, 1 op/s 2024-03-04T08:56:33.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:33 smithi071 ceph-mon[32789]: osdmap e88: 12 total, 12 up, 12 in 2024-03-04T08:56:33.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.651+0000 7fee537fe640 1 -- 172.21.15.71:0/709735585 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v89) v1 ==== 228+0+0 (secure 0 0 0) 0x7fee5409ba80 con 0x7fee5c101160 2024-03-04T08:56:33.747 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data_ec' created 2024-03-04T08:56:33.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.653+0000 7fee60da0640 1 -- 172.21.15.71:0/709735585 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fee480a1d80 msgr2=0x7fee480a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:33.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.653+0000 7fee60da0640 1 --2- 172.21.15.71:0/709735585 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fee480a1d80 0x7fee480a4240 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fee4c036040 tx=0x7fee4c005fd0 comp rx=0 tx=0).stop 2024-03-04T08:56:33.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.653+0000 7fee60da0640 1 -- 172.21.15.71:0/709735585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee5c101160 msgr2=0x7fee5c1a1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:33.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.653+0000 7fee60da0640 1 --2- 172.21.15.71:0/709735585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee5c101160 0x7fee5c1a1050 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fee5400cce0 tx=0x7fee5400a620 comp rx=0 tx=0).stop 2024-03-04T08:56:33.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.653+0000 7fee60da0640 1 -- 172.21.15.71:0/709735585 shutdown_connections 2024-03-04T08:56:33.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.653+0000 7fee60da0640 1 --2- 172.21.15.71:0/709735585 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fee480a1d80 0x7fee480a4240 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:33.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.653+0000 7fee60da0640 1 --2- 172.21.15.71:0/709735585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee5c101160 0x7fee5c1a1050 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:33.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.653+0000 7fee60da0640 1 --2- 172.21.15.71:0/709735585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee5c1007c0 0x7fee5c199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:33.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.653+0000 7fee60da0640 1 --2- 172.21.15.71:0/709735585 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fee5c0ffe60 0x7fee5c199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:33.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.653+0000 7fee60da0640 1 -- 172.21.15.71:0/709735585 >> 172.21.15.71:0/709735585 conn(0x7fee5c0fba60 msgr2=0x7fee5c0fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:33.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.654+0000 7fee60da0640 1 -- 172.21.15.71:0/709735585 shutdown_connections 2024-03-04T08:56:33.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.654+0000 7fee60da0640 1 -- 172.21.15.71:0/709735585 wait complete. 2024-03-04T08:56:33.749 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-04T08:56:33.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.864+0000 7ff911b8c640 1 -- 172.21.15.71:0/693395008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff90c107320 msgr2=0x7ff90c109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:33.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.864+0000 7ff911b8c640 1 --2- 172.21.15.71:0/693395008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff90c107320 0x7ff90c109710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff904009ca0 tx=0x7ff90402c0f0 comp rx=0 tx=0).stop 2024-03-04T08:56:33.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.864+0000 7ff911b8c640 1 -- 172.21.15.71:0/693395008 shutdown_connections 2024-03-04T08:56:33.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.864+0000 7ff911b8c640 1 --2- 172.21.15.71:0/693395008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff90c107320 0x7ff90c109710 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:33.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.864+0000 7ff911b8c640 1 --2- 172.21.15.71:0/693395008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff90c1049f0 0x7ff90c106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:33.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.864+0000 7ff911b8c640 1 --2- 172.21.15.71:0/693395008 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff90c1020c0 0x7ff90c1044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:33.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.864+0000 7ff911b8c640 1 -- 172.21.15.71:0/693395008 >> 172.21.15.71:0/693395008 conn(0x7ff90c0fba50 msgr2=0x7ff90c0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:33.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.864+0000 7ff911b8c640 1 -- 172.21.15.71:0/693395008 shutdown_connections 2024-03-04T08:56:33.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.864+0000 7ff911b8c640 1 -- 172.21.15.71:0/693395008 wait complete. 2024-03-04T08:56:33.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.864+0000 7ff911b8c640 1 Processor -- start 2024-03-04T08:56:33.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.865+0000 7ff911b8c640 1 -- start start 2024-03-04T08:56:33.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.865+0000 7ff911b8c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff90c1020c0 0x7ff90c197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:33.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.865+0000 7ff911b8c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff90c1049f0 0x7ff90c197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:33.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.865+0000 7ff911b8c640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff90c107320 0x7ff90c19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:33.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.865+0000 7ff911b8c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90c10d250 con 0x7ff90c1020c0 2024-03-04T08:56:33.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.865+0000 7ff911b8c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90c10d0d0 con 0x7ff90c1049f0 2024-03-04T08:56:33.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.865+0000 7ff911b8c640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90c10d3d0 con 0x7ff90c107320 2024-03-04T08:56:33.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.865+0000 7ff90b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff90c1020c0 0x7ff90c197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:33.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.865+0000 7ff90b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff90c1020c0 0x7ff90c197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47224/0 (socket says 172.21.15.71:47224) 2024-03-04T08:56:33.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.865+0000 7ff90b7fe640 1 -- 172.21.15.71:0/1873066219 learned_addr learned my addr 172.21.15.71:0/1873066219 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:33.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.865+0000 7ff90affd640 1 --2- 172.21.15.71:0/1873066219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff90c1049f0 0x7ff90c197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:33.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.865+0000 7ff90bfff640 1 --2- 172.21.15.71:0/1873066219 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff90c107320 0x7ff90c19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:33.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.865+0000 7ff90b7fe640 1 -- 172.21.15.71:0/1873066219 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff90c107320 msgr2=0x7ff90c19ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:33.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.865+0000 7ff90b7fe640 1 --2- 172.21.15.71:0/1873066219 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff90c107320 0x7ff90c19ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:33.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.865+0000 7ff90b7fe640 1 -- 172.21.15.71:0/1873066219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff90c1049f0 msgr2=0x7ff90c197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:33.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.865+0000 7ff90b7fe640 1 --2- 172.21.15.71:0/1873066219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff90c1049f0 0x7ff90c197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:33.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.865+0000 7ff90b7fe640 1 -- 172.21.15.71:0/1873066219 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90c19f490 con 0x7ff90c1020c0 2024-03-04T08:56:33.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.865+0000 7ff90affd640 1 --2- 172.21.15.71:0/1873066219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff90c1049f0 0x7ff90c197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:56:33.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.865+0000 7ff90b7fe640 1 --2- 172.21.15.71:0/1873066219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff90c1020c0 0x7ff90c197870 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00d950 tx=0x7ff8fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:33.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.865+0000 7ff908ff9640 1 -- 172.21.15.71:0/1873066219 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc017070 con 0x7ff90c1020c0 2024-03-04T08:56:33.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.865+0000 7ff911b8c640 1 -- 172.21.15.71:0/1873066219 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff90c19f780 con 0x7ff90c1020c0 2024-03-04T08:56:33.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.865+0000 7ff908ff9640 1 -- 172.21.15.71:0/1873066219 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff8fc009c30 con 0x7ff90c1020c0 2024-03-04T08:56:33.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.866+0000 7ff911b8c640 1 -- 172.21.15.71:0/1873066219 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff90c19fd20 con 0x7ff90c1020c0 2024-03-04T08:56:33.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.866+0000 7ff908ff9640 1 -- 172.21.15.71:0/1873066219 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc020060 con 0x7ff90c1020c0 2024-03-04T08:56:33.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.867+0000 7ff911b8c640 1 -- 172.21.15.71:0/1873066219 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff90c0fd540 con 0x7ff90c1020c0 2024-03-04T08:56:33.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.867+0000 7ff908ff9640 1 -- 172.21.15.71:0/1873066219 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7ff8fc0109d0 con 0x7ff90c1020c0 2024-03-04T08:56:33.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.868+0000 7ff908ff9640 1 --2- 172.21.15.71:0/1873066219 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff8f40a1d80 0x7ff8f40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:33.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.868+0000 7ff90affd640 1 --2- 172.21.15.71:0/1873066219 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff8f40a1d80 0x7ff8f40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:33.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.868+0000 7ff908ff9640 1 -- 172.21.15.71:0/1873066219 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8770+0+0 (secure 0 0 0) 0x7ff8fc0cdfe0 con 0x7ff90c1020c0 2024-03-04T08:56:33.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.869+0000 7ff90affd640 1 --2- 172.21.15.71:0/1873066219 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff8f40a1d80 0x7ff8f40a4240 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff90c198d80 tx=0x7ff900005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:33.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:33.873+0000 7ff908ff9640 1 -- 172.21.15.71:0/1873066219 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff8fc009de0 con 0x7ff90c1020c0 2024-03-04T08:56:34.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.048+0000 7ff911b8c640 1 -- 172.21.15.71:0/1873066219 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7ff90c02cff0 con 0x7ff90c1020c0 2024-03-04T08:56:34.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.653+0000 7ff908ff9640 1 -- 172.21.15.71:0/1873066219 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v90) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff8fc09b1f0 con 0x7ff90c1020c0 2024-03-04T08:56:34.649 INFO:teuthology.orchestra.run.smithi071.stderr:set pool 4 allow_ec_overwrites to true 2024-03-04T08:56:34.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.655+0000 7ff911b8c640 1 -- 172.21.15.71:0/1873066219 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff8f40a1d80 msgr2=0x7ff8f40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:34.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.655+0000 7ff911b8c640 1 --2- 172.21.15.71:0/1873066219 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff8f40a1d80 0x7ff8f40a4240 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff90c198d80 tx=0x7ff900005f90 comp rx=0 tx=0).stop 2024-03-04T08:56:34.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.655+0000 7ff911b8c640 1 -- 172.21.15.71:0/1873066219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff90c1020c0 msgr2=0x7ff90c197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:34.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.655+0000 7ff911b8c640 1 --2- 172.21.15.71:0/1873066219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff90c1020c0 0x7ff90c197870 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00d950 tx=0x7ff8fc00de10 comp rx=0 tx=0).stop 2024-03-04T08:56:34.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.655+0000 7ff911b8c640 1 -- 172.21.15.71:0/1873066219 shutdown_connections 2024-03-04T08:56:34.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.655+0000 7ff911b8c640 1 --2- 172.21.15.71:0/1873066219 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff8f40a1d80 0x7ff8f40a4240 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:34.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.655+0000 7ff911b8c640 1 --2- 172.21.15.71:0/1873066219 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff90c107320 0x7ff90c19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:34.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.655+0000 7ff911b8c640 1 --2- 172.21.15.71:0/1873066219 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff90c1049f0 0x7ff90c197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:34.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.655+0000 7ff911b8c640 1 --2- 172.21.15.71:0/1873066219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff90c1020c0 0x7ff90c197870 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:34.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.655+0000 7ff911b8c640 1 -- 172.21.15.71:0/1873066219 >> 172.21.15.71:0/1873066219 conn(0x7ff90c0fba50 msgr2=0x7ff90c1029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:34.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.655+0000 7ff911b8c640 1 -- 172.21.15.71:0/1873066219 shutdown_connections 2024-03-04T08:56:34.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.655+0000 7ff911b8c640 1 -- 172.21.15.71:0/1873066219 wait complete. 2024-03-04T08:56:34.670 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-04T08:56:34.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.780+0000 7f866ac39640 1 -- 172.21.15.71:0/2725886896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8664102090 msgr2=0x7f8664108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:34.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.780+0000 7f866ac39640 1 --2- 172.21.15.71:0/2725886896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8664102090 0x7f8664108920 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f865c009ca0 tx=0x7f865c02c0f0 comp rx=0 tx=0).stop 2024-03-04T08:56:34.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.780+0000 7f866ac39640 1 -- 172.21.15.71:0/2725886896 shutdown_connections 2024-03-04T08:56:34.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.780+0000 7f866ac39640 1 --2- 172.21.15.71:0/2725886896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8664102090 0x7f8664108920 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:34.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.780+0000 7f866ac39640 1 --2- 172.21.15.71:0/2725886896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86641016d0 0x7f8664101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:34.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.781+0000 7f866ac39640 1 --2- 172.21.15.71:0/2725886896 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86641004e0 0x7f8664100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:34.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.781+0000 7f866ac39640 1 -- 172.21.15.71:0/2725886896 >> 172.21.15.71:0/2725886896 conn(0x7f86640fba70 msgr2=0x7f86640fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:34.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.781+0000 7f866ac39640 1 -- 172.21.15.71:0/2725886896 shutdown_connections 2024-03-04T08:56:34.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.781+0000 7f866ac39640 1 -- 172.21.15.71:0/2725886896 wait complete. 2024-03-04T08:56:34.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.781+0000 7f866ac39640 1 Processor -- start 2024-03-04T08:56:34.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.781+0000 7f866ac39640 1 -- start start 2024-03-04T08:56:34.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.781+0000 7f866ac39640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86641004e0 0x7f86641967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:34.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.781+0000 7f866ac39640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86641016d0 0x7f8664196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:34.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.781+0000 7f866ac39640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8664102090 0x7f866419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:34.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.781+0000 7f866ac39640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866410c170 con 0x7f8664102090 2024-03-04T08:56:34.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.781+0000 7f866ac39640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866410bff0 con 0x7f86641016d0 2024-03-04T08:56:34.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.781+0000 7f866ac39640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866410c2f0 con 0x7f86641004e0 2024-03-04T08:56:34.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.781+0000 7f86691d6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8664102090 0x7f866419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:34.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.781+0000 7f86691d6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8664102090 0x7f866419dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47248/0 (socket says 172.21.15.71:47248) 2024-03-04T08:56:34.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.781+0000 7f86691d6640 1 -- 172.21.15.71:0/589390750 learned_addr learned my addr 172.21.15.71:0/589390750 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:34.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.781+0000 7f8663fff640 1 --2- 172.21.15.71:0/589390750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86641016d0 0x7f8664196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:34.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.782+0000 7f86689d5640 1 --2- 172.21.15.71:0/589390750 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86641004e0 0x7f86641967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:34.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.782+0000 7f86691d6640 1 -- 172.21.15.71:0/589390750 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86641004e0 msgr2=0x7f86641967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:34.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.782+0000 7f86691d6640 1 --2- 172.21.15.71:0/589390750 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86641004e0 0x7f86641967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:34.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.782+0000 7f86691d6640 1 -- 172.21.15.71:0/589390750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86641016d0 msgr2=0x7f8664196d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:34.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.782+0000 7f86691d6640 1 --2- 172.21.15.71:0/589390750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86641016d0 0x7f8664196d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:34.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.782+0000 7f86691d6640 1 -- 172.21.15.71:0/589390750 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866419e410 con 0x7f8664102090 2024-03-04T08:56:34.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.782+0000 7f86689d5640 1 --2- 172.21.15.71:0/589390750 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86641004e0 0x7f86641967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:56:34.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.782+0000 7f86691d6640 1 --2- 172.21.15.71:0/589390750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8664102090 0x7f866419dda0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f865c037040 tx=0x7f865c032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:34.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.782+0000 7f8661ffb640 1 -- 172.21.15.71:0/589390750 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c041020 con 0x7f8664102090 2024-03-04T08:56:34.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.782+0000 7f8661ffb640 1 -- 172.21.15.71:0/589390750 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f865c034d90 con 0x7f8664102090 2024-03-04T08:56:34.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.782+0000 7f866ac39640 1 -- 172.21.15.71:0/589390750 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866419e6a0 con 0x7f8664102090 2024-03-04T08:56:34.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.782+0000 7f8661ffb640 1 -- 172.21.15.71:0/589390750 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c039040 con 0x7f8664102090 2024-03-04T08:56:34.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.782+0000 7f866ac39640 1 -- 172.21.15.71:0/589390750 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866419eb90 con 0x7f8664102090 2024-03-04T08:56:34.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.783+0000 7f866ac39640 1 -- 172.21.15.71:0/589390750 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8664107800 con 0x7f8664102090 2024-03-04T08:56:34.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.784+0000 7f8661ffb640 1 -- 172.21.15.71:0/589390750 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f865c044020 con 0x7f8664102090 2024-03-04T08:56:34.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.785+0000 7f8661ffb640 1 --2- 172.21.15.71:0/589390750 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f86540a1d80 0x7f86540a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:34.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.785+0000 7f86689d5640 1 --2- 172.21.15.71:0/589390750 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f86540a1d80 0x7f86540a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:34.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.785+0000 7f8661ffb640 1 -- 172.21.15.71:0/589390750 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8770+0+0 (secure 0 0 0) 0x7f865c0eff60 con 0x7f8664102090 2024-03-04T08:56:34.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.785+0000 7f86689d5640 1 --2- 172.21.15.71:0/589390750 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f86540a1d80 0x7f86540a4240 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f86580058c0 tx=0x7f8658005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:34.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:34.789+0000 7f8661ffb640 1 -- 172.21.15.71:0/589390750 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f865c04b020 con 0x7f8664102090 2024-03-04T08:56:34.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:34 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/709735585' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-04T08:56:34.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:34 smithi071 ceph-mon[32789]: osdmap e89: 12 total, 12 up, 12 in 2024-03-04T08:56:34.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:34 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1873066219' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-04T08:56:35.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.006+0000 7f866ac39640 1 -- 172.21.15.71:0/589390750 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f86640621d0 con 0x7f8664102090 2024-03-04T08:56:35.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:34 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/709735585' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-04T08:56:35.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:34 smithi150 ceph-mon[36807]: osdmap e89: 12 total, 12 up, 12 in 2024-03-04T08:56:35.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:34 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1873066219' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-04T08:56:35.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:34 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/709735585' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-04T08:56:35.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:34 smithi158 ceph-mon[37241]: osdmap e89: 12 total, 12 up, 12 in 2024-03-04T08:56:35.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:34 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1873066219' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-04T08:56:35.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.673+0000 7f8661ffb640 1 -- 172.21.15.71:0/589390750 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f865c0bd170 con 0x7f8664102090 2024-03-04T08:56:35.669 INFO:teuthology.orchestra.run.smithi071.stderr:added data pool 4 to fsmap 2024-03-04T08:56:35.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.676+0000 7f866ac39640 1 -- 172.21.15.71:0/589390750 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f86540a1d80 msgr2=0x7f86540a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:35.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.676+0000 7f866ac39640 1 --2- 172.21.15.71:0/589390750 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f86540a1d80 0x7f86540a4240 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f86580058c0 tx=0x7f8658005d50 comp rx=0 tx=0).stop 2024-03-04T08:56:35.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.676+0000 7f866ac39640 1 -- 172.21.15.71:0/589390750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8664102090 msgr2=0x7f866419dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:35.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.676+0000 7f866ac39640 1 --2- 172.21.15.71:0/589390750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8664102090 0x7f866419dda0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f865c037040 tx=0x7f865c032f20 comp rx=0 tx=0).stop 2024-03-04T08:56:35.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.676+0000 7f866ac39640 1 -- 172.21.15.71:0/589390750 shutdown_connections 2024-03-04T08:56:35.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.676+0000 7f866ac39640 1 --2- 172.21.15.71:0/589390750 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f86540a1d80 0x7f86540a4240 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:35.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.676+0000 7f866ac39640 1 --2- 172.21.15.71:0/589390750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8664102090 0x7f866419dda0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:35.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.676+0000 7f866ac39640 1 --2- 172.21.15.71:0/589390750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86641016d0 0x7f8664196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:35.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.676+0000 7f866ac39640 1 --2- 172.21.15.71:0/589390750 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86641004e0 0x7f86641967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:35.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.676+0000 7f866ac39640 1 -- 172.21.15.71:0/589390750 >> 172.21.15.71:0/589390750 conn(0x7f86640fba70 msgr2=0x7f86641049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:35.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.676+0000 7f866ac39640 1 -- 172.21.15.71:0/589390750 shutdown_connections 2024-03-04T08:56:35.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.676+0000 7f866ac39640 1 -- 172.21.15.71:0/589390750 wait complete. 2024-03-04T08:56:35.686 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:56:35.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.797+0000 7fef2a09d640 1 -- 172.21.15.71:0/3711628672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef241031a0 msgr2=0x7fef24109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:35.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.797+0000 7fef2a09d640 1 --2- 172.21.15.71:0/3711628672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef241031a0 0x7fef24109a30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009ca0 tx=0x7fef1c02c0f0 comp rx=0 tx=0).stop 2024-03-04T08:56:35.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.797+0000 7fef2a09d640 1 -- 172.21.15.71:0/3711628672 shutdown_connections 2024-03-04T08:56:35.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.797+0000 7fef2a09d640 1 --2- 172.21.15.71:0/3711628672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef241031a0 0x7fef24109a30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:35.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.797+0000 7fef2a09d640 1 --2- 172.21.15.71:0/3711628672 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef241027e0 0x7fef24102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:35.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.797+0000 7fef2a09d640 1 --2- 172.21.15.71:0/3711628672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef241015f0 0x7fef24101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:35.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.797+0000 7fef2a09d640 1 -- 172.21.15.71:0/3711628672 >> 172.21.15.71:0/3711628672 conn(0x7fef240fcb80 msgr2=0x7fef240fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:35.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.797+0000 7fef2a09d640 1 -- 172.21.15.71:0/3711628672 shutdown_connections 2024-03-04T08:56:35.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.797+0000 7fef2a09d640 1 -- 172.21.15.71:0/3711628672 wait complete. 2024-03-04T08:56:35.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.798+0000 7fef2a09d640 1 Processor -- start 2024-03-04T08:56:35.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.798+0000 7fef2a09d640 1 -- start start 2024-03-04T08:56:35.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.798+0000 7fef2a09d640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef241015f0 0x7fef24075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:35.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.798+0000 7fef2a09d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef241027e0 0x7fef240739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:35.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.798+0000 7fef2a09d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef241031a0 0x7fef24073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:35.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.798+0000 7fef2a09d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2410d250 con 0x7fef241031a0 2024-03-04T08:56:35.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.798+0000 7fef2a09d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2410d0d0 con 0x7fef241027e0 2024-03-04T08:56:35.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.798+0000 7fef2a09d640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2410d3d0 con 0x7fef241015f0 2024-03-04T08:56:35.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.798+0000 7fef23fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef241031a0 0x7fef24073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:35.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.798+0000 7fef23fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef241031a0 0x7fef24073f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56494/0 (socket says 172.21.15.71:56494) 2024-03-04T08:56:35.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.798+0000 7fef23fff640 1 -- 172.21.15.71:0/2914054926 learned_addr learned my addr 172.21.15.71:0/2914054926 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:35.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.798+0000 7fef22ffd640 1 --2- 172.21.15.71:0/2914054926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef241027e0 0x7fef240739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:35.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.798+0000 7fef237fe640 1 --2- 172.21.15.71:0/2914054926 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef241015f0 0x7fef24075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:35.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.798+0000 7fef23fff640 1 -- 172.21.15.71:0/2914054926 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef241015f0 msgr2=0x7fef24075380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:35.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.798+0000 7fef23fff640 1 --2- 172.21.15.71:0/2914054926 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef241015f0 0x7fef24075380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:35.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.798+0000 7fef23fff640 1 -- 172.21.15.71:0/2914054926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef241027e0 msgr2=0x7fef240739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:35.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.798+0000 7fef23fff640 1 --2- 172.21.15.71:0/2914054926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef241027e0 0x7fef240739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:35.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.798+0000 7fef23fff640 1 -- 172.21.15.71:0/2914054926 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef24074790 con 0x7fef241031a0 2024-03-04T08:56:35.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.798+0000 7fef237fe640 1 --2- 172.21.15.71:0/2914054926 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef241015f0 0x7fef24075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:56:35.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.798+0000 7fef23fff640 1 --2- 172.21.15.71:0/2914054926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef241031a0 0x7fef24073f00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fef1c037040 tx=0x7fef1c032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:35.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.799+0000 7fef20ff9640 1 -- 172.21.15.71:0/2914054926 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c041020 con 0x7fef241031a0 2024-03-04T08:56:35.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.799+0000 7fef20ff9640 1 -- 172.21.15.71:0/2914054926 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef1c034cd0 con 0x7fef241031a0 2024-03-04T08:56:35.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.799+0000 7fef2a09d640 1 -- 172.21.15.71:0/2914054926 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef241a18c0 con 0x7fef241031a0 2024-03-04T08:56:35.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.799+0000 7fef20ff9640 1 -- 172.21.15.71:0/2914054926 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c039040 con 0x7fef241031a0 2024-03-04T08:56:35.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.799+0000 7fef2a09d640 1 -- 172.21.15.71:0/2914054926 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef241a1e10 con 0x7fef241031a0 2024-03-04T08:56:35.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.800+0000 7fef2a09d640 1 -- 172.21.15.71:0/2914054926 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef241088b0 con 0x7fef241031a0 2024-03-04T08:56:35.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.801+0000 7fef20ff9640 1 -- 172.21.15.71:0/2914054926 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fef1c044020 con 0x7fef241031a0 2024-03-04T08:56:35.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.801+0000 7fef20ff9640 1 --2- 172.21.15.71:0/2914054926 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fef0c0a1d80 0x7fef0c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:35.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.802+0000 7fef237fe640 1 --2- 172.21.15.71:0/2914054926 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fef0c0a1d80 0x7fef0c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:35.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.802+0000 7fef20ff9640 1 -- 172.21.15.71:0/2914054926 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fef1c0eff50 con 0x7fef241031a0 2024-03-04T08:56:35.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.802+0000 7fef237fe640 1 --2- 172.21.15.71:0/2914054926 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fef0c0a1d80 0x7fef0c0a4240 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fef24102640 tx=0x7fef14005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:35.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:35.806+0000 7fef20ff9640 1 -- 172.21.15.71:0/2914054926 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fef1c04b020 con 0x7fef241031a0 2024-03-04T08:56:35.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:35 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1873066219' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-04T08:56:35.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:35 smithi071 ceph-mon[32789]: osdmap e90: 12 total, 12 up, 12 in 2024-03-04T08:56:35.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:35 smithi071 ceph-mon[32789]: pgmap v326: 129 pgs: 89 active+clean, 13 creating+peering, 27 unknown; 578 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-03-04T08:56:35.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:35 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/589390750' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-04T08:56:36.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.019+0000 7fef2a09d640 1 -- 172.21.15.71:0/2914054926 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef2402cff0 con 0x7fef241031a0 2024-03-04T08:56:36.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.020+0000 7fef20ff9640 1 -- 172.21.15.71:0/2914054926 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11330 (secure 0 0 0) 0x7fef1c0dc070 con 0x7fef241031a0 2024-03-04T08:56:36.016 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:36.018 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3975379807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3975379807},{"type":"v1","addr":"172.21.15.71:6835","nonce":3975379807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1962929347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1962929347},{"type":"v1","addr":"172.21.15.71:6837","nonce":1962929347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/1459252176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1459252176},{"type":"v1","addr":"172.21.15.71:6839","nonce":1459252176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6841/1083546600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":1083546600},{"type":"v1","addr":"172.21.15.71:6841","nonce":1083546600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/178179528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":178179528},{"type":"v1","addr":"172.21.15.158:6833","nonce":178179528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2468877675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2468877675},{"type":"v1","addr":"172.21.15.150:6833","nonce":2468877675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1652094309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1652094309},{"type":"v1","addr":"172.21.15.158:6835","nonce":1652094309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1802416462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1802416462},{"type":"v1","addr":"172.21.15.150:6835","nonce":1802416462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/617058380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":617058380},{"type":"v1","addr":"172.21.15.158:6837","nonce":617058380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/913465600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":913465600},{"type":"v1","addr":"172.21.15.150:6837","nonce":913465600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6839/2921704804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6838","nonce":2921704804},{"type":"v1","addr":"172.21.15.158:6839","nonce":2921704804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T08:56:30.343357+0000","modified":"2024-03-04T08:56:35.008032+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/2639278083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2639278083},{"type":"v1","addr":"172.21.15.150:6839","nonce":2639278083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24491,"qdb_cluster":[24491]},"id":1}]} 2024-03-04T08:56:36.019 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 17 2024-03-04T08:56:36.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.022+0000 7fef2a09d640 1 -- 172.21.15.71:0/2914054926 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fef0c0a1d80 msgr2=0x7fef0c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:36.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.022+0000 7fef2a09d640 1 --2- 172.21.15.71:0/2914054926 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fef0c0a1d80 0x7fef0c0a4240 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fef24102640 tx=0x7fef14005f90 comp rx=0 tx=0).stop 2024-03-04T08:56:36.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.022+0000 7fef2a09d640 1 -- 172.21.15.71:0/2914054926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef241031a0 msgr2=0x7fef24073f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:36.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.022+0000 7fef2a09d640 1 --2- 172.21.15.71:0/2914054926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef241031a0 0x7fef24073f00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fef1c037040 tx=0x7fef1c032f20 comp rx=0 tx=0).stop 2024-03-04T08:56:36.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.022+0000 7fef2a09d640 1 -- 172.21.15.71:0/2914054926 shutdown_connections 2024-03-04T08:56:36.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.022+0000 7fef2a09d640 1 --2- 172.21.15.71:0/2914054926 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fef0c0a1d80 0x7fef0c0a4240 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.022+0000 7fef2a09d640 1 --2- 172.21.15.71:0/2914054926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fef241031a0 0x7fef24073f00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.022+0000 7fef2a09d640 1 --2- 172.21.15.71:0/2914054926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef241027e0 0x7fef240739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.022+0000 7fef2a09d640 1 --2- 172.21.15.71:0/2914054926 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fef241015f0 0x7fef24075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.022+0000 7fef2a09d640 1 -- 172.21.15.71:0/2914054926 >> 172.21.15.71:0/2914054926 conn(0x7fef240fcb80 msgr2=0x7fef24105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:36.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.022+0000 7fef2a09d640 1 -- 172.21.15.71:0/2914054926 shutdown_connections 2024-03-04T08:56:36.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.022+0000 7fef2a09d640 1 -- 172.21.15.71:0/2914054926 wait complete. 2024-03-04T08:56:36.030 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T08:56:36.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:35 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1873066219' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-04T08:56:36.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:35 smithi150 ceph-mon[36807]: osdmap e90: 12 total, 12 up, 12 in 2024-03-04T08:56:36.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:35 smithi150 ceph-mon[36807]: pgmap v326: 129 pgs: 89 active+clean, 13 creating+peering, 27 unknown; 578 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-03-04T08:56:36.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:35 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/589390750' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-04T08:56:36.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:35 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1873066219' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-04T08:56:36.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:35 smithi158 ceph-mon[37241]: osdmap e90: 12 total, 12 up, 12 in 2024-03-04T08:56:36.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:35 smithi158 ceph-mon[37241]: pgmap v326: 129 pgs: 89 active+clean, 13 creating+peering, 27 unknown; 578 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-03-04T08:56:36.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:35 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/589390750' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-04T08:56:36.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.135+0000 7feafd037640 1 -- 172.21.15.71:0/4198818802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feaf8107370 msgr2=0x7feaf8109760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:36.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.135+0000 7feafd037640 1 --2- 172.21.15.71:0/4198818802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feaf8107370 0x7feaf8109760 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009ca0 tx=0x7feaf002c0f0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.136+0000 7feafd037640 1 -- 172.21.15.71:0/4198818802 shutdown_connections 2024-03-04T08:56:36.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.136+0000 7feafd037640 1 --2- 172.21.15.71:0/4198818802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feaf8107370 0x7feaf8109760 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.136+0000 7feafd037640 1 --2- 172.21.15.71:0/4198818802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feaf8104a40 0x7feaf8106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.136+0000 7feafd037640 1 --2- 172.21.15.71:0/4198818802 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feaf8102110 0x7feaf8104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.136+0000 7feafd037640 1 -- 172.21.15.71:0/4198818802 >> 172.21.15.71:0/4198818802 conn(0x7feaf80fba60 msgr2=0x7feaf80fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:36.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.136+0000 7feafd037640 1 -- 172.21.15.71:0/4198818802 shutdown_connections 2024-03-04T08:56:36.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.136+0000 7feafd037640 1 -- 172.21.15.71:0/4198818802 wait complete. 2024-03-04T08:56:36.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.136+0000 7feafd037640 1 Processor -- start 2024-03-04T08:56:36.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.136+0000 7feafd037640 1 -- start start 2024-03-04T08:56:36.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.137+0000 7feafd037640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feaf8102110 0x7feaf81978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:36.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.137+0000 7feafd037640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feaf8104a40 0x7feaf8197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:36.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.137+0000 7feafd037640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feaf8107370 0x7feaf819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:36.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.137+0000 7feafd037640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf810d280 con 0x7feaf8107370 2024-03-04T08:56:36.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.137+0000 7feafd037640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf810d100 con 0x7feaf8104a40 2024-03-04T08:56:36.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.137+0000 7feafd037640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf810d400 con 0x7feaf8102110 2024-03-04T08:56:36.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.137+0000 7feaf759e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feaf8107370 0x7feaf819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:36.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.137+0000 7feaf759e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feaf8107370 0x7feaf819ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56508/0 (socket says 172.21.15.71:56508) 2024-03-04T08:56:36.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.137+0000 7feaf659c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feaf8104a40 0x7feaf8197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:36.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.137+0000 7feaf759e640 1 -- 172.21.15.71:0/933159328 learned_addr learned my addr 172.21.15.71:0/933159328 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:36.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.137+0000 7feaf6d9d640 1 --2- 172.21.15.71:0/933159328 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feaf8102110 0x7feaf81978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:36.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.137+0000 7feaf759e640 1 -- 172.21.15.71:0/933159328 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feaf8102110 msgr2=0x7feaf81978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:36.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.137+0000 7feaf759e640 1 --2- 172.21.15.71:0/933159328 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feaf8102110 0x7feaf81978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.137+0000 7feaf759e640 1 -- 172.21.15.71:0/933159328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feaf8104a40 msgr2=0x7feaf8197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:36.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.137+0000 7feaf759e640 1 --2- 172.21.15.71:0/933159328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feaf8104a40 0x7feaf8197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.137+0000 7feaf759e640 1 -- 172.21.15.71:0/933159328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf819f4c0 con 0x7feaf8107370 2024-03-04T08:56:36.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.137+0000 7feaf659c640 1 --2- 172.21.15.71:0/933159328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feaf8104a40 0x7feaf8197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:56:36.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.137+0000 7feaf759e640 1 --2- 172.21.15.71:0/933159328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feaf8107370 0x7feaf819ee50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7feaf0037040 tx=0x7feaf0032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:36.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.137+0000 7feaeffff640 1 -- 172.21.15.71:0/933159328 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf0041020 con 0x7feaf8107370 2024-03-04T08:56:36.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.137+0000 7feafd037640 1 -- 172.21.15.71:0/933159328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf819f750 con 0x7feaf8107370 2024-03-04T08:56:36.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.137+0000 7feaeffff640 1 -- 172.21.15.71:0/933159328 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feaf0034cd0 con 0x7feaf8107370 2024-03-04T08:56:36.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.137+0000 7feaeffff640 1 -- 172.21.15.71:0/933159328 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf0039040 con 0x7feaf8107370 2024-03-04T08:56:36.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.137+0000 7feafd037640 1 -- 172.21.15.71:0/933159328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf819fc40 con 0x7feaf8107370 2024-03-04T08:56:36.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.138+0000 7feafd037640 1 -- 172.21.15.71:0/933159328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf80fd550 con 0x7feaf8107370 2024-03-04T08:56:36.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.139+0000 7feaeffff640 1 -- 172.21.15.71:0/933159328 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7feaf0044020 con 0x7feaf8107370 2024-03-04T08:56:36.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.140+0000 7feaeffff640 1 --2- 172.21.15.71:0/933159328 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7feae40a1d80 0x7feae40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:36.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.140+0000 7feaf6d9d640 1 --2- 172.21.15.71:0/933159328 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7feae40a1d80 0x7feae40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:36.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.140+0000 7feaeffff640 1 -- 172.21.15.71:0/933159328 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7feaf00efef0 con 0x7feaf8107370 2024-03-04T08:56:36.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.140+0000 7feaf6d9d640 1 --2- 172.21.15.71:0/933159328 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7feae40a1d80 0x7feae40a4240 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7feae0009b60 tx=0x7feae0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:36.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.144+0000 7feaeffff640 1 -- 172.21.15.71:0/933159328 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7feaf004b020 con 0x7feaf8107370 2024-03-04T08:56:36.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.319+0000 7feafd037640 1 -- 172.21.15.71:0/933159328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feaf80621d0 con 0x7feaf8107370 2024-03-04T08:56:36.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.321+0000 7feaeffff640 1 -- 172.21.15.71:0/933159328 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24153 (secure 0 0 0) 0x7feaf00bd170 con 0x7feaf8107370 2024-03-04T08:56:36.317 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:36.317 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":91,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","created":"2024-03-04T08:47:28.546102+0000","modified":"2024-03-04T08:56:35.007858+0000","last_up_change":"2024-03-04T08:55:30.744961+0000","last_in_change":"2024-03-04T08:55:11.570990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T08:51:14.933487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T08:56:28.253074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T08:56:29.543622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T08:56:32.884220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7e43b47d-8888-4631-bc96-8a96fcf8cd46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6803","nonce":1173641530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6805","nonce":1173641530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6809","nonce":1173641530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6807","nonce":1173641530}]},"public_addr":"172.21.15.71:6803/1173641530","cluster_addr":"172.21.15.71:6805/1173641530","heartbeat_back_addr":"172.21.15.71:6809/1173641530","heartbeat_front_addr":"172.21.15.71:6807/1173641530","state":["exists","up"]},{"osd":1,"uuid":"1209f214-f2e2-4209-bf69-4528a43a2123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6801","nonce":3383948110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6803","nonce":3383948110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6807","nonce":3383948110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6805","nonce":3383948110}]},"public_addr":"172.21.15.150:6801/3383948110","cluster_addr":"172.21.15.150:6803/3383948110","heartbeat_back_addr":"172.21.15.150:6807/3383948110","heartbeat_front_addr":"172.21.15.150:6805/3383948110","state":["exists","up"]},{"osd":2,"uuid":"b88c3082-93a9-4c9b-9221-a7105a44c91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6801","nonce":586944144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6803","nonce":586944144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6807","nonce":586944144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6805","nonce":586944144}]},"public_addr":"172.21.15.158:6801/586944144","cluster_addr":"172.21.15.158:6803/586944144","heartbeat_back_addr":"172.21.15.158:6807/586944144","heartbeat_front_addr":"172.21.15.158:6805/586944144","state":["exists","up"]},{"osd":3,"uuid":"e4fc7723-fd73-40b2-b9d0-7f9bda282ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6811","nonce":1311903262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6813","nonce":1311903262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6817","nonce":1311903262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6815","nonce":1311903262}]},"public_addr":"172.21.15.71:6811/1311903262","cluster_addr":"172.21.15.71:6813/1311903262","heartbeat_back_addr":"172.21.15.71:6817/1311903262","heartbeat_front_addr":"172.21.15.71:6815/1311903262","state":["exists","up"]},{"osd":4,"uuid":"e7eb72ac-f066-4ccd-b2bf-22a6a26daf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6809","nonce":2300837573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6811","nonce":2300837573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6815","nonce":2300837573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6813","nonce":2300837573}]},"public_addr":"172.21.15.150:6809/2300837573","cluster_addr":"172.21.15.150:6811/2300837573","heartbeat_back_addr":"172.21.15.150:6815/2300837573","heartbeat_front_addr":"172.21.15.150:6813/2300837573","state":["exists","up"]},{"osd":5,"uuid":"9b295840-e104-4a66-9cc4-7020627c1765","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6809","nonce":1922694045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6811","nonce":1922694045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6815","nonce":1922694045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6813","nonce":1922694045}]},"public_addr":"172.21.15.158:6809/1922694045","cluster_addr":"172.21.15.158:6811/1922694045","heartbeat_back_addr":"172.21.15.158:6815/1922694045","heartbeat_front_addr":"172.21.15.158:6813/1922694045","state":["exists","up"]},{"osd":6,"uuid":"6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6819","nonce":3848842707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6821","nonce":3848842707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6825","nonce":3848842707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6823","nonce":3848842707}]},"public_addr":"172.21.15.71:6819/3848842707","cluster_addr":"172.21.15.71:6821/3848842707","heartbeat_back_addr":"172.21.15.71:6825/3848842707","heartbeat_front_addr":"172.21.15.71:6823/3848842707","state":["exists","up"]},{"osd":7,"uuid":"6160a40a-d30f-4569-949c-0afb1bb3d133","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6817","nonce":1328330305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6819","nonce":1328330305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6823","nonce":1328330305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6821","nonce":1328330305}]},"public_addr":"172.21.15.150:6817/1328330305","cluster_addr":"172.21.15.150:6819/1328330305","heartbeat_back_addr":"172.21.15.150:6823/1328330305","heartbeat_front_addr":"172.21.15.150:6821/1328330305","state":["exists","up"]},{"osd":8,"uuid":"95ed2341-84a4-465c-9948-ac667680ce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6817","nonce":2195304577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6819","nonce":2195304577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6823","nonce":2195304577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6821","nonce":2195304577}]},"public_addr":"172.21.15.158:6817/2195304577","cluster_addr":"172.21.15.158:6819/2195304577","heartbeat_back_addr":"172.21.15.158:6823/2195304577","heartbeat_front_addr":"172.21.15.158:6821/2195304577","state":["exists","up"]},{"osd":9,"uuid":"088f6419-63a0-4768-afae-dbb8a5f5a565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6827","nonce":1913779347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6829","nonce":1913779347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6833","nonce":1913779347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6831","nonce":1913779347}]},"public_addr":"172.21.15.71:6827/1913779347","cluster_addr":"172.21.15.71:6829/1913779347","heartbeat_back_addr":"172.21.15.71:6833/1913779347","heartbeat_front_addr":"172.21.15.71:6831/1913779347","state":["exists","up"]},{"osd":10,"uuid":"18c85e24-068d-4f1e-a3b8-f7915b363234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6825","nonce":3295420996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6827","nonce":3295420996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6831","nonce":3295420996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6829","nonce":3295420996}]},"public_addr":"172.21.15.150:6825/3295420996","cluster_addr":"172.21.15.150:6827/3295420996","heartbeat_back_addr":"172.21.15.150:6831/3295420996","heartbeat_front_addr":"172.21.15.150:6829/3295420996","state":["exists","up"]},{"osd":11,"uuid":"4f6b3ddf-c482-4740-b1fd-de836e047c2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6825","nonce":3332083865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6827","nonce":3332083865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6831","nonce":3332083865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6829","nonce":3332083865}]},"public_addr":"172.21.15.158:6825/3332083865","cluster_addr":"172.21.15.158:6827/3332083865","heartbeat_back_addr":"172.21.15.158:6831/3332083865","heartbeat_front_addr":"172.21.15.158:6829/3332083865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:05.997832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:29.314235+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:08.819465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:41.194275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:06.032479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:38.814498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:13.067960+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:38.568479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:03.416713+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:38.349479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:04.096945+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:29.425250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1287712358":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/3770452860":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/718743094":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/1904717532":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:6800/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6800/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/2862184313":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/3621754630":"2024-03-05T08:48:22.593931+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T08:56:36.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.323+0000 7feafd037640 1 -- 172.21.15.71:0/933159328 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7feae40a1d80 msgr2=0x7feae40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:36.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.323+0000 7feafd037640 1 --2- 172.21.15.71:0/933159328 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7feae40a1d80 0x7feae40a4240 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7feae0009b60 tx=0x7feae0005f90 comp rx=0 tx=0).stop 2024-03-04T08:56:36.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.323+0000 7feafd037640 1 -- 172.21.15.71:0/933159328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feaf8107370 msgr2=0x7feaf819ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:36.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.323+0000 7feafd037640 1 --2- 172.21.15.71:0/933159328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feaf8107370 0x7feaf819ee50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7feaf0037040 tx=0x7feaf0032f40 comp rx=0 tx=0).stop 2024-03-04T08:56:36.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.323+0000 7feafd037640 1 -- 172.21.15.71:0/933159328 shutdown_connections 2024-03-04T08:56:36.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.323+0000 7feafd037640 1 --2- 172.21.15.71:0/933159328 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7feae40a1d80 0x7feae40a4240 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.323+0000 7feafd037640 1 --2- 172.21.15.71:0/933159328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feaf8107370 0x7feaf819ee50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.323+0000 7feafd037640 1 --2- 172.21.15.71:0/933159328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feaf8104a40 0x7feaf8197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.323+0000 7feafd037640 1 --2- 172.21.15.71:0/933159328 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feaf8102110 0x7feaf81978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.323+0000 7feafd037640 1 -- 172.21.15.71:0/933159328 >> 172.21.15.71:0/933159328 conn(0x7feaf80fba60 msgr2=0x7feaf81029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:36.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.323+0000 7feafd037640 1 -- 172.21.15.71:0/933159328 shutdown_connections 2024-03-04T08:56:36.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.323+0000 7feafd037640 1 -- 172.21.15.71:0/933159328 wait complete. 2024-03-04T08:56:36.333 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T08:56:36.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.438+0000 7f513840e640 1 -- 172.21.15.71:0/2707250110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51301004f0 msgr2=0x7f5130100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:36.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.438+0000 7f513840e640 1 --2- 172.21.15.71:0/2707250110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51301004f0 0x7f5130100910 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5124009900 tx=0x7f512402bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:36.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.439+0000 7f513840e640 1 -- 172.21.15.71:0/2707250110 shutdown_connections 2024-03-04T08:56:36.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.439+0000 7f513840e640 1 --2- 172.21.15.71:0/2707250110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51301020a0 0x7f5130108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.439+0000 7f513840e640 1 --2- 172.21.15.71:0/2707250110 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51301016e0 0x7f5130101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.439+0000 7f513840e640 1 --2- 172.21.15.71:0/2707250110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51301004f0 0x7f5130100910 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.439+0000 7f513840e640 1 -- 172.21.15.71:0/2707250110 >> 172.21.15.71:0/2707250110 conn(0x7f51300fba60 msgr2=0x7f51300fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:36.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.439+0000 7f513840e640 1 -- 172.21.15.71:0/2707250110 shutdown_connections 2024-03-04T08:56:36.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.439+0000 7f513840e640 1 -- 172.21.15.71:0/2707250110 wait complete. 2024-03-04T08:56:36.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.439+0000 7f513840e640 1 Processor -- start 2024-03-04T08:56:36.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.439+0000 7f513840e640 1 -- start start 2024-03-04T08:56:36.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.439+0000 7f513840e640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51301004f0 0x7f51300692a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:36.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.439+0000 7f513840e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51301016e0 0x7f513019c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:36.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.439+0000 7f513840e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51301020a0 0x7f513019eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:36.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.439+0000 7f513840e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513010c180 con 0x7f51301016e0 2024-03-04T08:56:36.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.439+0000 7f513840e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513010c000 con 0x7f51301020a0 2024-03-04T08:56:36.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.439+0000 7f513840e640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513010c300 con 0x7f51301004f0 2024-03-04T08:56:36.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.439+0000 7f51359a9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51301016e0 0x7f513019c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:36.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.439+0000 7f51359a9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51301016e0 0x7f513019c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56522/0 (socket says 172.21.15.71:56522) 2024-03-04T08:56:36.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.439+0000 7f51359a9640 1 -- 172.21.15.71:0/3402143609 learned_addr learned my addr 172.21.15.71:0/3402143609 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:36.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.439+0000 7f51369ab640 1 --2- 172.21.15.71:0/3402143609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51301020a0 0x7f513019eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:36.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.440+0000 7f51361aa640 1 --2- 172.21.15.71:0/3402143609 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51301004f0 0x7f51300692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:36.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.440+0000 7f51359a9640 1 -- 172.21.15.71:0/3402143609 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51301004f0 msgr2=0x7f51300692a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:36.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.440+0000 7f51359a9640 1 --2- 172.21.15.71:0/3402143609 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51301004f0 0x7f51300692a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.440+0000 7f51359a9640 1 -- 172.21.15.71:0/3402143609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51301020a0 msgr2=0x7f513019eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:36.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.440+0000 7f51359a9640 1 --2- 172.21.15.71:0/3402143609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51301020a0 0x7f513019eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.440+0000 7f51359a9640 1 -- 172.21.15.71:0/3402143609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513019f5b0 con 0x7f51301016e0 2024-03-04T08:56:36.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.440+0000 7f51361aa640 1 --2- 172.21.15.71:0/3402143609 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51301004f0 0x7f51300692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:56:36.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.440+0000 7f51359a9640 1 --2- 172.21.15.71:0/3402143609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51301016e0 0x7f513019c680 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f512c00c970 tx=0x7f512c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:36.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.440+0000 7f51237fe640 1 -- 172.21.15.71:0/3402143609 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c007bb0 con 0x7f51301016e0 2024-03-04T08:56:36.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.440+0000 7f51237fe640 1 -- 172.21.15.71:0/3402143609 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f512c007d50 con 0x7f51301016e0 2024-03-04T08:56:36.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.440+0000 7f513840e640 1 -- 172.21.15.71:0/3402143609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f513019f840 con 0x7f51301016e0 2024-03-04T08:56:36.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.440+0000 7f51237fe640 1 -- 172.21.15.71:0/3402143609 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c00f450 con 0x7f51301016e0 2024-03-04T08:56:36.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.440+0000 7f513840e640 1 -- 172.21.15.71:0/3402143609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f513019fd90 con 0x7f51301016e0 2024-03-04T08:56:36.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.441+0000 7f513840e640 1 -- 172.21.15.71:0/3402143609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5130107860 con 0x7f51301016e0 2024-03-04T08:56:36.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.442+0000 7f51237fe640 1 -- 172.21.15.71:0/3402143609 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f512c005020 con 0x7f51301016e0 2024-03-04T08:56:36.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.443+0000 7f51237fe640 1 --2- 172.21.15.71:0/3402143609 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f511c0a1e00 0x7f511c0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:36.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.443+0000 7f51361aa640 1 --2- 172.21.15.71:0/3402143609 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f511c0a1e00 0x7f511c0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:36.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.443+0000 7f51237fe640 1 -- 172.21.15.71:0/3402143609 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f512c0cdd30 con 0x7f51301016e0 2024-03-04T08:56:36.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.443+0000 7f51361aa640 1 --2- 172.21.15.71:0/3402143609 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f511c0a1e00 0x7f511c0a42c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5124036040 tx=0x7f5124000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:36.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.447+0000 7f51237fe640 1 -- 172.21.15.71:0/3402143609 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f512c014030 con 0x7f51301016e0 2024-03-04T08:56:36.530 INFO:journalctl@ceph.mds.b.smithi150.stdout:Mar 04 08:56:36 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-b[66660]: 2024-03-04T08:56:36.458+0000 7fded2d64700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-04T08:56:36.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.622+0000 7f513840e640 1 -- 172.21.15.71:0/3402143609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51300621d0 con 0x7f51301016e0 2024-03-04T08:56:36.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.623+0000 7f51237fe640 1 -- 172.21.15.71:0/3402143609 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24153 (secure 0 0 0) 0x7f512c09af20 con 0x7f51301016e0 2024-03-04T08:56:36.620 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:36.620 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":91,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","created":"2024-03-04T08:47:28.546102+0000","modified":"2024-03-04T08:56:35.007858+0000","last_up_change":"2024-03-04T08:55:30.744961+0000","last_in_change":"2024-03-04T08:55:11.570990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T08:51:14.933487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T08:56:28.253074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T08:56:29.543622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T08:56:32.884220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7e43b47d-8888-4631-bc96-8a96fcf8cd46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6803","nonce":1173641530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6805","nonce":1173641530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6809","nonce":1173641530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6807","nonce":1173641530}]},"public_addr":"172.21.15.71:6803/1173641530","cluster_addr":"172.21.15.71:6805/1173641530","heartbeat_back_addr":"172.21.15.71:6809/1173641530","heartbeat_front_addr":"172.21.15.71:6807/1173641530","state":["exists","up"]},{"osd":1,"uuid":"1209f214-f2e2-4209-bf69-4528a43a2123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6801","nonce":3383948110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6803","nonce":3383948110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6807","nonce":3383948110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6805","nonce":3383948110}]},"public_addr":"172.21.15.150:6801/3383948110","cluster_addr":"172.21.15.150:6803/3383948110","heartbeat_back_addr":"172.21.15.150:6807/3383948110","heartbeat_front_addr":"172.21.15.150:6805/3383948110","state":["exists","up"]},{"osd":2,"uuid":"b88c3082-93a9-4c9b-9221-a7105a44c91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6801","nonce":586944144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6803","nonce":586944144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6807","nonce":586944144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6805","nonce":586944144}]},"public_addr":"172.21.15.158:6801/586944144","cluster_addr":"172.21.15.158:6803/586944144","heartbeat_back_addr":"172.21.15.158:6807/586944144","heartbeat_front_addr":"172.21.15.158:6805/586944144","state":["exists","up"]},{"osd":3,"uuid":"e4fc7723-fd73-40b2-b9d0-7f9bda282ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6811","nonce":1311903262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6813","nonce":1311903262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6817","nonce":1311903262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6815","nonce":1311903262}]},"public_addr":"172.21.15.71:6811/1311903262","cluster_addr":"172.21.15.71:6813/1311903262","heartbeat_back_addr":"172.21.15.71:6817/1311903262","heartbeat_front_addr":"172.21.15.71:6815/1311903262","state":["exists","up"]},{"osd":4,"uuid":"e7eb72ac-f066-4ccd-b2bf-22a6a26daf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6809","nonce":2300837573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6811","nonce":2300837573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6815","nonce":2300837573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6813","nonce":2300837573}]},"public_addr":"172.21.15.150:6809/2300837573","cluster_addr":"172.21.15.150:6811/2300837573","heartbeat_back_addr":"172.21.15.150:6815/2300837573","heartbeat_front_addr":"172.21.15.150:6813/2300837573","state":["exists","up"]},{"osd":5,"uuid":"9b295840-e104-4a66-9cc4-7020627c1765","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6809","nonce":1922694045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6811","nonce":1922694045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6815","nonce":1922694045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6813","nonce":1922694045}]},"public_addr":"172.21.15.158:6809/1922694045","cluster_addr":"172.21.15.158:6811/1922694045","heartbeat_back_addr":"172.21.15.158:6815/1922694045","heartbeat_front_addr":"172.21.15.158:6813/1922694045","state":["exists","up"]},{"osd":6,"uuid":"6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6819","nonce":3848842707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6821","nonce":3848842707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6825","nonce":3848842707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6823","nonce":3848842707}]},"public_addr":"172.21.15.71:6819/3848842707","cluster_addr":"172.21.15.71:6821/3848842707","heartbeat_back_addr":"172.21.15.71:6825/3848842707","heartbeat_front_addr":"172.21.15.71:6823/3848842707","state":["exists","up"]},{"osd":7,"uuid":"6160a40a-d30f-4569-949c-0afb1bb3d133","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6817","nonce":1328330305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6819","nonce":1328330305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6823","nonce":1328330305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6821","nonce":1328330305}]},"public_addr":"172.21.15.150:6817/1328330305","cluster_addr":"172.21.15.150:6819/1328330305","heartbeat_back_addr":"172.21.15.150:6823/1328330305","heartbeat_front_addr":"172.21.15.150:6821/1328330305","state":["exists","up"]},{"osd":8,"uuid":"95ed2341-84a4-465c-9948-ac667680ce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6817","nonce":2195304577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6819","nonce":2195304577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6823","nonce":2195304577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6821","nonce":2195304577}]},"public_addr":"172.21.15.158:6817/2195304577","cluster_addr":"172.21.15.158:6819/2195304577","heartbeat_back_addr":"172.21.15.158:6823/2195304577","heartbeat_front_addr":"172.21.15.158:6821/2195304577","state":["exists","up"]},{"osd":9,"uuid":"088f6419-63a0-4768-afae-dbb8a5f5a565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6827","nonce":1913779347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6829","nonce":1913779347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6833","nonce":1913779347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6831","nonce":1913779347}]},"public_addr":"172.21.15.71:6827/1913779347","cluster_addr":"172.21.15.71:6829/1913779347","heartbeat_back_addr":"172.21.15.71:6833/1913779347","heartbeat_front_addr":"172.21.15.71:6831/1913779347","state":["exists","up"]},{"osd":10,"uuid":"18c85e24-068d-4f1e-a3b8-f7915b363234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6825","nonce":3295420996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6827","nonce":3295420996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6831","nonce":3295420996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6829","nonce":3295420996}]},"public_addr":"172.21.15.150:6825/3295420996","cluster_addr":"172.21.15.150:6827/3295420996","heartbeat_back_addr":"172.21.15.150:6831/3295420996","heartbeat_front_addr":"172.21.15.150:6829/3295420996","state":["exists","up"]},{"osd":11,"uuid":"4f6b3ddf-c482-4740-b1fd-de836e047c2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6825","nonce":3332083865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6827","nonce":3332083865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6831","nonce":3332083865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6829","nonce":3332083865}]},"public_addr":"172.21.15.158:6825/3332083865","cluster_addr":"172.21.15.158:6827/3332083865","heartbeat_back_addr":"172.21.15.158:6831/3332083865","heartbeat_front_addr":"172.21.15.158:6829/3332083865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:05.997832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:29.314235+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:08.819465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:41.194275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:06.032479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:38.814498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:13.067960+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:38.568479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:03.416713+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:38.349479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:04.096945+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:29.425250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1287712358":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/3770452860":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/718743094":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/1904717532":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:6800/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6800/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/2862184313":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/3621754630":"2024-03-05T08:48:22.593931+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T08:56:36.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.625+0000 7f513840e640 1 -- 172.21.15.71:0/3402143609 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f511c0a1e00 msgr2=0x7f511c0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.625+0000 7f513840e640 1 --2- 172.21.15.71:0/3402143609 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f511c0a1e00 0x7f511c0a42c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5124036040 tx=0x7f5124000bc0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.625+0000 7f513840e640 1 -- 172.21.15.71:0/3402143609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51301016e0 msgr2=0x7f513019c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.625+0000 7f513840e640 1 --2- 172.21.15.71:0/3402143609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51301016e0 0x7f513019c680 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f512c00c970 tx=0x7f512c00ce30 comp rx=0 tx=0).stop 2024-03-04T08:56:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.625+0000 7f513840e640 1 -- 172.21.15.71:0/3402143609 shutdown_connections 2024-03-04T08:56:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.625+0000 7f513840e640 1 --2- 172.21.15.71:0/3402143609 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f511c0a1e00 0x7f511c0a42c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.625+0000 7f513840e640 1 --2- 172.21.15.71:0/3402143609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51301020a0 0x7f513019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.625+0000 7f513840e640 1 --2- 172.21.15.71:0/3402143609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51301016e0 0x7f513019c680 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.625+0000 7f513840e640 1 --2- 172.21.15.71:0/3402143609 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51301004f0 0x7f51300692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.626+0000 7f513840e640 1 -- 172.21.15.71:0/3402143609 >> 172.21.15.71:0/3402143609 conn(0x7f51300fba60 msgr2=0x7f5130104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:36.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.626+0000 7f513840e640 1 -- 172.21.15.71:0/3402143609 shutdown_connections 2024-03-04T08:56:36.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.626+0000 7f513840e640 1 -- 172.21.15.71:0/3402143609 wait complete. 2024-03-04T08:56:36.649 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:56:36.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.755+0000 7fe875fa1640 1 -- 172.21.15.71:0/3270082450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe870102070 msgr2=0x7fe870104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:36.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.755+0000 7fe875fa1640 1 --2- 172.21.15.71:0/3270082450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe870102070 0x7fe870104460 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe860009900 tx=0x7fe86002bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:36.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.755+0000 7fe875fa1640 1 -- 172.21.15.71:0/3270082450 shutdown_connections 2024-03-04T08:56:36.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.755+0000 7fe875fa1640 1 --2- 172.21.15.71:0/3270082450 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8701072d0 0x7fe870109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.755+0000 7fe875fa1640 1 --2- 172.21.15.71:0/3270082450 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe8701049a0 0x7fe870106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.755+0000 7fe875fa1640 1 --2- 172.21.15.71:0/3270082450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe870102070 0x7fe870104460 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.755+0000 7fe875fa1640 1 -- 172.21.15.71:0/3270082450 >> 172.21.15.71:0/3270082450 conn(0x7fe8700fba80 msgr2=0x7fe8700fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:36.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.755+0000 7fe875fa1640 1 -- 172.21.15.71:0/3270082450 shutdown_connections 2024-03-04T08:56:36.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.755+0000 7fe875fa1640 1 -- 172.21.15.71:0/3270082450 wait complete. 2024-03-04T08:56:36.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.756+0000 7fe875fa1640 1 Processor -- start 2024-03-04T08:56:36.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.756+0000 7fe875fa1640 1 -- start start 2024-03-04T08:56:36.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.756+0000 7fe875fa1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe870102070 0x7fe870197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:36.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.756+0000 7fe875fa1640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe8701049a0 0x7fe870197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:36.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.756+0000 7fe875fa1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8701072d0 0x7fe87019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:36.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.756+0000 7fe875fa1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87010d260 con 0x7fe870102070 2024-03-04T08:56:36.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.756+0000 7fe875fa1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87010d0e0 con 0x7fe8701072d0 2024-03-04T08:56:36.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.756+0000 7fe875fa1640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87010d3e0 con 0x7fe8701049a0 2024-03-04T08:56:36.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.756+0000 7fe86f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe870102070 0x7fe870197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:36.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.756+0000 7fe86f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe870102070 0x7fe870197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56540/0 (socket says 172.21.15.71:56540) 2024-03-04T08:56:36.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.756+0000 7fe86f7fe640 1 -- 172.21.15.71:0/3865287095 learned_addr learned my addr 172.21.15.71:0/3865287095 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:36.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.756+0000 7fe86effd640 1 --2- 172.21.15.71:0/3865287095 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe8701049a0 0x7fe870197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:36.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.756+0000 7fe86ffff640 1 --2- 172.21.15.71:0/3865287095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8701072d0 0x7fe87019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:36.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.757+0000 7fe86f7fe640 1 -- 172.21.15.71:0/3865287095 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe8701049a0 msgr2=0x7fe870197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:36.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.757+0000 7fe86f7fe640 1 --2- 172.21.15.71:0/3865287095 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe8701049a0 0x7fe870197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.757+0000 7fe86f7fe640 1 -- 172.21.15.71:0/3865287095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8701072d0 msgr2=0x7fe87019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:36.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.757+0000 7fe86f7fe640 1 --2- 172.21.15.71:0/3865287095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8701072d0 0x7fe87019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.757+0000 7fe86f7fe640 1 -- 172.21.15.71:0/3865287095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87019f530 con 0x7fe870102070 2024-03-04T08:56:36.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.757+0000 7fe86f7fe640 1 --2- 172.21.15.71:0/3865287095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe870102070 0x7fe870197910 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe860009230 tx=0x7fe860005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:36.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.757+0000 7fe86cff9640 1 -- 172.21.15.71:0/3865287095 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860041070 con 0x7fe870102070 2024-03-04T08:56:36.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.757+0000 7fe86cff9640 1 -- 172.21.15.71:0/3865287095 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8600044d0 con 0x7fe870102070 2024-03-04T08:56:36.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.757+0000 7fe86cff9640 1 -- 172.21.15.71:0/3865287095 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860034440 con 0x7fe870102070 2024-03-04T08:56:36.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.757+0000 7fe875fa1640 1 -- 172.21.15.71:0/3865287095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe87019f7c0 con 0x7fe870102070 2024-03-04T08:56:36.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.757+0000 7fe875fa1640 1 -- 172.21.15.71:0/3865287095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87019fd00 con 0x7fe870102070 2024-03-04T08:56:36.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.758+0000 7fe875fa1640 1 -- 172.21.15.71:0/3865287095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8700fd570 con 0x7fe870102070 2024-03-04T08:56:36.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.759+0000 7fe86cff9640 1 -- 172.21.15.71:0/3865287095 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fe860044070 con 0x7fe870102070 2024-03-04T08:56:36.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.759+0000 7fe86cff9640 1 --2- 172.21.15.71:0/3865287095 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe8580a1e00 0x7fe8580a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:36.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.759+0000 7fe86effd640 1 --2- 172.21.15.71:0/3865287095 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe8580a1e00 0x7fe8580a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:36.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.759+0000 7fe86cff9640 1 -- 172.21.15.71:0/3865287095 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe8600efb00 con 0x7fe870102070 2024-03-04T08:56:36.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.760+0000 7fe86effd640 1 --2- 172.21.15.71:0/3865287095 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe8580a1e00 0x7fe8580a42c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe870198e20 tx=0x7fe86400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:36.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.764+0000 7fe86cff9640 1 -- 172.21.15.71:0/3865287095 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe8600bc6a0 con 0x7fe870102070 2024-03-04T08:56:36.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:36 smithi071 ceph-mon[32789]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T08:56:36.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:36 smithi071 ceph-mon[32789]: osdmap e91: 12 total, 12 up, 12 in 2024-03-04T08:56:36.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:36 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/589390750' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-04T08:56:36.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:36 smithi071 ceph-mon[32789]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T08:56:36.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:36 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2914054926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:36.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:36 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/933159328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:36.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:36 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3402143609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:36.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.976+0000 7fe875fa1640 1 -- 172.21.15.71:0/3865287095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe87002cff0 con 0x7fe870102070 2024-03-04T08:56:36.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.977+0000 7fe86cff9640 1 -- 172.21.15.71:0/3865287095 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11330 (secure 0 0 0) 0x7fe8600c0740 con 0x7fe870102070 2024-03-04T08:56:36.973 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:36.974 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3975379807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3975379807},{"type":"v1","addr":"172.21.15.71:6835","nonce":3975379807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1962929347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1962929347},{"type":"v1","addr":"172.21.15.71:6837","nonce":1962929347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/1459252176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1459252176},{"type":"v1","addr":"172.21.15.71:6839","nonce":1459252176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6841/1083546600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":1083546600},{"type":"v1","addr":"172.21.15.71:6841","nonce":1083546600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/178179528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":178179528},{"type":"v1","addr":"172.21.15.158:6833","nonce":178179528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2468877675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2468877675},{"type":"v1","addr":"172.21.15.150:6833","nonce":2468877675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1652094309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1652094309},{"type":"v1","addr":"172.21.15.158:6835","nonce":1652094309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1802416462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1802416462},{"type":"v1","addr":"172.21.15.150:6835","nonce":1802416462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/617058380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":617058380},{"type":"v1","addr":"172.21.15.158:6837","nonce":617058380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/913465600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":913465600},{"type":"v1","addr":"172.21.15.150:6837","nonce":913465600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6839/2921704804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6838","nonce":2921704804},{"type":"v1","addr":"172.21.15.158:6839","nonce":2921704804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T08:56:30.343357+0000","modified":"2024-03-04T08:56:35.008032+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/2639278083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2639278083},{"type":"v1","addr":"172.21.15.150:6839","nonce":2639278083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24491,"qdb_cluster":[24491]},"id":1}]} 2024-03-04T08:56:36.975 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 17 2024-03-04T08:56:36.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.979+0000 7fe875fa1640 1 -- 172.21.15.71:0/3865287095 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe8580a1e00 msgr2=0x7fe8580a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:36.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.979+0000 7fe875fa1640 1 --2- 172.21.15.71:0/3865287095 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe8580a1e00 0x7fe8580a42c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe870198e20 tx=0x7fe86400b390 comp rx=0 tx=0).stop 2024-03-04T08:56:36.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.979+0000 7fe875fa1640 1 -- 172.21.15.71:0/3865287095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe870102070 msgr2=0x7fe870197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:36.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.979+0000 7fe875fa1640 1 --2- 172.21.15.71:0/3865287095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe870102070 0x7fe870197910 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe860009230 tx=0x7fe860005d50 comp rx=0 tx=0).stop 2024-03-04T08:56:36.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.979+0000 7fe875fa1640 1 -- 172.21.15.71:0/3865287095 shutdown_connections 2024-03-04T08:56:36.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.979+0000 7fe875fa1640 1 --2- 172.21.15.71:0/3865287095 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe8580a1e00 0x7fe8580a42c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.979+0000 7fe875fa1640 1 --2- 172.21.15.71:0/3865287095 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8701072d0 0x7fe87019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.979+0000 7fe875fa1640 1 --2- 172.21.15.71:0/3865287095 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe8701049a0 0x7fe870197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.979+0000 7fe875fa1640 1 --2- 172.21.15.71:0/3865287095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe870102070 0x7fe870197910 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:36.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.979+0000 7fe875fa1640 1 -- 172.21.15.71:0/3865287095 >> 172.21.15.71:0/3865287095 conn(0x7fe8700fba80 msgr2=0x7fe870102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:36.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.979+0000 7fe875fa1640 1 -- 172.21.15.71:0/3865287095 shutdown_connections 2024-03-04T08:56:36.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:36.979+0000 7fe875fa1640 1 -- 172.21.15.71:0/3865287095 wait complete. 2024-03-04T08:56:36.986 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T08:56:37.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:36 smithi150 ceph-mon[36807]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T08:56:37.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:36 smithi150 ceph-mon[36807]: osdmap e91: 12 total, 12 up, 12 in 2024-03-04T08:56:37.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:36 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/589390750' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-04T08:56:37.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:36 smithi150 ceph-mon[36807]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T08:56:37.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:36 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2914054926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:37.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:36 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/933159328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:37.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:36 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3402143609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:37.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:36 smithi158 ceph-mon[37241]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T08:56:37.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:36 smithi158 ceph-mon[37241]: osdmap e91: 12 total, 12 up, 12 in 2024-03-04T08:56:37.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:36 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/589390750' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-04T08:56:37.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:36 smithi158 ceph-mon[37241]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T08:56:37.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:36 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2914054926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:37.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:36 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/933159328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:37.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:36 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3402143609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:37.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.092+0000 7f6144c83640 1 -- 172.21.15.71:0/3049444295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61400ff8e0 msgr2=0x7f61400ffd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:37.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.092+0000 7f6144c83640 1 --2- 172.21.15.71:0/3049444295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61400ff8e0 0x7f61400ffd00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6138009ca0 tx=0x7f613802c0f0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.092+0000 7f6144c83640 1 -- 172.21.15.71:0/3049444295 shutdown_connections 2024-03-04T08:56:37.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.092+0000 7f6144c83640 1 --2- 172.21.15.71:0/3049444295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6140101000 0x7f6140101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.092+0000 7f6144c83640 1 --2- 172.21.15.71:0/3049444295 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6140100640 0x7f6140100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.092+0000 7f6144c83640 1 --2- 172.21.15.71:0/3049444295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61400ff8e0 0x7f61400ffd00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.092+0000 7f6144c83640 1 -- 172.21.15.71:0/3049444295 >> 172.21.15.71:0/3049444295 conn(0x7f61400fb210 msgr2=0x7f61400fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:37.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.092+0000 7f6144c83640 1 -- 172.21.15.71:0/3049444295 shutdown_connections 2024-03-04T08:56:37.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.092+0000 7f6144c83640 1 -- 172.21.15.71:0/3049444295 wait complete. 2024-03-04T08:56:37.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.093+0000 7f6144c83640 1 Processor -- start 2024-03-04T08:56:37.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.093+0000 7f6144c83640 1 -- start start 2024-03-04T08:56:37.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.093+0000 7f6144c83640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61400ff8e0 0x7f6140199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:37.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.093+0000 7f6144c83640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6140100640 0x7f6140199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:37.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.093+0000 7f6144c83640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6140101000 0x7f61401a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:37.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.093+0000 7f6144c83640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614010f490 con 0x7f6140101000 2024-03-04T08:56:37.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.093+0000 7f6144c83640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614010f310 con 0x7f61400ff8e0 2024-03-04T08:56:37.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.093+0000 7f6144c83640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614010f610 con 0x7f6140100640 2024-03-04T08:56:37.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.093+0000 7f613ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6140101000 0x7f61401a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:37.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.093+0000 7f613ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6140101000 0x7f61401a1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56552/0 (socket says 172.21.15.71:56552) 2024-03-04T08:56:37.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.093+0000 7f613ed9d640 1 -- 172.21.15.71:0/3710040884 learned_addr learned my addr 172.21.15.71:0/3710040884 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:37.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.093+0000 7f613dd9b640 1 --2- 172.21.15.71:0/3710040884 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6140100640 0x7f6140199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:37.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.093+0000 7f613e59c640 1 --2- 172.21.15.71:0/3710040884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61400ff8e0 0x7f6140199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:37.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.093+0000 7f613ed9d640 1 -- 172.21.15.71:0/3710040884 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6140100640 msgr2=0x7f6140199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:37.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.093+0000 7f613ed9d640 1 --2- 172.21.15.71:0/3710040884 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6140100640 0x7f6140199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.093+0000 7f613ed9d640 1 -- 172.21.15.71:0/3710040884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61400ff8e0 msgr2=0x7f6140199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:37.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.093+0000 7f613ed9d640 1 --2- 172.21.15.71:0/3710040884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61400ff8e0 0x7f6140199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.093+0000 7f613ed9d640 1 -- 172.21.15.71:0/3710040884 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61401a1670 con 0x7f6140101000 2024-03-04T08:56:37.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.093+0000 7f613e59c640 1 --2- 172.21.15.71:0/3710040884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61400ff8e0 0x7f6140199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:56:37.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.093+0000 7f613ed9d640 1 --2- 172.21.15.71:0/3710040884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6140101000 0x7f61401a1000 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f613000e970 tx=0x7f613000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:37.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.094+0000 7f612f7fe640 1 -- 172.21.15.71:0/3710040884 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613000ccc0 con 0x7f6140101000 2024-03-04T08:56:37.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.094+0000 7f6144c83640 1 -- 172.21.15.71:0/3710040884 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61401a1960 con 0x7f6140101000 2024-03-04T08:56:37.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.094+0000 7f612f7fe640 1 -- 172.21.15.71:0/3710040884 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f613000ce60 con 0x7f6140101000 2024-03-04T08:56:37.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.094+0000 7f612f7fe640 1 -- 172.21.15.71:0/3710040884 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6130010490 con 0x7f6140101000 2024-03-04T08:56:37.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.094+0000 7f6144c83640 1 -- 172.21.15.71:0/3710040884 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61401a1e60 con 0x7f6140101000 2024-03-04T08:56:37.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.095+0000 7f6144c83640 1 -- 172.21.15.71:0/3710040884 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f614010b540 con 0x7f6140101000 2024-03-04T08:56:37.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.096+0000 7f612f7fe640 1 -- 172.21.15.71:0/3710040884 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f6130005020 con 0x7f6140101000 2024-03-04T08:56:37.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.096+0000 7f612f7fe640 1 --2- 172.21.15.71:0/3710040884 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f61280a1d80 0x7f61280a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:37.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.096+0000 7f613e59c640 1 --2- 172.21.15.71:0/3710040884 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f61280a1d80 0x7f61280a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:37.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.096+0000 7f612f7fe640 1 -- 172.21.15.71:0/3710040884 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f61300ce430 con 0x7f6140101000 2024-03-04T08:56:37.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.097+0000 7f613e59c640 1 --2- 172.21.15.71:0/3710040884 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f61280a1d80 0x7f61280a4240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6138006010 tx=0x7f6138032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:37.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.100+0000 7f612f7fe640 1 -- 172.21.15.71:0/3710040884 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f613009b620 con 0x7f6140101000 2024-03-04T08:56:37.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.278+0000 7f6144c83640 1 -- 172.21.15.71:0/3710040884 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6140062270 con 0x7f6140101000 2024-03-04T08:56:37.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.280+0000 7f612f7fe640 1 -- 172.21.15.71:0/3710040884 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24153 (secure 0 0 0) 0x7f613009af20 con 0x7f6140101000 2024-03-04T08:56:37.276 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:37.276 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":91,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","created":"2024-03-04T08:47:28.546102+0000","modified":"2024-03-04T08:56:35.007858+0000","last_up_change":"2024-03-04T08:55:30.744961+0000","last_in_change":"2024-03-04T08:55:11.570990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T08:51:14.933487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T08:56:28.253074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T08:56:29.543622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T08:56:32.884220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7e43b47d-8888-4631-bc96-8a96fcf8cd46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6803","nonce":1173641530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6805","nonce":1173641530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6809","nonce":1173641530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6807","nonce":1173641530}]},"public_addr":"172.21.15.71:6803/1173641530","cluster_addr":"172.21.15.71:6805/1173641530","heartbeat_back_addr":"172.21.15.71:6809/1173641530","heartbeat_front_addr":"172.21.15.71:6807/1173641530","state":["exists","up"]},{"osd":1,"uuid":"1209f214-f2e2-4209-bf69-4528a43a2123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6801","nonce":3383948110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6803","nonce":3383948110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6807","nonce":3383948110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6805","nonce":3383948110}]},"public_addr":"172.21.15.150:6801/3383948110","cluster_addr":"172.21.15.150:6803/3383948110","heartbeat_back_addr":"172.21.15.150:6807/3383948110","heartbeat_front_addr":"172.21.15.150:6805/3383948110","state":["exists","up"]},{"osd":2,"uuid":"b88c3082-93a9-4c9b-9221-a7105a44c91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6801","nonce":586944144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6803","nonce":586944144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6807","nonce":586944144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6805","nonce":586944144}]},"public_addr":"172.21.15.158:6801/586944144","cluster_addr":"172.21.15.158:6803/586944144","heartbeat_back_addr":"172.21.15.158:6807/586944144","heartbeat_front_addr":"172.21.15.158:6805/586944144","state":["exists","up"]},{"osd":3,"uuid":"e4fc7723-fd73-40b2-b9d0-7f9bda282ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6811","nonce":1311903262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6813","nonce":1311903262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6817","nonce":1311903262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6815","nonce":1311903262}]},"public_addr":"172.21.15.71:6811/1311903262","cluster_addr":"172.21.15.71:6813/1311903262","heartbeat_back_addr":"172.21.15.71:6817/1311903262","heartbeat_front_addr":"172.21.15.71:6815/1311903262","state":["exists","up"]},{"osd":4,"uuid":"e7eb72ac-f066-4ccd-b2bf-22a6a26daf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6809","nonce":2300837573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6811","nonce":2300837573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6815","nonce":2300837573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6813","nonce":2300837573}]},"public_addr":"172.21.15.150:6809/2300837573","cluster_addr":"172.21.15.150:6811/2300837573","heartbeat_back_addr":"172.21.15.150:6815/2300837573","heartbeat_front_addr":"172.21.15.150:6813/2300837573","state":["exists","up"]},{"osd":5,"uuid":"9b295840-e104-4a66-9cc4-7020627c1765","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6809","nonce":1922694045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6811","nonce":1922694045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6815","nonce":1922694045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6813","nonce":1922694045}]},"public_addr":"172.21.15.158:6809/1922694045","cluster_addr":"172.21.15.158:6811/1922694045","heartbeat_back_addr":"172.21.15.158:6815/1922694045","heartbeat_front_addr":"172.21.15.158:6813/1922694045","state":["exists","up"]},{"osd":6,"uuid":"6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6819","nonce":3848842707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6821","nonce":3848842707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6825","nonce":3848842707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6823","nonce":3848842707}]},"public_addr":"172.21.15.71:6819/3848842707","cluster_addr":"172.21.15.71:6821/3848842707","heartbeat_back_addr":"172.21.15.71:6825/3848842707","heartbeat_front_addr":"172.21.15.71:6823/3848842707","state":["exists","up"]},{"osd":7,"uuid":"6160a40a-d30f-4569-949c-0afb1bb3d133","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6817","nonce":1328330305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6819","nonce":1328330305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6823","nonce":1328330305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6821","nonce":1328330305}]},"public_addr":"172.21.15.150:6817/1328330305","cluster_addr":"172.21.15.150:6819/1328330305","heartbeat_back_addr":"172.21.15.150:6823/1328330305","heartbeat_front_addr":"172.21.15.150:6821/1328330305","state":["exists","up"]},{"osd":8,"uuid":"95ed2341-84a4-465c-9948-ac667680ce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6817","nonce":2195304577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6819","nonce":2195304577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6823","nonce":2195304577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6821","nonce":2195304577}]},"public_addr":"172.21.15.158:6817/2195304577","cluster_addr":"172.21.15.158:6819/2195304577","heartbeat_back_addr":"172.21.15.158:6823/2195304577","heartbeat_front_addr":"172.21.15.158:6821/2195304577","state":["exists","up"]},{"osd":9,"uuid":"088f6419-63a0-4768-afae-dbb8a5f5a565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6827","nonce":1913779347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6829","nonce":1913779347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6833","nonce":1913779347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6831","nonce":1913779347}]},"public_addr":"172.21.15.71:6827/1913779347","cluster_addr":"172.21.15.71:6829/1913779347","heartbeat_back_addr":"172.21.15.71:6833/1913779347","heartbeat_front_addr":"172.21.15.71:6831/1913779347","state":["exists","up"]},{"osd":10,"uuid":"18c85e24-068d-4f1e-a3b8-f7915b363234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6825","nonce":3295420996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6827","nonce":3295420996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6831","nonce":3295420996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6829","nonce":3295420996}]},"public_addr":"172.21.15.150:6825/3295420996","cluster_addr":"172.21.15.150:6827/3295420996","heartbeat_back_addr":"172.21.15.150:6831/3295420996","heartbeat_front_addr":"172.21.15.150:6829/3295420996","state":["exists","up"]},{"osd":11,"uuid":"4f6b3ddf-c482-4740-b1fd-de836e047c2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6825","nonce":3332083865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6827","nonce":3332083865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6831","nonce":3332083865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6829","nonce":3332083865}]},"public_addr":"172.21.15.158:6825/3332083865","cluster_addr":"172.21.15.158:6827/3332083865","heartbeat_back_addr":"172.21.15.158:6831/3332083865","heartbeat_front_addr":"172.21.15.158:6829/3332083865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:05.997832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:29.314235+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:08.819465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:41.194275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:06.032479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:38.814498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:13.067960+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:38.568479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:03.416713+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:38.349479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:04.096945+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:29.425250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1287712358":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/3770452860":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/718743094":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/1904717532":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:6800/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6800/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/2862184313":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/3621754630":"2024-03-05T08:48:22.593931+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T08:56:37.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.282+0000 7f6144c83640 1 -- 172.21.15.71:0/3710040884 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f61280a1d80 msgr2=0x7f61280a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:37.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.282+0000 7f6144c83640 1 --2- 172.21.15.71:0/3710040884 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f61280a1d80 0x7f61280a4240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6138006010 tx=0x7f6138032590 comp rx=0 tx=0).stop 2024-03-04T08:56:37.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.282+0000 7f6144c83640 1 -- 172.21.15.71:0/3710040884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6140101000 msgr2=0x7f61401a1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:37.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.282+0000 7f6144c83640 1 --2- 172.21.15.71:0/3710040884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6140101000 0x7f61401a1000 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f613000e970 tx=0x7f613000ed70 comp rx=0 tx=0).stop 2024-03-04T08:56:37.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.282+0000 7f6144c83640 1 -- 172.21.15.71:0/3710040884 shutdown_connections 2024-03-04T08:56:37.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.282+0000 7f6144c83640 1 --2- 172.21.15.71:0/3710040884 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f61280a1d80 0x7f61280a4240 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.282+0000 7f6144c83640 1 --2- 172.21.15.71:0/3710040884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6140101000 0x7f61401a1000 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.282+0000 7f6144c83640 1 --2- 172.21.15.71:0/3710040884 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6140100640 0x7f6140199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.282+0000 7f6144c83640 1 --2- 172.21.15.71:0/3710040884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61400ff8e0 0x7f6140199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.282+0000 7f6144c83640 1 -- 172.21.15.71:0/3710040884 >> 172.21.15.71:0/3710040884 conn(0x7f61400fb210 msgr2=0x7f61400fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:37.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.282+0000 7f6144c83640 1 -- 172.21.15.71:0/3710040884 shutdown_connections 2024-03-04T08:56:37.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.282+0000 7f6144c83640 1 -- 172.21.15.71:0/3710040884 wait complete. 2024-03-04T08:56:37.290 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-04T08:56:30.343357+0000', 'modified': '2024-03-04T08:56:35.008032+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24491}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24491': {'gid': 24491, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6839/2639278083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6838', 'nonce': 2639278083}, {'type': 'v1', 'addr': '172.21.15.150:6839', 'nonce': 2639278083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24491, 'qdb_cluster': [24491]} 2024-03-04T08:56:37.290 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-04T08:56:37.290 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-04T08:56:37.405 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.396+0000 7f9b34e81640 1 -- 172.21.15.71:0/1989734221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b30101160 msgr2=0x7f9b301015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.396+0000 7f9b34e81640 1 --2- 172.21.15.71:0/1989734221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b30101160 0x7f9b301015e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9b28009ca0 tx=0x7f9b2802c0f0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.396+0000 7f9b34e81640 1 -- 172.21.15.71:0/1989734221 shutdown_connections 2024-03-04T08:56:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.396+0000 7f9b34e81640 1 --2- 172.21.15.71:0/1989734221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b30101160 0x7f9b301015e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.396+0000 7f9b34e81640 1 --2- 172.21.15.71:0/1989734221 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b301007c0 0x7f9b30100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.396+0000 7f9b34e81640 1 --2- 172.21.15.71:0/1989734221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b300ffe60 0x7f9b30100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.396+0000 7f9b34e81640 1 -- 172.21.15.71:0/1989734221 >> 172.21.15.71:0/1989734221 conn(0x7f9b300fba80 msgr2=0x7f9b300fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.396+0000 7f9b34e81640 1 -- 172.21.15.71:0/1989734221 shutdown_connections 2024-03-04T08:56:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.396+0000 7f9b34e81640 1 -- 172.21.15.71:0/1989734221 wait complete. 2024-03-04T08:56:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.397+0000 7f9b34e81640 1 Processor -- start 2024-03-04T08:56:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.397+0000 7f9b34e81640 1 -- start start 2024-03-04T08:56:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.397+0000 7f9b34e81640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b300ffe60 0x7f9b30199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:37.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.397+0000 7f9b34e81640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b301007c0 0x7f9b30199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.397+0000 7f9b34e81640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b30101160 0x7f9b301a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.397+0000 7f9b34e81640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3010f440 con 0x7f9b301007c0 2024-03-04T08:56:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.397+0000 7f9b34e81640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3010f2c0 con 0x7f9b30101160 2024-03-04T08:56:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.397+0000 7f9b34e81640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3010f5c0 con 0x7f9b300ffe60 2024-03-04T08:56:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.397+0000 7f9b2dd9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b301007c0 0x7f9b30199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.397+0000 7f9b2dd9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b301007c0 0x7f9b30199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56572/0 (socket says 172.21.15.71:56572) 2024-03-04T08:56:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.397+0000 7f9b2dd9b640 1 -- 172.21.15.71:0/3500350689 learned_addr learned my addr 172.21.15.71:0/3500350689 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.397+0000 7f9b2ed9d640 1 --2- 172.21.15.71:0/3500350689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b30101160 0x7f9b301a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.398+0000 7f9b2e59c640 1 --2- 172.21.15.71:0/3500350689 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b300ffe60 0x7f9b30199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.398+0000 7f9b2dd9b640 1 -- 172.21.15.71:0/3500350689 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b300ffe60 msgr2=0x7f9b30199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.398+0000 7f9b2dd9b640 1 --2- 172.21.15.71:0/3500350689 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b300ffe60 0x7f9b30199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.398+0000 7f9b2dd9b640 1 -- 172.21.15.71:0/3500350689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b30101160 msgr2=0x7f9b301a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.398+0000 7f9b2dd9b640 1 --2- 172.21.15.71:0/3500350689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b30101160 0x7f9b301a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.398+0000 7f9b2dd9b640 1 -- 172.21.15.71:0/3500350689 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b301a1680 con 0x7f9b301007c0 2024-03-04T08:56:37.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.398+0000 7f9b2dd9b640 1 --2- 172.21.15.71:0/3500350689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b301007c0 0x7f9b30199ff0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400c890 tx=0x7f9b2400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:37.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.398+0000 7f9b1f7fe640 1 -- 172.21.15.71:0/3500350689 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b24004220 con 0x7f9b301007c0 2024-03-04T08:56:37.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.398+0000 7f9b1f7fe640 1 -- 172.21.15.71:0/3500350689 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b240043c0 con 0x7f9b301007c0 2024-03-04T08:56:37.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.398+0000 7f9b34e81640 1 -- 172.21.15.71:0/3500350689 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b301a1970 con 0x7f9b301007c0 2024-03-04T08:56:37.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.398+0000 7f9b1f7fe640 1 -- 172.21.15.71:0/3500350689 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b240049e0 con 0x7f9b301007c0 2024-03-04T08:56:37.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.398+0000 7f9b34e81640 1 -- 172.21.15.71:0/3500350689 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b301a1ec0 con 0x7f9b301007c0 2024-03-04T08:56:37.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.400+0000 7f9b1f7fe640 1 -- 172.21.15.71:0/3500350689 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f9b240079b0 con 0x7f9b301007c0 2024-03-04T08:56:37.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.400+0000 7f9b37a68740 1 Processor -- start 2024-03-04T08:56:37.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.401+0000 7f9b37a68740 1 -- start start 2024-03-04T08:56:37.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.401+0000 7f9b37a68740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e997b0 0x562562e99bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:37.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.401+0000 7f9b1f7fe640 1 --2- 172.21.15.71:0/3500350689 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f9b180a1d80 0x7f9b180a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:37.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.401+0000 7f9b37a68740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x562562e9a980 0x562562e9ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:37.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.401+0000 7f9b37a68740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562562e9b340 0x562562ea1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:37.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.401+0000 7f9b37a68740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562562dfe6b0 con 0x562562e997b0 2024-03-04T08:56:37.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.401+0000 7f9b37a68740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562562da7a60 con 0x562562e9b340 2024-03-04T08:56:37.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.401+0000 7f9b37a68740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x562562dfa750 con 0x562562e9a980 2024-03-04T08:56:37.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.401+0000 7f9b2e59c640 1 --2- 172.21.15.71:0/3500350689 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f9b180a1d80 0x7f9b180a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:37.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.401+0000 7f9b2ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e997b0 0x562562e99bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:37.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.401+0000 7f9b2ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e997b0 0x562562e99bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56576/0 (socket says 172.21.15.71:56576) 2024-03-04T08:56:37.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.401+0000 7f9b2ed9d640 1 -- 172.21.15.71:0/968931897 learned_addr learned my addr 172.21.15.71:0/968931897 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:37.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.401+0000 7f9b2dd9b640 1 --2- 172.21.15.71:0/968931897 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x562562e9a980 0x562562e9ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:37.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.401+0000 7f9b2ed9d640 1 --2- 172.21.15.71:0/968931897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562562e9b340 0x562562ea1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:37.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.401+0000 7f9b1f7fe640 1 -- 172.21.15.71:0/3500350689 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f9b240ce070 con 0x7f9b301007c0 2024-03-04T08:56:37.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.401+0000 7f9b2ed9d640 1 -- 172.21.15.71:0/968931897 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x562562e9a980 msgr2=0x562562e9ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:56:37.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.401+0000 7f9b2ed9d640 1 --2- 172.21.15.71:0/968931897 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x562562e9a980 0x562562e9ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.401+0000 7f9b2ed9d640 1 -- 172.21.15.71:0/968931897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562562e9b340 msgr2=0x562562ea1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:56:37.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.401+0000 7f9b2ed9d640 1 --2- 172.21.15.71:0/968931897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562562e9b340 0x562562ea1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.401+0000 7f9b2ed9d640 1 -- 172.21.15.71:0/968931897 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562562dc5f30 con 0x562562e997b0 2024-03-04T08:56:37.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.401+0000 7f9b2dd9b640 1 --2- 172.21.15.71:0/968931897 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x562562e9a980 0x562562e9ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:56:37.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.401+0000 7f9b2e59c640 1 --2- 172.21.15.71:0/3500350689 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f9b180a1d80 0x7f9b180a4240 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9b200095f0 tx=0x7f9b20005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:37.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.402+0000 7f9b2ed9d640 1 --2- 172.21.15.71:0/968931897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e997b0 0x562562e99bd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9b280324c0 tx=0x7f9b28032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb94c3ab3924d436 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:37.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.402+0000 7f9b1cff9640 1 -- 172.21.15.71:0/968931897 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b28039040 con 0x562562e997b0 2024-03-04T08:56:37.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.402+0000 7f9b1cff9640 1 -- 172.21.15.71:0/968931897 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b28003970 con 0x562562e997b0 2024-03-04T08:56:37.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.402+0000 7f9b1cff9640 1 -- 172.21.15.71:0/968931897 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b28004450 con 0x562562e997b0 2024-03-04T08:56:37.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.402+0000 7f9b37a68740 1 -- 172.21.15.71:0/968931897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e997b0 msgr2=0x562562e99bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:37.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.402+0000 7f9b37a68740 1 --2- 172.21.15.71:0/968931897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e997b0 0x562562e99bd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9b280324c0 tx=0x7f9b28032340 comp rx=0 tx=0).stop 2024-03-04T08:56:37.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.402+0000 7f9b37a68740 1 -- 172.21.15.71:0/968931897 shutdown_connections 2024-03-04T08:56:37.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.402+0000 7f9b37a68740 1 --2- 172.21.15.71:0/968931897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562562e9b340 0x562562ea1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.402+0000 7f9b37a68740 1 --2- 172.21.15.71:0/968931897 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x562562e9a980 0x562562e9ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.402+0000 7f9b37a68740 1 --2- 172.21.15.71:0/968931897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e997b0 0x562562e99bd0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.402+0000 7f9b37a68740 1 -- 172.21.15.71:0/968931897 >> 172.21.15.71:0/968931897 conn(0x562562e94ba0 msgr2=0x562562e94fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:37.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.403+0000 7f9b37a68740 1 -- 172.21.15.71:0/968931897 shutdown_connections 2024-03-04T08:56:37.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.403+0000 7f9b37a68740 1 -- 172.21.15.71:0/968931897 wait complete. 2024-03-04T08:56:37.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.403+0000 7f9b37a68740 1 -- start start 2024-03-04T08:56:37.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.403+0000 7f9b37a68740 1 Processor -- start 2024-03-04T08:56:37.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.403+0000 7f9b37a68740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562562e997b0 0x562562ea3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:37.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.403+0000 7f9b37a68740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e9a980 0x562562ea4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:37.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.403+0000 7f9b37a68740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x562562e9b340 0x562562eab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:37.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.403+0000 7f9b37a68740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562562ea22f0 con 0x562562e9a980 2024-03-04T08:56:37.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.403+0000 7f9b37a68740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562562dbaea0 con 0x562562e997b0 2024-03-04T08:56:37.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.403+0000 7f9b37a68740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x562562ea2470 con 0x562562e9b340 2024-03-04T08:56:37.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.403+0000 7f9b37a68740 10 client.0 _ll_register_callbacks cb 0x562562dba870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T08:56:37.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.403+0000 7f9b2dd9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e9a980 0x562562ea4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:37.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.403+0000 7f9b2dd9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e9a980 0x562562ea4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56586/0 (socket says 172.21.15.71:56586) 2024-03-04T08:56:37.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.403+0000 7f9b2dd9b640 1 -- 172.21.15.71:0/4189555455 learned_addr learned my addr 172.21.15.71:0/4189555455 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:37.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.403+0000 7f9b2e59c640 1 --2- 172.21.15.71:0/4189555455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562562e997b0 0x562562ea3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:37.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.403+0000 7f9b2dd9b640 1 -- 172.21.15.71:0/4189555455 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x562562e9b340 msgr2=0x562562eab250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:56:37.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.403+0000 7f9b2dd9b640 1 --2- 172.21.15.71:0/4189555455 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x562562e9b340 0x562562eab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.403+0000 7f9b2dd9b640 1 -- 172.21.15.71:0/4189555455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562562e997b0 msgr2=0x562562ea3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:56:37.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.403+0000 7f9b2dd9b640 1 --2- 172.21.15.71:0/4189555455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562562e997b0 0x562562ea3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.403+0000 7f9b2dd9b640 1 -- 172.21.15.71:0/4189555455 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562562dbb270 con 0x562562e9a980 2024-03-04T08:56:37.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.404+0000 7f9b2dd9b640 1 --2- 172.21.15.71:0/4189555455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e9a980 0x562562ea4210 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9b2409aaf0 tx=0x7f9b24096f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26661bcf59fc269 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:37.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.404+0000 7f9b15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-04T08:56:37.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.404+0000 7f9b15ffb640 1 -- 172.21.15.71:0/4189555455 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2401d040 con 0x562562e9a980 2024-03-04T08:56:37.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.404+0000 7f9b15ffb640 1 -- 172.21.15.71:0/4189555455 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f9b24098c00 con 0x562562e9a980 2024-03-04T08:56:37.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.404+0000 7f9b37a68740 1 -- 172.21.15.71:0/4189555455 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b28003970 con 0x562562e9a980 2024-03-04T08:56:37.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.404+0000 7f9b15ffb640 1 -- 172.21.15.71:0/4189555455 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b240a01d0 con 0x562562e9a980 2024-03-04T08:56:37.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.404+0000 7f9b37a68740 10 client.14763 fetch_fsmap learned FSMap version 17 2024-03-04T08:56:37.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.404+0000 7f9b37a68740 1 -- 172.21.15.71:0/4189555455 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x562562b132d0 con 0x562562e9a980 2024-03-04T08:56:37.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.404+0000 7f9b15ffb640 1 -- 172.21.15.71:0/4189555455 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b28003970 con 0x562562e9a980 2024-03-04T08:56:37.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.404+0000 7f9b15ffb640 1 -- 172.21.15.71:0/4189555455 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 17) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f9b24007ca0 con 0x562562e9a980 2024-03-04T08:56:37.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.404+0000 7f9b37a68740 10 client.14763 fetch_fsmap finished waiting for FSMap version 17 2024-03-04T08:56:37.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.404+0000 7f9b37a68740 10 client.14763 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-04T08:56:37.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.404+0000 7f9b37a68740 20 client.14763 populate_metadata read hostname 'smithi071' 2024-03-04T08:56:37.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.404+0000 7f9b37a68740 1 --2- 172.21.15.71:0/4189555455 >> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] conn(0x562562eb08b0 0x562562eaf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:37.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.404+0000 7f9b37a68740 4 client.14763 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-04T08:56:37.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.404+0000 7f9b37a68740 1 -- 172.21.15.71:0/4189555455 --> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562562e0bbf0 con 0x562562eb08b0 2024-03-04T08:56:37.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.404+0000 7f9b2e59c640 1 --2- 172.21.15.71:0/4189555455 >> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] conn(0x562562eb08b0 0x562562eaf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:37.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.405+0000 7f9b15ffb640 10 client.14763 ms_handle_connect on v2:172.21.15.150:6838/2639278083 2024-03-04T08:56:37.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.405+0000 7f9b2e59c640 1 --2- 172.21.15.71:0/4189555455 >> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] conn(0x562562eb08b0 0x562562eaf930 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1810ee119eb81fe5 server_cookie=345f1394ca81a117 in_seq=0 out_seq=0 2024-03-04T08:56:37.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.405+0000 7f9b15ffb640 1 -- 172.21.15.71:0/4189555455 <== mds.0 v2:172.21.15.150:6838/2639278083 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562562e0bbf0 con 0x562562eb08b0 2024-03-04T08:56:37.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.405+0000 7f9b15ffb640 10 client.14763 handle_command_reply: tid=0 2024-03-04T08:56:37.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.405+0000 7f9b37a68740 1 client.14763 shutdown 2024-03-04T08:56:37.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.406+0000 7f9b37a68740 1 -- 172.21.15.71:0/4189555455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e9a980 msgr2=0x562562ea4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:37.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.406+0000 7f9b37a68740 1 --2- 172.21.15.71:0/4189555455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e9a980 0x562562ea4210 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9b2409aaf0 tx=0x7f9b24096f30 comp rx=0 tx=0).stop 2024-03-04T08:56:37.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.406+0000 7f9b37a68740 1 -- 172.21.15.71:0/4189555455 shutdown_connections 2024-03-04T08:56:37.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.406+0000 7f9b37a68740 1 --2- 172.21.15.71:0/4189555455 >> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] conn(0x562562eb08b0 0x562562eaf930 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.406+0000 7f9b37a68740 1 --2- 172.21.15.71:0/4189555455 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x562562e9b340 0x562562eab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.406+0000 7f9b37a68740 1 --2- 172.21.15.71:0/4189555455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e9a980 0x562562ea4210 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.406+0000 7f9b37a68740 1 --2- 172.21.15.71:0/4189555455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562562e997b0 0x562562ea3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.406+0000 7f9b37a68740 1 -- 172.21.15.71:0/4189555455 >> 172.21.15.71:0/4189555455 conn(0x562562e94ba0 msgr2=0x562562e93da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:37.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.406+0000 7f9b15ffb640 0 client.14763 ms_handle_reset on v2:172.21.15.150:6838/2639278083 2024-03-04T08:56:37.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.406+0000 7f9b37a68740 1 -- 172.21.15.71:0/4189555455 shutdown_connections 2024-03-04T08:56:37.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.406+0000 7f9b37a68740 1 -- 172.21.15.71:0/4189555455 wait complete. 2024-03-04T08:56:37.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.406+0000 7f9b37a68740 20 client.14763 trim_cache size 0 max 16384 2024-03-04T08:56:37.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.422+0000 7f9b37a68740 1 Processor -- start 2024-03-04T08:56:37.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.422+0000 7f9b37a68740 1 -- start start 2024-03-04T08:56:37.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.422+0000 7f9b37a68740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e997b0 0x562562e9d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:37.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.422+0000 7f9b37a68740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562562e9a980 0x562562eab210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:37.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.422+0000 7f9b37a68740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x562562e9b340 0x562562ea38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:37.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.422+0000 7f9b37a68740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562562eb00d0 con 0x562562e997b0 2024-03-04T08:56:37.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.422+0000 7f9b37a68740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562562eb0730 con 0x562562e9a980 2024-03-04T08:56:37.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.422+0000 7f9b37a68740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x562562eb0d90 con 0x562562e9b340 2024-03-04T08:56:37.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.422+0000 7f9b2e59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e997b0 0x562562e9d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:37.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.422+0000 7f9b2e59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e997b0 0x562562e9d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56600/0 (socket says 172.21.15.71:56600) 2024-03-04T08:56:37.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.422+0000 7f9b2e59c640 1 -- 172.21.15.71:0/252491440 learned_addr learned my addr 172.21.15.71:0/252491440 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:37.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.422+0000 7f9b2dd9b640 1 --2- 172.21.15.71:0/252491440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562562e9a980 0x562562eab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:37.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.423+0000 7f9b2e59c640 1 -- 172.21.15.71:0/252491440 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x562562e9b340 msgr2=0x562562ea38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:56:37.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.423+0000 7f9b2e59c640 1 --2- 172.21.15.71:0/252491440 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x562562e9b340 0x562562ea38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.423+0000 7f9b2e59c640 1 -- 172.21.15.71:0/252491440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562562e9a980 msgr2=0x562562eab210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:56:37.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.423+0000 7f9b2e59c640 1 --2- 172.21.15.71:0/252491440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562562e9a980 0x562562eab210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.423+0000 7f9b2e59c640 1 -- 172.21.15.71:0/252491440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562562bdd6b0 con 0x562562e997b0 2024-03-04T08:56:37.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.423+0000 7f9b2e59c640 1 --2- 172.21.15.71:0/252491440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e997b0 0x562562e9d980 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009420 tx=0x7f9b20014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b47fad0b84b77 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:37.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.423+0000 7f9b17fff640 1 -- 172.21.15.71:0/252491440 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2002d070 con 0x562562e997b0 2024-03-04T08:56:37.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.423+0000 7f9b17fff640 1 -- 172.21.15.71:0/252491440 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b200282d0 con 0x562562e997b0 2024-03-04T08:56:37.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.423+0000 7f9b17fff640 1 -- 172.21.15.71:0/252491440 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2001fac0 con 0x562562e997b0 2024-03-04T08:56:37.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.423+0000 7f9b37a68740 1 -- 172.21.15.71:0/252491440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e997b0 msgr2=0x562562e9d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:37.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.423+0000 7f9b37a68740 1 --2- 172.21.15.71:0/252491440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e997b0 0x562562e9d980 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009420 tx=0x7f9b20014000 comp rx=0 tx=0).stop 2024-03-04T08:56:37.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.423+0000 7f9b37a68740 1 -- 172.21.15.71:0/252491440 shutdown_connections 2024-03-04T08:56:37.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.423+0000 7f9b37a68740 1 --2- 172.21.15.71:0/252491440 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x562562e9b340 0x562562ea38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.423+0000 7f9b37a68740 1 --2- 172.21.15.71:0/252491440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562562e9a980 0x562562eab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.423+0000 7f9b37a68740 1 --2- 172.21.15.71:0/252491440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e997b0 0x562562e9d980 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.423+0000 7f9b37a68740 1 -- 172.21.15.71:0/252491440 >> 172.21.15.71:0/252491440 conn(0x562562e94ba0 msgr2=0x562562e94f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:37.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.423+0000 7f9b37a68740 1 -- 172.21.15.71:0/252491440 shutdown_connections 2024-03-04T08:56:37.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.424+0000 7f9b37a68740 1 -- 172.21.15.71:0/252491440 wait complete. 2024-03-04T08:56:37.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.424+0000 7f9b37a68740 1 -- start start 2024-03-04T08:56:37.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.424+0000 7f9b37a68740 1 Processor -- start 2024-03-04T08:56:37.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.424+0000 7f9b37a68740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x562562e997b0 0x562562ea5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:37.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.424+0000 7f9b37a68740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e9a980 0x562562ea5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:37.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.424+0000 7f9b37a68740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562562e9b340 0x562562eaa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:37.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.424+0000 7f9b37a68740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562562eb1a50 con 0x562562e9a980 2024-03-04T08:56:37.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.424+0000 7f9b37a68740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562562eb13f0 con 0x562562e9b340 2024-03-04T08:56:37.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.424+0000 7f9b37a68740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x562562eb20b0 con 0x562562e997b0 2024-03-04T08:56:37.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.424+0000 7f9b37a68740 10 client.0 _ll_register_callbacks cb 0x7f9b2409a930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T08:56:37.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.424+0000 7f9b2dd9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e9a980 0x562562ea5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:37.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.424+0000 7f9b2dd9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e9a980 0x562562ea5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56608/0 (socket says 172.21.15.71:56608) 2024-03-04T08:56:37.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.424+0000 7f9b2dd9b640 1 -- 172.21.15.71:0/203283581 learned_addr learned my addr 172.21.15.71:0/203283581 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:37.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.424+0000 7f9b2ed9d640 1 --2- 172.21.15.71:0/203283581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562562e9b340 0x562562eaa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:37.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.424+0000 7f9b2e59c640 1 --2- 172.21.15.71:0/203283581 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x562562e997b0 0x562562ea5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:37.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.424+0000 7f9b2dd9b640 1 -- 172.21.15.71:0/203283581 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x562562e997b0 msgr2=0x562562ea5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:56:37.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.424+0000 7f9b2dd9b640 1 --2- 172.21.15.71:0/203283581 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x562562e997b0 0x562562ea5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.424+0000 7f9b2dd9b640 1 -- 172.21.15.71:0/203283581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562562e9b340 msgr2=0x562562eaa5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:56:37.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.424+0000 7f9b2dd9b640 1 --2- 172.21.15.71:0/203283581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562562e9b340 0x562562eaa5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.424+0000 7f9b2dd9b640 1 -- 172.21.15.71:0/203283581 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562562eaab90 con 0x562562e9a980 2024-03-04T08:56:37.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.425+0000 7f9b2dd9b640 1 --2- 172.21.15.71:0/203283581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e9a980 0x562562ea5970 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9b24005450 tx=0x7f9b24096e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4071cf54de0039a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:37.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.425+0000 7f9b15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-04T08:56:37.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.425+0000 7f9b15ffb640 1 -- 172.21.15.71:0/203283581 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2401d040 con 0x562562e9a980 2024-03-04T08:56:37.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.425+0000 7f9b15ffb640 1 -- 172.21.15.71:0/203283581 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f9b240a3070 con 0x562562e9a980 2024-03-04T08:56:37.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.425+0000 7f9b37a68740 1 -- 172.21.15.71:0/203283581 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b200282d0 con 0x562562e9a980 2024-03-04T08:56:37.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.425+0000 7f9b15ffb640 1 -- 172.21.15.71:0/203283581 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b2409fbd0 con 0x562562e9a980 2024-03-04T08:56:37.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.425+0000 7f9b37a68740 10 client.14769 fetch_fsmap learned FSMap version 17 2024-03-04T08:56:37.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.425+0000 7f9b37a68740 1 -- 172.21.15.71:0/203283581 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x562562f0c050 con 0x562562e9a980 2024-03-04T08:56:37.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.425+0000 7f9b15ffb640 1 -- 172.21.15.71:0/203283581 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b200282d0 con 0x562562e9a980 2024-03-04T08:56:37.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.425+0000 7f9b15ffb640 1 -- 172.21.15.71:0/203283581 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 17) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f9b24060170 con 0x562562e9a980 2024-03-04T08:56:37.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.425+0000 7f9b37a68740 10 client.14769 fetch_fsmap finished waiting for FSMap version 17 2024-03-04T08:56:37.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.425+0000 7f9b37a68740 10 client.14769 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-04T08:56:37.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.425+0000 7f9b37a68740 20 client.14769 populate_metadata read hostname 'smithi071' 2024-03-04T08:56:37.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.425+0000 7f9b37a68740 1 --2- 172.21.15.71:0/203283581 >> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] conn(0x562562f103b0 0x562562f0c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:37.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.425+0000 7f9b37a68740 4 client.14769 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-04T08:56:37.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.425+0000 7f9b37a68740 1 -- 172.21.15.71:0/203283581 --> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562562e05f40 con 0x562562f103b0 2024-03-04T08:56:37.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.425+0000 7f9b2e59c640 1 --2- 172.21.15.71:0/203283581 >> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] conn(0x562562f103b0 0x562562f0c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:37.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.426+0000 7f9b2e59c640 1 --2- 172.21.15.71:0/203283581 >> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] conn(0x562562f103b0 0x562562f0c050 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f764be25106fb598 server_cookie=ad8cd7783cfb6fe5 in_seq=0 out_seq=0 2024-03-04T08:56:37.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.426+0000 7f9b15ffb640 10 client.14769 ms_handle_connect on v2:172.21.15.150:6838/2639278083 2024-03-04T08:56:37.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.426+0000 7f9b15ffb640 1 -- 172.21.15.71:0/203283581 <== mds.0 v2:172.21.15.150:6838/2639278083 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x562562e05f40 con 0x562562f103b0 2024-03-04T08:56:37.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.426+0000 7f9b15ffb640 10 client.14769 handle_command_reply: tid=0 2024-03-04T08:56:37.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.426+0000 7f9b37a68740 1 client.14769 shutdown 2024-03-04T08:56:37.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.426+0000 7f9b37a68740 1 -- 172.21.15.71:0/203283581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e9a980 msgr2=0x562562ea5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:37.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.426+0000 7f9b37a68740 1 --2- 172.21.15.71:0/203283581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e9a980 0x562562ea5970 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9b24005450 tx=0x7f9b24096e80 comp rx=0 tx=0).stop 2024-03-04T08:56:37.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.426+0000 7f9b37a68740 1 -- 172.21.15.71:0/203283581 shutdown_connections 2024-03-04T08:56:37.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.426+0000 7f9b37a68740 1 --2- 172.21.15.71:0/203283581 >> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] conn(0x562562f103b0 0x562562f0c050 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.426+0000 7f9b37a68740 1 --2- 172.21.15.71:0/203283581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562562e9b340 0x562562eaa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.426+0000 7f9b15ffb640 0 client.14769 ms_handle_reset on v2:172.21.15.150:6838/2639278083 2024-03-04T08:56:37.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.426+0000 7f9b37a68740 1 --2- 172.21.15.71:0/203283581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562562e9a980 0x562562ea5970 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.426+0000 7f9b37a68740 1 --2- 172.21.15.71:0/203283581 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x562562e997b0 0x562562ea5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.426+0000 7f9b37a68740 1 -- 172.21.15.71:0/203283581 >> 172.21.15.71:0/203283581 conn(0x562562e94ba0 msgr2=0x562562ea38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:37.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.426+0000 7f9b37a68740 1 -- 172.21.15.71:0/203283581 shutdown_connections 2024-03-04T08:56:37.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.426+0000 7f9b37a68740 1 -- 172.21.15.71:0/203283581 wait complete. 2024-03-04T08:56:37.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.426+0000 7f9b37a68740 20 client.14769 trim_cache size 0 max 16384 2024-03-04T08:56:37.441 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-03-04T08:56:37.441 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "b3c85c40-da03-11ee-95c6-87774f69a715", 2024-03-04T08:56:37.441 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2024-03-04T08:56:37.441 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 24491, 2024-03-04T08:56:37.442 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-03-04T08:56:37.442 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-03-04T08:56:37.442 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-03-04T08:56:37.442 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 6.4340700870000003, 2024-03-04T08:56:37.442 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 17, 2024-03-04T08:56:37.442 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 91, 2024-03-04T08:56:37.443 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 91, 2024-03-04T08:56:37.443 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 11.580270499999999 2024-03-04T08:56:37.443 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-03-04T08:56:37.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.427+0000 7f9b34e81640 1 -- 172.21.15.71:0/3500350689 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f9b180a1d80 msgr2=0x7f9b180a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:37.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.427+0000 7f9b34e81640 1 --2- 172.21.15.71:0/3500350689 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f9b180a1d80 0x7f9b180a4240 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9b200095f0 tx=0x7f9b20005f90 comp rx=0 tx=0).stop 2024-03-04T08:56:37.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.427+0000 7f9b34e81640 1 -- 172.21.15.71:0/3500350689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b301007c0 msgr2=0x7f9b30199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:37.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.427+0000 7f9b34e81640 1 --2- 172.21.15.71:0/3500350689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b301007c0 0x7f9b30199ff0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400c890 tx=0x7f9b2400cd50 comp rx=0 tx=0).stop 2024-03-04T08:56:37.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.427+0000 7f9b34e81640 1 -- 172.21.15.71:0/3500350689 shutdown_connections 2024-03-04T08:56:37.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.427+0000 7f9b34e81640 1 --2- 172.21.15.71:0/3500350689 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f9b180a1d80 0x7f9b180a4240 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.427+0000 7f9b34e81640 1 --2- 172.21.15.71:0/3500350689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b30101160 0x7f9b301a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.427+0000 7f9b34e81640 1 --2- 172.21.15.71:0/3500350689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b301007c0 0x7f9b30199ff0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.427+0000 7f9b34e81640 1 --2- 172.21.15.71:0/3500350689 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9b300ffe60 0x7f9b30199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:37.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.427+0000 7f9b34e81640 1 -- 172.21.15.71:0/3500350689 >> 172.21.15.71:0/3500350689 conn(0x7f9b300fba80 msgr2=0x7f9b300fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:37.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.427+0000 7f9b34e81640 1 -- 172.21.15.71:0/3500350689 shutdown_connections 2024-03-04T08:56:37.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:37.427+0000 7f9b34e81640 1 -- 172.21.15.71:0/3500350689 wait complete. 2024-03-04T08:56:37.448 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-04T08:56:37.448 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-04T08:56:37.449 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph fs ls 2024-03-04T08:56:37.685 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:56:37.943 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:37 smithi071 ceph-mon[32789]: pgmap v328: 129 pgs: 105 active+clean, 5 creating+peering, 19 unknown; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2024-03-04T08:56:37.943 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:37 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3865287095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:37.943 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:37 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3710040884' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:37.953 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:37 smithi158 ceph-mon[37241]: pgmap v328: 129 pgs: 105 active+clean, 5 creating+peering, 19 unknown; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2024-03-04T08:56:37.953 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:37 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3865287095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:37.954 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:37 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3710040884' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:38.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:37 smithi150 ceph-mon[36807]: pgmap v328: 129 pgs: 105 active+clean, 5 creating+peering, 19 unknown; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2024-03-04T08:56:38.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:37 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3865287095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:38.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:37 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3710040884' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:38.956 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:38 smithi071 ceph-mon[32789]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-04T08:56:38.956 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:38 smithi071 ceph-mon[32789]: Cluster is now healthy 2024-03-04T08:56:38.966 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:38 smithi158 ceph-mon[37241]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-04T08:56:38.966 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:38 smithi158 ceph-mon[37241]: Cluster is now healthy 2024-03-04T08:56:39.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:38 smithi150 ceph-mon[36807]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-04T08:56:39.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:38 smithi150 ceph-mon[36807]: Cluster is now healthy 2024-03-04T08:56:39.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.488+0000 7fa0b7548700 1 -- 172.21.15.71:0/3345275717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0b0100fc0 msgr2=0x7fa0b0101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:39.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.488+0000 7fa0b7548700 1 --2- 172.21.15.71:0/3345275717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0b0100fc0 0x7fa0b0101420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0009990 tx=0x7fa0a0009ca0 comp rx=0 tx=0).stop 2024-03-04T08:56:39.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.488+0000 7fa0b7548700 1 -- 172.21.15.71:0/3345275717 shutdown_connections 2024-03-04T08:56:39.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.488+0000 7fa0b7548700 1 --2- 172.21.15.71:0/3345275717 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa0b0101960 0x7fa0b010de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:39.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.488+0000 7fa0b7548700 1 --2- 172.21.15.71:0/3345275717 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0b0107000 0x7fa0b01073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:39.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.488+0000 7fa0b7548700 1 --2- 172.21.15.71:0/3345275717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0b0100fc0 0x7fa0b0101420 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:39.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.488+0000 7fa0b7548700 1 -- 172.21.15.71:0/3345275717 >> 172.21.15.71:0/3345275717 conn(0x7fa0b0077ef0 msgr2=0x7fa0b0078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:39.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.488+0000 7fa0b7548700 1 -- 172.21.15.71:0/3345275717 shutdown_connections 2024-03-04T08:56:39.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.488+0000 7fa0b7548700 1 -- 172.21.15.71:0/3345275717 wait complete. 2024-03-04T08:56:39.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.489+0000 7fa0b7548700 1 Processor -- start 2024-03-04T08:56:39.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.489+0000 7fa0b7548700 1 -- start start 2024-03-04T08:56:39.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.489+0000 7fa0b7548700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa0b0100fc0 0x7fa0b0199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:39.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.489+0000 7fa0b7548700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0b0101960 0x7fa0b0199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:39.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.489+0000 7fa0b7548700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0b0107000 0x7fa0b01a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:39.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.489+0000 7fa0b7548700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b0111280 con 0x7fa0b0107000 2024-03-04T08:56:39.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.489+0000 7fa0b7548700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b0111100 con 0x7fa0b0101960 2024-03-04T08:56:39.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.489+0000 7fa0b7548700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b0111400 con 0x7fa0b0100fc0 2024-03-04T08:56:39.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.489+0000 7fa0b5ae5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0b0107000 0x7fa0b01a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:39.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.489+0000 7fa0b52e4700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa0b0100fc0 0x7fa0b0199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:39.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.489+0000 7fa0b5ae5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0b0107000 0x7fa0b01a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56634/0 (socket says 172.21.15.71:56634) 2024-03-04T08:56:39.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.489+0000 7fa0b4ae3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0b0101960 0x7fa0b0199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:39.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.489+0000 7fa0b52e4700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa0b0100fc0 0x7fa0b0199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.71:35758/0 (socket says 172.21.15.71:35758) 2024-03-04T08:56:39.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.489+0000 7fa0b5ae5700 1 -- 172.21.15.71:0/1520826073 learned_addr learned my addr 172.21.15.71:0/1520826073 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:39.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.489+0000 7fa0b5ae5700 1 -- 172.21.15.71:0/1520826073 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa0b0100fc0 msgr2=0x7fa0b0199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:39.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.489+0000 7fa0b5ae5700 1 --2- 172.21.15.71:0/1520826073 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa0b0100fc0 0x7fa0b0199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:39.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.489+0000 7fa0b5ae5700 1 -- 172.21.15.71:0/1520826073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0b0101960 msgr2=0x7fa0b0199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:39.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.489+0000 7fa0b5ae5700 1 --2- 172.21.15.71:0/1520826073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0b0101960 0x7fa0b0199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:39.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.490+0000 7fa0b5ae5700 1 -- 172.21.15.71:0/1520826073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b01a4c40 con 0x7fa0b0107000 2024-03-04T08:56:39.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.490+0000 7fa0b5ae5700 1 --2- 172.21.15.71:0/1520826073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0b0107000 0x7fa0b01a4700 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800d980 tx=0x7fa0a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:39.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.490+0000 7fa0a67fc700 1 -- 172.21.15.71:0/1520826073 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8017070 con 0x7fa0b0107000 2024-03-04T08:56:39.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.490+0000 7fa0a67fc700 1 -- 172.21.15.71:0/1520826073 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0a8005020 con 0x7fa0b0107000 2024-03-04T08:56:39.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.490+0000 7fa0b7548700 1 -- 172.21.15.71:0/1520826073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b01a4ed0 con 0x7fa0b0107000 2024-03-04T08:56:39.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.490+0000 7fa0a67fc700 1 -- 172.21.15.71:0/1520826073 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a800fbf0 con 0x7fa0b0107000 2024-03-04T08:56:39.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.490+0000 7fa0b7548700 1 -- 172.21.15.71:0/1520826073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b01a5330 con 0x7fa0b0107000 2024-03-04T08:56:39.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.491+0000 7fa0b7548700 1 -- 172.21.15.71:0/1520826073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b0068d80 con 0x7fa0b0107000 2024-03-04T08:56:39.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.493+0000 7fa0a67fc700 1 -- 172.21.15.71:0/1520826073 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fa0a80051d0 con 0x7fa0b0107000 2024-03-04T08:56:39.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.493+0000 7fa0a67fc700 1 --2- 172.21.15.71:0/1520826073 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa09c0a1e10 0x7fa09c0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:39.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.493+0000 7fa0b52e4700 1 --2- 172.21.15.71:0/1520826073 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa09c0a1e10 0x7fa09c0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:39.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.493+0000 7fa0a67fc700 1 -- 172.21.15.71:0/1520826073 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fa0a80ce000 con 0x7fa0b0107000 2024-03-04T08:56:39.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.494+0000 7fa0b52e4700 1 --2- 172.21.15.71:0/1520826073 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa09c0a1e10 0x7fa09c0a42d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00f540 tx=0x7fa0ac00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:39.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.497+0000 7fa0a67fc700 1 -- 172.21.15.71:0/1520826073 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa0a8020020 con 0x7fa0b0107000 2024-03-04T08:56:39.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.737+0000 7fa0b7548700 1 -- 172.21.15.71:0/1520826073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa0b002d9b0 con 0x7fa0b0107000 2024-03-04T08:56:39.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.737+0000 7fa0a67fc700 1 -- 172.21.15.71:0/1520826073 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7fa0a809b1f0 con 0x7fa0b0107000 2024-03-04T08:56:39.735 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-04T08:56:39.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.739+0000 7fa0b7548700 1 -- 172.21.15.71:0/1520826073 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa09c0a1e10 msgr2=0x7fa09c0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:39.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.739+0000 7fa0b7548700 1 --2- 172.21.15.71:0/1520826073 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa09c0a1e10 0x7fa09c0a42d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00f540 tx=0x7fa0ac00b040 comp rx=0 tx=0).stop 2024-03-04T08:56:39.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.739+0000 7fa0b7548700 1 -- 172.21.15.71:0/1520826073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0b0107000 msgr2=0x7fa0b01a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:39.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.739+0000 7fa0b7548700 1 --2- 172.21.15.71:0/1520826073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0b0107000 0x7fa0b01a4700 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800d980 tx=0x7fa0a800dc90 comp rx=0 tx=0).stop 2024-03-04T08:56:39.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.739+0000 7fa0b7548700 1 -- 172.21.15.71:0/1520826073 shutdown_connections 2024-03-04T08:56:39.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.739+0000 7fa0b7548700 1 --2- 172.21.15.71:0/1520826073 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa09c0a1e10 0x7fa09c0a42d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:39.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.739+0000 7fa0b7548700 1 --2- 172.21.15.71:0/1520826073 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa0b0100fc0 0x7fa0b0199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:39.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.739+0000 7fa0b7548700 1 --2- 172.21.15.71:0/1520826073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0b0107000 0x7fa0b01a4700 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:39.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.739+0000 7fa0b7548700 1 --2- 172.21.15.71:0/1520826073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0b0101960 0x7fa0b0199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:39.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.739+0000 7fa0b7548700 1 -- 172.21.15.71:0/1520826073 >> 172.21.15.71:0/1520826073 conn(0x7fa0b0077ef0 msgr2=0x7fa0b00ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:39.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.739+0000 7fa0b7548700 1 -- 172.21.15.71:0/1520826073 shutdown_connections 2024-03-04T08:56:39.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:39.739+0000 7fa0b7548700 1 -- 172.21.15.71:0/1520826073 wait complete. 2024-03-04T08:56:39.747 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:39 smithi071 ceph-mon[32789]: pgmap v329: 129 pgs: 124 active+clean, 5 creating+peering; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-03-04T08:56:40.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:39 smithi150 ceph-mon[36807]: pgmap v329: 129 pgs: 124 active+clean, 5 creating+peering; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-03-04T08:56:40.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:39 smithi158 ceph-mon[37241]: pgmap v329: 129 pgs: 124 active+clean, 5 creating+peering; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-03-04T08:56:40.560 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-04T08:56:40.561 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-04T08:56:40.561 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2024-03-04T08:56:40.561 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-04T08:56:40.561 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-04T08:56:40.561 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-04T08:56:40.561 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-04T08:56:40.561 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-04T08:56:40.561 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-04T08:56:40.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:56:40.562 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2024-03-04T08:56:40.578 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-04T08:56:40.578 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-04T08:56:40.578 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2024-03-04T08:56:40.578 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-03-04T08:56:40.578 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2024-03-04T08:56:40.578 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-03-04T08:56:40.578 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T08:56:40.578 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2024-03-04T08:56:40.578 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2024-03-04T08:56:40.579 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T08:56:40.579 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2024-03-04T08:56:40.579 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2024-03-04T08:56:40.579 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T08:56:40.579 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2024-03-04T08:56:40.579 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-04T08:56:40.579 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2024-03-04T08:56:40.579 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-04T08:56:40.579 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 42113sec preferred_lft 42113sec 2024-03-04T08:56:40.579 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link 2024-03-04T08:56:40.579 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-03-04T08:56:40.580 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-04T08:56:40.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:56:40.580 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2024-03-04T08:56:40.580 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-03-04T08:56:40.580 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link add name ceph-brx type bridge 2024-03-04T08:56:40.580 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip addr flush dev ceph-brx 2024-03-04T08:56:40.580 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set ceph-brx up 2024-03-04T08:56:40.580 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-04T08:56:40.580 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-03-04T08:56:40.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:56:40.742 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-04T08:56:40.770 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-03-04T08:56:40.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:56:40.772 DEBUG:teuthology.orchestra.run.smithi071:> route 2024-03-04T08:56:40.849 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2024-03-04T08:56:40.849 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-04T08:56:40.849 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-04T08:56:40.850 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-04T08:56:40.850 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-04T08:56:40.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:56:40.851 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2024-03-04T08:56:40.851 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-03-04T08:56:40.851 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-04T08:56:40.851 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-04T08:56:40.851 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-04T08:56:40.851 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-03-04T08:56:40.854 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:40 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1520826073' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-04T08:56:41.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:40 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1520826073' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-04T08:56:41.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:40 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1520826073' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-04T08:56:41.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:56:41.097 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-03-04T08:56:41.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:56:41.112 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list-id 2024-03-04T08:56:41.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:56:41.168 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2024-03-04T08:56:41.168 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-03-04T08:56:41.168 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-04T08:56:41.168 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-04T08:56:41.168 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-03-04T08:56:41.252 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-04T08:56:41.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:56:41.252 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2024-03-04T08:56:41.253 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-03-04T08:56:41.253 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-04T08:56:41.253 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-04T08:56:41.253 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-04T08:56:41.253 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-04T08:56:41.253 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-04T08:56:41.253 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-03-04T08:56:41.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:56:41.353 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2024-03-04T08:56:41.353 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-03-04T08:56:41.353 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2024-03-04T08:56:41.353 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set dev brx.0 master ceph-brx 2024-03-04T08:56:41.353 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-03-04T08:56:41.395 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-04T08:56:41.395 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T08:56:41.395 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-04T08:56:41.449 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-04T08:56:41.450 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T08:56:41.450 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-04T08:56:41.504 DEBUG:teuthology.orchestra.run.smithi071:> sudo modprobe fuse 2024-03-04T08:56:41.573 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2024-03-04T08:56:41.628 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2024-03-04T08:56:41.628 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2024-03-04T08:56:41.628 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2024-03-04T08:56:41.628 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2024-03-04T08:56:41.629 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2024-03-04T08:56:41.629 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2024-03-04T08:56:41.629 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2024-03-04T08:56:41.629 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2024-03-04T08:56:41.629 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2024-03-04T08:56:41.629 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2024-03-04T08:56:41.629 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2024-03-04T08:56:41.629 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/selinux 2024-03-04T08:56:41.629 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2024-03-04T08:56:41.629 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2024-03-04T08:56:41.629 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2024-03-04T08:56:41.629 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2024-03-04T08:56:41.629 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2024-03-04T08:56:41.630 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-04T08:56:41.630 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2024-03-04T08:56:41.630 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2024-03-04T08:56:41.630 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysctl.service 2024-03-04T08:56:41.630 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-04T08:56:41.630 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/nfs/rpc_pipefs 2024-03-04T08:56:41.630 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2024-03-04T08:56:41.630 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2024-03-04T08:56:41.630 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay 2024-03-04T08:56:41.630 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/87b35563918bb354b9ac25973b7171eb98d5a3000e4a03d6ac592655362bb4c4/merged 2024-03-04T08:56:41.630 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/cc24157c6f7558520ec2743dbd2f9b42ff18f330502ca969b6807a79a12fa94c/merged 2024-03-04T08:56:41.630 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/0 2024-03-04T08:56:41.631 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2024-03-04T08:56:41.631 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/fff35c553fbf5033c0244251bada67687ea5ab3a10db283d64cfd5d17e769d52/merged 2024-03-04T08:56:41.631 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/d0cacd189ad81cf24f226dd2d4b8e68bafcfd92bb8da9f70e8c83d3992a202db/merged 2024-03-04T08:56:41.631 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/ffb29f5234f998e679cfeb05a2e44667bbfc466af57fd330d4895f3b31916cee/merged 2024-03-04T08:56:41.631 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/ae0c93d47c392366d6c024f95dfefec915afbb2b23f5011557e17d9dd0d8a813/merged 2024-03-04T08:56:41.631 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/8c7c42ac3a07f1192c2d7a391a1ff25d45ed24d217ee0896951b24d40996613b/merged 2024-03-04T08:56:41.631 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/2510c4ccc008e3f5f03db2a1240bdd03adf243232cad498fb50969db10ea463c/merged 2024-03-04T08:56:41.631 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/6a7aec9b6e413b4b5b2f4108277758f14ba49deacc975d7574b1cd613d0b53d8/merged 2024-03-04T08:56:41.631 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/81cfd40680b6e08ea1f9eb1ff95887d001dcd66814c74fc79a82fcddacc82445/merged 2024-03-04T08:56:41.631 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2024-03-04T08:56:41.632 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-04T08:56:41.632 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-04T08:56:41.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:56:41.632 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2024-03-04T08:56:41.683 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-04T08:56:41.683 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-04T08:56:41.726 DEBUG:teuthology.orchestra.run.smithi071:> sudo modprobe fuse 2024-03-04T08:56:41.752 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2024-03-04T08:56:41.808 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2024-03-04T08:56:41.809 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2024-03-04T08:56:41.809 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2024-03-04T08:56:41.809 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2024-03-04T08:56:41.809 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2024-03-04T08:56:41.809 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2024-03-04T08:56:41.809 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2024-03-04T08:56:41.809 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2024-03-04T08:56:41.809 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2024-03-04T08:56:41.809 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2024-03-04T08:56:41.809 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2024-03-04T08:56:41.809 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/selinux 2024-03-04T08:56:41.809 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2024-03-04T08:56:41.810 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2024-03-04T08:56:41.810 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2024-03-04T08:56:41.810 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2024-03-04T08:56:41.810 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2024-03-04T08:56:41.810 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-04T08:56:41.810 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2024-03-04T08:56:41.810 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2024-03-04T08:56:41.810 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysctl.service 2024-03-04T08:56:41.810 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-04T08:56:41.810 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/nfs/rpc_pipefs 2024-03-04T08:56:41.810 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2024-03-04T08:56:41.810 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2024-03-04T08:56:41.810 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay 2024-03-04T08:56:41.810 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/87b35563918bb354b9ac25973b7171eb98d5a3000e4a03d6ac592655362bb4c4/merged 2024-03-04T08:56:41.811 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/cc24157c6f7558520ec2743dbd2f9b42ff18f330502ca969b6807a79a12fa94c/merged 2024-03-04T08:56:41.811 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/0 2024-03-04T08:56:41.811 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2024-03-04T08:56:41.811 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/fff35c553fbf5033c0244251bada67687ea5ab3a10db283d64cfd5d17e769d52/merged 2024-03-04T08:56:41.811 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/d0cacd189ad81cf24f226dd2d4b8e68bafcfd92bb8da9f70e8c83d3992a202db/merged 2024-03-04T08:56:41.811 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/ffb29f5234f998e679cfeb05a2e44667bbfc466af57fd330d4895f3b31916cee/merged 2024-03-04T08:56:41.811 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/ae0c93d47c392366d6c024f95dfefec915afbb2b23f5011557e17d9dd0d8a813/merged 2024-03-04T08:56:41.811 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/8c7c42ac3a07f1192c2d7a391a1ff25d45ed24d217ee0896951b24d40996613b/merged 2024-03-04T08:56:41.811 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/2510c4ccc008e3f5f03db2a1240bdd03adf243232cad498fb50969db10ea463c/merged 2024-03-04T08:56:41.811 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/6a7aec9b6e413b4b5b2f4108277758f14ba49deacc975d7574b1cd613d0b53d8/merged 2024-03-04T08:56:41.811 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/81cfd40680b6e08ea1f9eb1ff95887d001dcd66814c74fc79a82fcddacc82445/merged 2024-03-04T08:56:41.811 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2024-03-04T08:56:41.811 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-04T08:56:41.811 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-04T08:56:41.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:56:41.812 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2024-03-04T08:56:41.823 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi071.stderr:2024-03-04T08:56:41.826+0000 7fe65b83e480 -1 init, newargv = 0x55c1c60a2bc0 newargc=15 2024-03-04T08:56:41.823 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi071.stderr:ceph-fuse[76854]: starting ceph client 2024-03-04T08:56:41.830 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi071.stderr:ceph-fuse[76854]: starting fuse 2024-03-04T08:56:41.838 INFO:teuthology.orchestra.run.smithi071.stdout:119 2024-03-04T08:56:41.838 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-03-04T08:56:41.839 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer -- python3 -c ' 2024-03-04T08:56:41.839 DEBUG:teuthology.orchestra.run.smithi071:> import glob 2024-03-04T08:56:41.839 DEBUG:teuthology.orchestra.run.smithi071:> import re 2024-03-04T08:56:41.839 DEBUG:teuthology.orchestra.run.smithi071:> import os 2024-03-04T08:56:41.839 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2024-03-04T08:56:41.839 DEBUG:teuthology.orchestra.run.smithi071:> 2024-03-04T08:56:41.839 DEBUG:teuthology.orchestra.run.smithi071:> def _find_admin_socket(client_name): 2024-03-04T08:56:41.839 DEBUG:teuthology.orchestra.run.smithi071:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-04T08:56:41.839 DEBUG:teuthology.orchestra.run.smithi071:> files = glob.glob(asok_path) 2024-03-04T08:56:41.839 DEBUG:teuthology.orchestra.run.smithi071:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-04T08:56:41.839 DEBUG:teuthology.orchestra.run.smithi071:> 2024-03-04T08:56:41.839 DEBUG:teuthology.orchestra.run.smithi071:> # Given a non-glob path, it better be there 2024-03-04T08:56:41.839 DEBUG:teuthology.orchestra.run.smithi071:> if "*" not in asok_path: 2024-03-04T08:56:41.839 DEBUG:teuthology.orchestra.run.smithi071:> assert(len(files) == 1) 2024-03-04T08:56:41.839 DEBUG:teuthology.orchestra.run.smithi071:> return files[0] 2024-03-04T08:56:41.840 DEBUG:teuthology.orchestra.run.smithi071:> 2024-03-04T08:56:41.840 DEBUG:teuthology.orchestra.run.smithi071:> for f in files: 2024-03-04T08:56:41.840 DEBUG:teuthology.orchestra.run.smithi071:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-04T08:56:41.840 DEBUG:teuthology.orchestra.run.smithi071:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-04T08:56:41.840 DEBUG:teuthology.orchestra.run.smithi071:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-04T08:56:41.840 DEBUG:teuthology.orchestra.run.smithi071:> contents = proc_f.read() 2024-03-04T08:56:41.840 DEBUG:teuthology.orchestra.run.smithi071:> if mountpoint in contents: 2024-03-04T08:56:41.840 DEBUG:teuthology.orchestra.run.smithi071:> return f 2024-03-04T08:56:41.840 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-04T08:56:41.840 DEBUG:teuthology.orchestra.run.smithi071:> 2024-03-04T08:56:41.840 DEBUG:teuthology.orchestra.run.smithi071:> print(_find_admin_socket("client.admin")) 2024-03-04T08:56:41.840 DEBUG:teuthology.orchestra.run.smithi071:> ' 2024-03-04T08:56:41.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:41 smithi071 ceph-mon[32789]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-03-04T08:56:41.959 INFO:teuthology.orchestra.run.smithi071.stdout:/var/run/ceph/ceph-client.admin.76854.asok 2024-03-04T08:56:41.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-04T08:56:41.968 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.76854.asok 2024-03-04T08:56:41.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:56:41.969 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.76854.asok status 2024-03-04T08:56:42.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:41 smithi150 ceph-mon[36807]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-03-04T08:56:42.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:41 smithi158 ceph-mon[37241]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-03-04T08:56:42.105 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-03-04T08:56:42.105 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": { 2024-03-04T08:56:42.105 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_sha1": "1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd", 2024-03-04T08:56:42.105 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 19.0.0-1739-g1bc250c7 (1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd) squid (dev)", 2024-03-04T08:56:42.105 INFO:teuthology.orchestra.run.smithi071.stdout: "entity_id": "admin", 2024-03-04T08:56:42.105 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": "smithi071", 2024-03-04T08:56:42.105 INFO:teuthology.orchestra.run.smithi071.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-04T08:56:42.105 INFO:teuthology.orchestra.run.smithi071.stdout: "pid": "76854", 2024-03-04T08:56:42.105 INFO:teuthology.orchestra.run.smithi071.stdout: "root": "/" 2024-03-04T08:56:42.105 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T08:56:42.105 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_count": 0, 2024-03-04T08:56:42.105 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_pinned_count": 0, 2024-03-04T08:56:42.106 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 14781, 2024-03-04T08:56:42.106 INFO:teuthology.orchestra.run.smithi071.stdout: "inst": { 2024-03-04T08:56:42.106 INFO:teuthology.orchestra.run.smithi071.stdout: "name": { 2024-03-04T08:56:42.106 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "client", 2024-03-04T08:56:42.106 INFO:teuthology.orchestra.run.smithi071.stdout: "num": 14781 2024-03-04T08:56:42.106 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T08:56:42.106 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2024-03-04T08:56:42.106 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-03-04T08:56:42.106 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2024-03-04T08:56:42.106 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3644963737 2024-03-04T08:56:42.106 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T08:56:42.106 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T08:56:42.106 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2024-03-04T08:56:42.106 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-03-04T08:56:42.107 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2024-03-04T08:56:42.107 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3644963737 2024-03-04T08:56:42.107 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T08:56:42.107 INFO:teuthology.orchestra.run.smithi071.stdout: "inst_str": "client.14781 192.168.0.1:0/3644963737", 2024-03-04T08:56:42.107 INFO:teuthology.orchestra.run.smithi071.stdout: "addr_str": "192.168.0.1:0/3644963737", 2024-03-04T08:56:42.107 INFO:teuthology.orchestra.run.smithi071.stdout: "inode_count": 1, 2024-03-04T08:56:42.107 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_epoch": 17, 2024-03-04T08:56:42.107 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch": 91, 2024-03-04T08:56:42.107 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch_barrier": 0, 2024-03-04T08:56:42.107 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklisted": false, 2024-03-04T08:56:42.107 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs" 2024-03-04T08:56:42.107 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-03-04T08:56:42.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:56:42.113 DEBUG:teuthology.orchestra.run.smithi071:> stat --file-system '--printf=%T 2024-03-04T08:56:42.113 DEBUG:teuthology.orchestra.run.smithi071:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-04T08:56:42.128 INFO:teuthology.orchestra.run.smithi071.stdout:fuseblk 2024-03-04T08:56:42.128 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-04T08:56:42.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:56:42.129 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-04T08:56:42.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:56:42.197 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-04T08:56:42.273 INFO:teuthology.orchestra.run.smithi071.stdout:# file: . 2024-03-04T08:56:42.273 INFO:teuthology.orchestra.run.smithi071.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-04T08:56:42.273 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:42.273 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-04T08:56:42.274 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T08:56:42.329 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T08:56:42.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.473+0000 7fc276719640 1 -- 172.21.15.71:0/3040535858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc270107350 msgr2=0x7fc270109740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:42.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.473+0000 7fc276719640 1 --2- 172.21.15.71:0/3040535858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc270107350 0x7fc270109740 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc268009ca0 tx=0x7fc26802c0f0 comp rx=0 tx=0).stop 2024-03-04T08:56:42.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.473+0000 7fc276719640 1 -- 172.21.15.71:0/3040535858 shutdown_connections 2024-03-04T08:56:42.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.473+0000 7fc276719640 1 --2- 172.21.15.71:0/3040535858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc270107350 0x7fc270109740 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:42.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.473+0000 7fc276719640 1 --2- 172.21.15.71:0/3040535858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc270104a20 0x7fc270106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:42.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.473+0000 7fc276719640 1 --2- 172.21.15.71:0/3040535858 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2701020f0 0x7fc2701044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:42.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.473+0000 7fc276719640 1 -- 172.21.15.71:0/3040535858 >> 172.21.15.71:0/3040535858 conn(0x7fc2700fba60 msgr2=0x7fc2700fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:42.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.473+0000 7fc276719640 1 -- 172.21.15.71:0/3040535858 shutdown_connections 2024-03-04T08:56:42.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.473+0000 7fc276719640 1 -- 172.21.15.71:0/3040535858 wait complete. 2024-03-04T08:56:42.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.474+0000 7fc276719640 1 Processor -- start 2024-03-04T08:56:42.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.474+0000 7fc276719640 1 -- start start 2024-03-04T08:56:42.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.474+0000 7fc276719640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2701020f0 0x7fc2701978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:42.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.474+0000 7fc276719640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc270104a20 0x7fc270197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:42.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.474+0000 7fc276719640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc270107350 0x7fc27019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:42.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.474+0000 7fc276719640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27010d280 con 0x7fc270104a20 2024-03-04T08:56:42.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.474+0000 7fc276719640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27010d100 con 0x7fc270107350 2024-03-04T08:56:42.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.474+0000 7fc276719640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27010d400 con 0x7fc2701020f0 2024-03-04T08:56:42.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.474+0000 7fc26f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc270104a20 0x7fc270197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:42.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.474+0000 7fc26f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc270104a20 0x7fc270197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56650/0 (socket says 172.21.15.71:56650) 2024-03-04T08:56:42.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.474+0000 7fc26f7fe640 1 -- 172.21.15.71:0/2878496835 learned_addr learned my addr 172.21.15.71:0/2878496835 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:42.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.474+0000 7fc274cb6640 1 --2- 172.21.15.71:0/2878496835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc270107350 0x7fc27019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:42.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.475+0000 7fc26ffff640 1 --2- 172.21.15.71:0/2878496835 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2701020f0 0x7fc2701978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:42.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.475+0000 7fc26f7fe640 1 -- 172.21.15.71:0/2878496835 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2701020f0 msgr2=0x7fc2701978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:42.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.475+0000 7fc26f7fe640 1 --2- 172.21.15.71:0/2878496835 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2701020f0 0x7fc2701978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:42.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.475+0000 7fc26f7fe640 1 -- 172.21.15.71:0/2878496835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc270107350 msgr2=0x7fc27019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:42.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.475+0000 7fc26f7fe640 1 --2- 172.21.15.71:0/2878496835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc270107350 0x7fc27019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:42.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.475+0000 7fc26f7fe640 1 -- 172.21.15.71:0/2878496835 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27019f510 con 0x7fc270104a20 2024-03-04T08:56:42.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.475+0000 7fc26f7fe640 1 --2- 172.21.15.71:0/2878496835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc270104a20 0x7fc270197e30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc26400e9c0 tx=0x7fc26400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:42.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.475+0000 7fc26d7fa640 1 -- 172.21.15.71:0/2878496835 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26400cd80 con 0x7fc270104a20 2024-03-04T08:56:42.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.475+0000 7fc26d7fa640 1 -- 172.21.15.71:0/2878496835 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc264005500 con 0x7fc270104a20 2024-03-04T08:56:42.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.475+0000 7fc276719640 1 -- 172.21.15.71:0/2878496835 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27019f800 con 0x7fc270104a20 2024-03-04T08:56:42.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.475+0000 7fc26d7fa640 1 -- 172.21.15.71:0/2878496835 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264010690 con 0x7fc270104a20 2024-03-04T08:56:42.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.475+0000 7fc276719640 1 -- 172.21.15.71:0/2878496835 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27019fd50 con 0x7fc270104a20 2024-03-04T08:56:42.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.476+0000 7fc276719640 1 -- 172.21.15.71:0/2878496835 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2700fd550 con 0x7fc270104a20 2024-03-04T08:56:42.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.477+0000 7fc26d7fa640 1 -- 172.21.15.71:0/2878496835 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fc2640046c0 con 0x7fc270104a20 2024-03-04T08:56:42.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.478+0000 7fc26d7fa640 1 --2- 172.21.15.71:0/2878496835 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc2580a1d80 0x7fc2580a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:42.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.478+0000 7fc26ffff640 1 --2- 172.21.15.71:0/2878496835 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc2580a1d80 0x7fc2580a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:42.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.478+0000 7fc26d7fa640 1 -- 172.21.15.71:0/2878496835 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fc2640ce540 con 0x7fc270104a20 2024-03-04T08:56:42.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.478+0000 7fc26ffff640 1 --2- 172.21.15.71:0/2878496835 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc2580a1d80 0x7fc2580a4240 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc260005fd0 tx=0x7fc26000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:42.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.482+0000 7fc26d7fa640 1 -- 172.21.15.71:0/2878496835 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc264053280 con 0x7fc270104a20 2024-03-04T08:56:42.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.657+0000 7fc276719640 1 -- 172.21.15.71:0/2878496835 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc27002d240 con 0x7fc270104a20 2024-03-04T08:56:42.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.660+0000 7fc26d7fa640 1 -- 172.21.15.71:0/2878496835 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24153 (secure 0 0 0) 0x7fc26409b730 con 0x7fc270104a20 2024-03-04T08:56:42.656 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:42.657 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":91,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","created":"2024-03-04T08:47:28.546102+0000","modified":"2024-03-04T08:56:35.007858+0000","last_up_change":"2024-03-04T08:55:30.744961+0000","last_in_change":"2024-03-04T08:55:11.570990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T08:51:14.933487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T08:56:28.253074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T08:56:29.543622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T08:56:32.884220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7e43b47d-8888-4631-bc96-8a96fcf8cd46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6803","nonce":1173641530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6805","nonce":1173641530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6809","nonce":1173641530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6807","nonce":1173641530}]},"public_addr":"172.21.15.71:6803/1173641530","cluster_addr":"172.21.15.71:6805/1173641530","heartbeat_back_addr":"172.21.15.71:6809/1173641530","heartbeat_front_addr":"172.21.15.71:6807/1173641530","state":["exists","up"]},{"osd":1,"uuid":"1209f214-f2e2-4209-bf69-4528a43a2123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6801","nonce":3383948110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6803","nonce":3383948110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6807","nonce":3383948110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6805","nonce":3383948110}]},"public_addr":"172.21.15.150:6801/3383948110","cluster_addr":"172.21.15.150:6803/3383948110","heartbeat_back_addr":"172.21.15.150:6807/3383948110","heartbeat_front_addr":"172.21.15.150:6805/3383948110","state":["exists","up"]},{"osd":2,"uuid":"b88c3082-93a9-4c9b-9221-a7105a44c91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6801","nonce":586944144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6803","nonce":586944144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6807","nonce":586944144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6805","nonce":586944144}]},"public_addr":"172.21.15.158:6801/586944144","cluster_addr":"172.21.15.158:6803/586944144","heartbeat_back_addr":"172.21.15.158:6807/586944144","heartbeat_front_addr":"172.21.15.158:6805/586944144","state":["exists","up"]},{"osd":3,"uuid":"e4fc7723-fd73-40b2-b9d0-7f9bda282ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6811","nonce":1311903262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6813","nonce":1311903262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6817","nonce":1311903262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6815","nonce":1311903262}]},"public_addr":"172.21.15.71:6811/1311903262","cluster_addr":"172.21.15.71:6813/1311903262","heartbeat_back_addr":"172.21.15.71:6817/1311903262","heartbeat_front_addr":"172.21.15.71:6815/1311903262","state":["exists","up"]},{"osd":4,"uuid":"e7eb72ac-f066-4ccd-b2bf-22a6a26daf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6809","nonce":2300837573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6811","nonce":2300837573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6815","nonce":2300837573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6813","nonce":2300837573}]},"public_addr":"172.21.15.150:6809/2300837573","cluster_addr":"172.21.15.150:6811/2300837573","heartbeat_back_addr":"172.21.15.150:6815/2300837573","heartbeat_front_addr":"172.21.15.150:6813/2300837573","state":["exists","up"]},{"osd":5,"uuid":"9b295840-e104-4a66-9cc4-7020627c1765","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6809","nonce":1922694045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6811","nonce":1922694045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6815","nonce":1922694045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6813","nonce":1922694045}]},"public_addr":"172.21.15.158:6809/1922694045","cluster_addr":"172.21.15.158:6811/1922694045","heartbeat_back_addr":"172.21.15.158:6815/1922694045","heartbeat_front_addr":"172.21.15.158:6813/1922694045","state":["exists","up"]},{"osd":6,"uuid":"6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6819","nonce":3848842707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6821","nonce":3848842707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6825","nonce":3848842707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6823","nonce":3848842707}]},"public_addr":"172.21.15.71:6819/3848842707","cluster_addr":"172.21.15.71:6821/3848842707","heartbeat_back_addr":"172.21.15.71:6825/3848842707","heartbeat_front_addr":"172.21.15.71:6823/3848842707","state":["exists","up"]},{"osd":7,"uuid":"6160a40a-d30f-4569-949c-0afb1bb3d133","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6817","nonce":1328330305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6819","nonce":1328330305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6823","nonce":1328330305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6821","nonce":1328330305}]},"public_addr":"172.21.15.150:6817/1328330305","cluster_addr":"172.21.15.150:6819/1328330305","heartbeat_back_addr":"172.21.15.150:6823/1328330305","heartbeat_front_addr":"172.21.15.150:6821/1328330305","state":["exists","up"]},{"osd":8,"uuid":"95ed2341-84a4-465c-9948-ac667680ce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6817","nonce":2195304577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6819","nonce":2195304577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6823","nonce":2195304577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6821","nonce":2195304577}]},"public_addr":"172.21.15.158:6817/2195304577","cluster_addr":"172.21.15.158:6819/2195304577","heartbeat_back_addr":"172.21.15.158:6823/2195304577","heartbeat_front_addr":"172.21.15.158:6821/2195304577","state":["exists","up"]},{"osd":9,"uuid":"088f6419-63a0-4768-afae-dbb8a5f5a565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6827","nonce":1913779347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6829","nonce":1913779347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6833","nonce":1913779347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6831","nonce":1913779347}]},"public_addr":"172.21.15.71:6827/1913779347","cluster_addr":"172.21.15.71:6829/1913779347","heartbeat_back_addr":"172.21.15.71:6833/1913779347","heartbeat_front_addr":"172.21.15.71:6831/1913779347","state":["exists","up"]},{"osd":10,"uuid":"18c85e24-068d-4f1e-a3b8-f7915b363234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6825","nonce":3295420996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6827","nonce":3295420996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6831","nonce":3295420996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6829","nonce":3295420996}]},"public_addr":"172.21.15.150:6825/3295420996","cluster_addr":"172.21.15.150:6827/3295420996","heartbeat_back_addr":"172.21.15.150:6831/3295420996","heartbeat_front_addr":"172.21.15.150:6829/3295420996","state":["exists","up"]},{"osd":11,"uuid":"4f6b3ddf-c482-4740-b1fd-de836e047c2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6825","nonce":3332083865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6827","nonce":3332083865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6831","nonce":3332083865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6829","nonce":3332083865}]},"public_addr":"172.21.15.158:6825/3332083865","cluster_addr":"172.21.15.158:6827/3332083865","heartbeat_back_addr":"172.21.15.158:6831/3332083865","heartbeat_front_addr":"172.21.15.158:6829/3332083865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:05.997832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:29.314235+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:08.819465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:41.194275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:06.032479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:38.814498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:13.067960+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:38.568479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:03.416713+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:38.349479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:04.096945+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:29.425250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1287712358":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/3770452860":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/718743094":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/1904717532":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:6800/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6800/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/2862184313":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/3621754630":"2024-03-05T08:48:22.593931+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T08:56:42.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.661+0000 7fc276719640 1 -- 172.21.15.71:0/2878496835 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc2580a1d80 msgr2=0x7fc2580a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:42.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.661+0000 7fc276719640 1 --2- 172.21.15.71:0/2878496835 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc2580a1d80 0x7fc2580a4240 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc260005fd0 tx=0x7fc26000b390 comp rx=0 tx=0).stop 2024-03-04T08:56:42.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.661+0000 7fc276719640 1 -- 172.21.15.71:0/2878496835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc270104a20 msgr2=0x7fc270197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:42.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.661+0000 7fc276719640 1 --2- 172.21.15.71:0/2878496835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc270104a20 0x7fc270197e30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc26400e9c0 tx=0x7fc26400ee80 comp rx=0 tx=0).stop 2024-03-04T08:56:42.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.661+0000 7fc276719640 1 -- 172.21.15.71:0/2878496835 shutdown_connections 2024-03-04T08:56:42.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.661+0000 7fc276719640 1 --2- 172.21.15.71:0/2878496835 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc2580a1d80 0x7fc2580a4240 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:42.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.661+0000 7fc276719640 1 --2- 172.21.15.71:0/2878496835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc270107350 0x7fc27019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:42.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.661+0000 7fc276719640 1 --2- 172.21.15.71:0/2878496835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc270104a20 0x7fc270197e30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:42.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.661+0000 7fc276719640 1 --2- 172.21.15.71:0/2878496835 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2701020f0 0x7fc2701978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:42.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.661+0000 7fc276719640 1 -- 172.21.15.71:0/2878496835 >> 172.21.15.71:0/2878496835 conn(0x7fc2700fba60 msgr2=0x7fc270102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:42.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.661+0000 7fc276719640 1 -- 172.21.15.71:0/2878496835 shutdown_connections 2024-03-04T08:56:42.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.661+0000 7fc276719640 1 -- 172.21.15.71:0/2878496835 wait complete. 2024-03-04T08:56:42.669 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T08:51:14.933487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T08:56:28.253074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T08:56:29.543622+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T08:56:32.884220+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T08:56:42.670 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T08:56:42.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.778+0000 7f54a9bb6640 1 -- 172.21.15.71:0/3148289583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a41031a0 msgr2=0x7f54a4109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:42.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.778+0000 7f54a9bb6640 1 --2- 172.21.15.71:0/3148289583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a41031a0 0x7f54a4109a30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f549c009ca0 tx=0x7f549c02c0f0 comp rx=0 tx=0).stop 2024-03-04T08:56:42.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.778+0000 7f54a9bb6640 1 -- 172.21.15.71:0/3148289583 shutdown_connections 2024-03-04T08:56:42.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.778+0000 7f54a9bb6640 1 --2- 172.21.15.71:0/3148289583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a41031a0 0x7f54a4109a30 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:42.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.778+0000 7f54a9bb6640 1 --2- 172.21.15.71:0/3148289583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54a41027e0 0x7f54a4102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:42.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.778+0000 7f54a9bb6640 1 --2- 172.21.15.71:0/3148289583 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f54a41015f0 0x7f54a4101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:42.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.778+0000 7f54a9bb6640 1 -- 172.21.15.71:0/3148289583 >> 172.21.15.71:0/3148289583 conn(0x7f54a40fcb40 msgr2=0x7f54a40fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:42.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.778+0000 7f54a9bb6640 1 -- 172.21.15.71:0/3148289583 shutdown_connections 2024-03-04T08:56:42.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.778+0000 7f54a9bb6640 1 -- 172.21.15.71:0/3148289583 wait complete. 2024-03-04T08:56:42.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.778+0000 7f54a9bb6640 1 Processor -- start 2024-03-04T08:56:42.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.778+0000 7f54a9bb6640 1 -- start start 2024-03-04T08:56:42.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.779+0000 7f54a9bb6640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f54a41015f0 0x7f54a4197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:42.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.779+0000 7f54a9bb6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a41027e0 0x7f54a4197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:42.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.779+0000 7f54a9bb6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54a41031a0 0x7f54a419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:42.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.779+0000 7f54a9bb6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a410d250 con 0x7f54a41027e0 2024-03-04T08:56:42.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.779+0000 7f54a9bb6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a410d0d0 con 0x7f54a41031a0 2024-03-04T08:56:42.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.779+0000 7f54a9bb6640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a410d3d0 con 0x7f54a41015f0 2024-03-04T08:56:42.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.779+0000 7f549affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a41027e0 0x7f54a4197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:42.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.779+0000 7f549affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a41027e0 0x7f54a4197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56668/0 (socket says 172.21.15.71:56668) 2024-03-04T08:56:42.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.779+0000 7f549affd640 1 -- 172.21.15.71:0/841603082 learned_addr learned my addr 172.21.15.71:0/841603082 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:42.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.779+0000 7f54a3fff640 1 --2- 172.21.15.71:0/841603082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54a41031a0 0x7f54a419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:42.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.779+0000 7f54a37fe640 1 --2- 172.21.15.71:0/841603082 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f54a41015f0 0x7f54a4197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:42.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.779+0000 7f549affd640 1 -- 172.21.15.71:0/841603082 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f54a41015f0 msgr2=0x7f54a4197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:42.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.779+0000 7f549affd640 1 --2- 172.21.15.71:0/841603082 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f54a41015f0 0x7f54a4197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:42.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.779+0000 7f549affd640 1 -- 172.21.15.71:0/841603082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54a41031a0 msgr2=0x7f54a419ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:42.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.779+0000 7f549affd640 1 --2- 172.21.15.71:0/841603082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54a41031a0 0x7f54a419ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:42.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.779+0000 7f549affd640 1 -- 172.21.15.71:0/841603082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a419f480 con 0x7f54a41027e0 2024-03-04T08:56:42.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.779+0000 7f549affd640 1 --2- 172.21.15.71:0/841603082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a41027e0 0x7f54a4197da0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f548c00e9f0 tx=0x7f548c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:42.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.780+0000 7f54a17fa640 1 -- 172.21.15.71:0/841603082 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c00cd80 con 0x7f54a41027e0 2024-03-04T08:56:42.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.780+0000 7f54a17fa640 1 -- 172.21.15.71:0/841603082 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f548c010910 con 0x7f54a41027e0 2024-03-04T08:56:42.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.780+0000 7f54a9bb6640 1 -- 172.21.15.71:0/841603082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a419f770 con 0x7f54a41027e0 2024-03-04T08:56:42.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.780+0000 7f54a17fa640 1 -- 172.21.15.71:0/841603082 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c018990 con 0x7f54a41027e0 2024-03-04T08:56:42.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.780+0000 7f54a9bb6640 1 -- 172.21.15.71:0/841603082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a419fcc0 con 0x7f54a41027e0 2024-03-04T08:56:42.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.781+0000 7f54a9bb6640 1 -- 172.21.15.71:0/841603082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54a41088b0 con 0x7f54a41027e0 2024-03-04T08:56:42.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.782+0000 7f54a17fa640 1 -- 172.21.15.71:0/841603082 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f548c010ac0 con 0x7f54a41027e0 2024-03-04T08:56:42.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.782+0000 7f54a17fa640 1 --2- 172.21.15.71:0/841603082 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f54840a1d80 0x7f54840a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:42.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.782+0000 7f54a37fe640 1 --2- 172.21.15.71:0/841603082 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f54840a1d80 0x7f54840a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:42.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.782+0000 7f54a17fa640 1 -- 172.21.15.71:0/841603082 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f548c0ce3b0 con 0x7f54a41027e0 2024-03-04T08:56:42.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.783+0000 7f54a37fe640 1 --2- 172.21.15.71:0/841603082 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f54840a1d80 0x7f54840a4240 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5494005fd0 tx=0x7f5494005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:42.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.787+0000 7f54a17fa640 1 -- 172.21.15.71:0/841603082 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f548c09b5a0 con 0x7f54a41027e0 2024-03-04T08:56:42.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:42 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2878496835' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:42.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.964+0000 7f54a9bb6640 1 -- 172.21.15.71:0/841603082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f54a40621d0 con 0x7f54a41027e0 2024-03-04T08:56:42.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.965+0000 7f54a17fa640 1 -- 172.21.15.71:0/841603082 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f548c09aea0 con 0x7f54a41027e0 2024-03-04T08:56:42.961 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-03-04T08:56:42.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.967+0000 7f54a9bb6640 1 -- 172.21.15.71:0/841603082 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f54840a1d80 msgr2=0x7f54840a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:42.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.967+0000 7f54a9bb6640 1 --2- 172.21.15.71:0/841603082 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f54840a1d80 0x7f54840a4240 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5494005fd0 tx=0x7f5494005850 comp rx=0 tx=0).stop 2024-03-04T08:56:42.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.967+0000 7f54a9bb6640 1 -- 172.21.15.71:0/841603082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a41027e0 msgr2=0x7f54a4197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:42.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.967+0000 7f54a9bb6640 1 --2- 172.21.15.71:0/841603082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a41027e0 0x7f54a4197da0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f548c00e9f0 tx=0x7f548c00eeb0 comp rx=0 tx=0).stop 2024-03-04T08:56:42.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.967+0000 7f54a9bb6640 1 -- 172.21.15.71:0/841603082 shutdown_connections 2024-03-04T08:56:42.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.967+0000 7f54a9bb6640 1 --2- 172.21.15.71:0/841603082 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f54840a1d80 0x7f54840a4240 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:42.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.967+0000 7f54a9bb6640 1 --2- 172.21.15.71:0/841603082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54a41031a0 0x7f54a419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:42.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.967+0000 7f54a9bb6640 1 --2- 172.21.15.71:0/841603082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54a41027e0 0x7f54a4197da0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:42.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.967+0000 7f54a9bb6640 1 --2- 172.21.15.71:0/841603082 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f54a41015f0 0x7f54a4197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:42.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.967+0000 7f54a9bb6640 1 -- 172.21.15.71:0/841603082 >> 172.21.15.71:0/841603082 conn(0x7f54a40fcb40 msgr2=0x7f54a4105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:42.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.967+0000 7f54a9bb6640 1 -- 172.21.15.71:0/841603082 shutdown_connections 2024-03-04T08:56:42.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:42.967+0000 7f54a9bb6640 1 -- 172.21.15.71:0/841603082 wait complete. 2024-03-04T08:56:42.973 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T08:56:43.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:42 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2878496835' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:43.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:42 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2878496835' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:43.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.082+0000 7fe1077e9640 1 -- 172.21.15.71:0/1199745301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1001004d0 msgr2=0x7fe1001008f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:43.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.082+0000 7fe1077e9640 1 --2- 172.21.15.71:0/1199745301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1001004d0 0x7fe1001008f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009900 tx=0x7fe0fc02bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:43.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.082+0000 7fe1077e9640 1 -- 172.21.15.71:0/1199745301 shutdown_connections 2024-03-04T08:56:43.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.082+0000 7fe1077e9640 1 --2- 172.21.15.71:0/1199745301 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe100102080 0x7fe100108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.082+0000 7fe1077e9640 1 --2- 172.21.15.71:0/1199745301 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe1001016c0 0x7fe100101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.082+0000 7fe1077e9640 1 --2- 172.21.15.71:0/1199745301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1001004d0 0x7fe1001008f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.082+0000 7fe1077e9640 1 -- 172.21.15.71:0/1199745301 >> 172.21.15.71:0/1199745301 conn(0x7fe1000fba60 msgr2=0x7fe1000fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:43.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.082+0000 7fe1077e9640 1 -- 172.21.15.71:0/1199745301 shutdown_connections 2024-03-04T08:56:43.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.082+0000 7fe1077e9640 1 -- 172.21.15.71:0/1199745301 wait complete. 2024-03-04T08:56:43.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.083+0000 7fe1077e9640 1 Processor -- start 2024-03-04T08:56:43.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.083+0000 7fe1077e9640 1 -- start start 2024-03-04T08:56:43.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.083+0000 7fe1077e9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1001004d0 0x7fe1001967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:43.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.083+0000 7fe1077e9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1001016c0 0x7fe100196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:43.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.083+0000 7fe1077e9640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe100102080 0x7fe10019dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:43.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.083+0000 7fe1077e9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10010c160 con 0x7fe1001016c0 2024-03-04T08:56:43.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.083+0000 7fe1077e9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10010bfe0 con 0x7fe1001004d0 2024-03-04T08:56:43.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.083+0000 7fe1077e9640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10010c2e0 con 0x7fe100102080 2024-03-04T08:56:43.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.083+0000 7fe104d84640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1001016c0 0x7fe100196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:43.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.083+0000 7fe104d84640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1001016c0 0x7fe100196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56700/0 (socket says 172.21.15.71:56700) 2024-03-04T08:56:43.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.083+0000 7fe104d84640 1 -- 172.21.15.71:0/3107980408 learned_addr learned my addr 172.21.15.71:0/3107980408 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:43.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.083+0000 7fe105585640 1 --2- 172.21.15.71:0/3107980408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1001004d0 0x7fe1001967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:43.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.083+0000 7fe105d86640 1 --2- 172.21.15.71:0/3107980408 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe100102080 0x7fe10019dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:43.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.083+0000 7fe104d84640 1 -- 172.21.15.71:0/3107980408 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe100102080 msgr2=0x7fe10019dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:43.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.083+0000 7fe104d84640 1 --2- 172.21.15.71:0/3107980408 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe100102080 0x7fe10019dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.083+0000 7fe104d84640 1 -- 172.21.15.71:0/3107980408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1001004d0 msgr2=0x7fe1001967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:43.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.083+0000 7fe104d84640 1 --2- 172.21.15.71:0/3107980408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1001004d0 0x7fe1001967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.083+0000 7fe104d84640 1 -- 172.21.15.71:0/3107980408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe100068730 con 0x7fe1001016c0 2024-03-04T08:56:43.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.083+0000 7fe104d84640 1 --2- 172.21.15.71:0/3107980408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1001016c0 0x7fe100196d00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000c920 tx=0x7fe0f000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:43.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.084+0000 7fe0f67fc640 1 -- 172.21.15.71:0/3107980408 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0007c70 con 0x7fe1001016c0 2024-03-04T08:56:43.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.084+0000 7fe0f67fc640 1 -- 172.21.15.71:0/3107980408 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0f0007e10 con 0x7fe1001016c0 2024-03-04T08:56:43.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.084+0000 7fe1077e9640 1 -- 172.21.15.71:0/3107980408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe100068a20 con 0x7fe1001016c0 2024-03-04T08:56:43.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.084+0000 7fe0f67fc640 1 -- 172.21.15.71:0/3107980408 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0018890 con 0x7fe1001016c0 2024-03-04T08:56:43.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.084+0000 7fe1077e9640 1 -- 172.21.15.71:0/3107980408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe100068f70 con 0x7fe1001016c0 2024-03-04T08:56:43.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.085+0000 7fe1077e9640 1 -- 172.21.15.71:0/3107980408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1001077f0 con 0x7fe1001016c0 2024-03-04T08:56:43.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.086+0000 7fe0f67fc640 1 -- 172.21.15.71:0/3107980408 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fe0f0018a50 con 0x7fe1001016c0 2024-03-04T08:56:43.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.086+0000 7fe0f67fc640 1 --2- 172.21.15.71:0/3107980408 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe0ec0a1d80 0x7fe0ec0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:43.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.087+0000 7fe105585640 1 --2- 172.21.15.71:0/3107980408 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe0ec0a1d80 0x7fe0ec0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:43.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.087+0000 7fe0f67fc640 1 -- 172.21.15.71:0/3107980408 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe0f00ce3b0 con 0x7fe1001016c0 2024-03-04T08:56:43.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.087+0000 7fe105585640 1 --2- 172.21.15.71:0/3107980408 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe0ec0a1d80 0x7fe0ec0a4240 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc000c00 tx=0x7fe0fc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:43.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.090+0000 7fe0f67fc640 1 -- 172.21.15.71:0/3107980408 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe0f0014030 con 0x7fe1001016c0 2024-03-04T08:56:43.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.265+0000 7fe1077e9640 1 -- 172.21.15.71:0/3107980408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe10002d240 con 0x7fe1001016c0 2024-03-04T08:56:43.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.265+0000 7fe0f67fc640 1 -- 172.21.15.71:0/3107980408 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe0f009b630 con 0x7fe1001016c0 2024-03-04T08:56:43.262 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-03-04T08:56:43.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.267+0000 7fe1077e9640 1 -- 172.21.15.71:0/3107980408 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe0ec0a1d80 msgr2=0x7fe0ec0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:43.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.267+0000 7fe1077e9640 1 --2- 172.21.15.71:0/3107980408 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe0ec0a1d80 0x7fe0ec0a4240 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc000c00 tx=0x7fe0fc005fd0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.267+0000 7fe1077e9640 1 -- 172.21.15.71:0/3107980408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1001016c0 msgr2=0x7fe100196d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:43.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.267+0000 7fe1077e9640 1 --2- 172.21.15.71:0/3107980408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1001016c0 0x7fe100196d00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000c920 tx=0x7fe0f000cde0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.267+0000 7fe1077e9640 1 -- 172.21.15.71:0/3107980408 shutdown_connections 2024-03-04T08:56:43.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.267+0000 7fe1077e9640 1 --2- 172.21.15.71:0/3107980408 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe0ec0a1d80 0x7fe0ec0a4240 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.267+0000 7fe1077e9640 1 --2- 172.21.15.71:0/3107980408 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe100102080 0x7fe10019dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.267+0000 7fe1077e9640 1 --2- 172.21.15.71:0/3107980408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1001016c0 0x7fe100196d00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.267+0000 7fe1077e9640 1 --2- 172.21.15.71:0/3107980408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1001004d0 0x7fe1001967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.267+0000 7fe1077e9640 1 -- 172.21.15.71:0/3107980408 >> 172.21.15.71:0/3107980408 conn(0x7fe1000fba60 msgr2=0x7fe1001049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:43.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.267+0000 7fe1077e9640 1 -- 172.21.15.71:0/3107980408 shutdown_connections 2024-03-04T08:56:43.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.267+0000 7fe1077e9640 1 -- 172.21.15.71:0/3107980408 wait complete. 2024-03-04T08:56:43.274 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T08:56:43.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.383+0000 7f3aea49a640 1 -- 172.21.15.71:0/3076629762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae40ffc70 msgr2=0x7f3ae4100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:43.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.383+0000 7f3aea49a640 1 --2- 172.21.15.71:0/3076629762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae40ffc70 0x7f3ae4100090 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009900 tx=0x7f3ad402bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:43.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.383+0000 7f3aea49a640 1 -- 172.21.15.71:0/3076629762 shutdown_connections 2024-03-04T08:56:43.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.383+0000 7f3aea49a640 1 --2- 172.21.15.71:0/3076629762 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3ae4101820 0x7f3ae41080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.383+0000 7f3aea49a640 1 --2- 172.21.15.71:0/3076629762 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ae4100e60 0x7f3ae41012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.383+0000 7f3aea49a640 1 --2- 172.21.15.71:0/3076629762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae40ffc70 0x7f3ae4100090 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.383+0000 7f3aea49a640 1 -- 172.21.15.71:0/3076629762 >> 172.21.15.71:0/3076629762 conn(0x7f3ae40fb200 msgr2=0x7f3ae40fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:43.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.383+0000 7f3aea49a640 1 -- 172.21.15.71:0/3076629762 shutdown_connections 2024-03-04T08:56:43.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.383+0000 7f3aea49a640 1 -- 172.21.15.71:0/3076629762 wait complete. 2024-03-04T08:56:43.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.383+0000 7f3aea49a640 1 Processor -- start 2024-03-04T08:56:43.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.383+0000 7f3aea49a640 1 -- start start 2024-03-04T08:56:43.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.384+0000 7f3aea49a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ae40ffc70 0x7f3ae4196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:43.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.384+0000 7f3aea49a640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3ae4100e60 0x7f3ae4196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:43.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.384+0000 7f3aea49a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae4101820 0x7f3ae419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:43.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.384+0000 7f3aea49a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae410b8f0 con 0x7f3ae4101820 2024-03-04T08:56:43.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.384+0000 7f3aea49a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae410b770 con 0x7f3ae40ffc70 2024-03-04T08:56:43.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.384+0000 7f3aea49a640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae410ba70 con 0x7f3ae4100e60 2024-03-04T08:56:43.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.384+0000 7f3ae8a37640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae4101820 0x7f3ae419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:43.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.384+0000 7f3ae8a37640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae4101820 0x7f3ae419de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56718/0 (socket says 172.21.15.71:56718) 2024-03-04T08:56:43.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.384+0000 7f3ae8a37640 1 -- 172.21.15.71:0/2698868475 learned_addr learned my addr 172.21.15.71:0/2698868475 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:43.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.384+0000 7f3ae3fff640 1 --2- 172.21.15.71:0/2698868475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ae40ffc70 0x7f3ae4196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:43.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.384+0000 7f3ae37fe640 1 --2- 172.21.15.71:0/2698868475 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3ae4100e60 0x7f3ae4196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:43.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.384+0000 7f3ae8a37640 1 -- 172.21.15.71:0/2698868475 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3ae4100e60 msgr2=0x7f3ae4196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:43.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.384+0000 7f3ae8a37640 1 --2- 172.21.15.71:0/2698868475 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3ae4100e60 0x7f3ae4196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.384+0000 7f3ae8a37640 1 -- 172.21.15.71:0/2698868475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ae40ffc70 msgr2=0x7f3ae4196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:43.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.384+0000 7f3ae8a37640 1 --2- 172.21.15.71:0/2698868475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ae40ffc70 0x7f3ae4196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.384+0000 7f3ae8a37640 1 -- 172.21.15.71:0/2698868475 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae419e4a0 con 0x7f3ae4101820 2024-03-04T08:56:43.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.384+0000 7f3ae3fff640 1 --2- 172.21.15.71:0/2698868475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ae40ffc70 0x7f3ae4196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:56:43.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.384+0000 7f3ae8a37640 1 --2- 172.21.15.71:0/2698868475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae4101820 0x7f3ae419de30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00eaa0 tx=0x7f3adc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:43.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.384+0000 7f3ae17fa640 1 -- 172.21.15.71:0/2698868475 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc00cdb0 con 0x7f3ae4101820 2024-03-04T08:56:43.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.385+0000 7f3aea49a640 1 -- 172.21.15.71:0/2698868475 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae419e790 con 0x7f3ae4101820 2024-03-04T08:56:43.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.385+0000 7f3ae17fa640 1 -- 172.21.15.71:0/2698868475 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3adc005020 con 0x7f3ae4101820 2024-03-04T08:56:43.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.385+0000 7f3ae17fa640 1 -- 172.21.15.71:0/2698868475 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc0116b0 con 0x7f3ae4101820 2024-03-04T08:56:43.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.385+0000 7f3aea49a640 1 -- 172.21.15.71:0/2698868475 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae419ece0 con 0x7f3ae4101820 2024-03-04T08:56:43.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.386+0000 7f3aea49a640 1 -- 172.21.15.71:0/2698868475 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae4107440 con 0x7f3ae4101820 2024-03-04T08:56:43.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.387+0000 7f3ae17fa640 1 -- 172.21.15.71:0/2698868475 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f3adc005560 con 0x7f3ae4101820 2024-03-04T08:56:43.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.387+0000 7f3ae17fa640 1 --2- 172.21.15.71:0/2698868475 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3acc0a1d80 0x7f3acc0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:43.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.387+0000 7f3ae3fff640 1 --2- 172.21.15.71:0/2698868475 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3acc0a1d80 0x7f3acc0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:43.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.387+0000 7f3ae17fa640 1 -- 172.21.15.71:0/2698868475 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f3adc0ce4e0 con 0x7f3ae4101820 2024-03-04T08:56:43.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.388+0000 7f3ae3fff640 1 --2- 172.21.15.71:0/2698868475 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3acc0a1d80 0x7f3acc0a4240 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4000c00 tx=0x7f3ad4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:43.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.391+0000 7f3ae17fa640 1 -- 172.21.15.71:0/2698868475 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3adc016030 con 0x7f3ae4101820 2024-03-04T08:56:43.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.565+0000 7f3aea49a640 1 -- 172.21.15.71:0/2698868475 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3ae402cff0 con 0x7f3ae4101820 2024-03-04T08:56:43.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.566+0000 7f3ae17fa640 1 -- 172.21.15.71:0/2698868475 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3adc09b080 con 0x7f3ae4101820 2024-03-04T08:56:43.562 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2024-03-04T08:56:43.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.568+0000 7f3aea49a640 1 -- 172.21.15.71:0/2698868475 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3acc0a1d80 msgr2=0x7f3acc0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:43.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.568+0000 7f3aea49a640 1 --2- 172.21.15.71:0/2698868475 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3acc0a1d80 0x7f3acc0a4240 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4000c00 tx=0x7f3ad4005fd0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.568+0000 7f3aea49a640 1 -- 172.21.15.71:0/2698868475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae4101820 msgr2=0x7f3ae419de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:43.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.568+0000 7f3aea49a640 1 --2- 172.21.15.71:0/2698868475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae4101820 0x7f3ae419de30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00eaa0 tx=0x7f3adc00ef60 comp rx=0 tx=0).stop 2024-03-04T08:56:43.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.568+0000 7f3aea49a640 1 -- 172.21.15.71:0/2698868475 shutdown_connections 2024-03-04T08:56:43.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.568+0000 7f3aea49a640 1 --2- 172.21.15.71:0/2698868475 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3acc0a1d80 0x7f3acc0a4240 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.568+0000 7f3aea49a640 1 --2- 172.21.15.71:0/2698868475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ae4101820 0x7f3ae419de30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.568+0000 7f3aea49a640 1 --2- 172.21.15.71:0/2698868475 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3ae4100e60 0x7f3ae4196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.568+0000 7f3aea49a640 1 --2- 172.21.15.71:0/2698868475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ae40ffc70 0x7f3ae4196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.568+0000 7f3aea49a640 1 -- 172.21.15.71:0/2698868475 >> 172.21.15.71:0/2698868475 conn(0x7f3ae40fb200 msgr2=0x7f3ae4104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:43.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.568+0000 7f3aea49a640 1 -- 172.21.15.71:0/2698868475 shutdown_connections 2024-03-04T08:56:43.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.568+0000 7f3aea49a640 1 -- 172.21.15.71:0/2698868475 wait complete. 2024-03-04T08:56:43.575 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T08:56:43.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.684+0000 7f2d30ba0640 1 -- 172.21.15.71:0/951833602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d2c1003b0 msgr2=0x7f2d2c1007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:43.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.684+0000 7f2d30ba0640 1 --2- 172.21.15.71:0/951833602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d2c1003b0 0x7f2d2c1007d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009860 tx=0x7f2d1c02bb70 comp rx=0 tx=0).stop 2024-03-04T08:56:43.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.684+0000 7f2d30ba0640 1 -- 172.21.15.71:0/951833602 shutdown_connections 2024-03-04T08:56:43.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.684+0000 7f2d30ba0640 1 --2- 172.21.15.71:0/951833602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d2c101f60 0x7f2d2c1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.684+0000 7f2d30ba0640 1 --2- 172.21.15.71:0/951833602 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2d2c1015a0 0x7f2d2c101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.684+0000 7f2d30ba0640 1 --2- 172.21.15.71:0/951833602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d2c1003b0 0x7f2d2c1007d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.684+0000 7f2d30ba0640 1 -- 172.21.15.71:0/951833602 >> 172.21.15.71:0/951833602 conn(0x7f2d2c0fb940 msgr2=0x7f2d2c0fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:43.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.684+0000 7f2d30ba0640 1 -- 172.21.15.71:0/951833602 shutdown_connections 2024-03-04T08:56:43.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.684+0000 7f2d30ba0640 1 -- 172.21.15.71:0/951833602 wait complete. 2024-03-04T08:56:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.684+0000 7f2d30ba0640 1 Processor -- start 2024-03-04T08:56:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.684+0000 7f2d30ba0640 1 -- start start 2024-03-04T08:56:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.685+0000 7f2d30ba0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d2c1003b0 0x7f2d2c197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.685+0000 7f2d30ba0640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2d2c1015a0 0x7f2d2c197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.685+0000 7f2d30ba0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d2c101f60 0x7f2d2c19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.685+0000 7f2d30ba0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c10c040 con 0x7f2d2c101f60 2024-03-04T08:56:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.685+0000 7f2d30ba0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c10bec0 con 0x7f2d2c1003b0 2024-03-04T08:56:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.685+0000 7f2d30ba0640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c10c1c0 con 0x7f2d2c1015a0 2024-03-04T08:56:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.685+0000 7f2d2bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d2c101f60 0x7f2d2c19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.685+0000 7f2d2b7fe640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d2c1003b0 0x7f2d2c197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.685+0000 7f2d2bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d2c101f60 0x7f2d2c19edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56728/0 (socket says 172.21.15.71:56728) 2024-03-04T08:56:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.685+0000 7f2d2bfff640 1 -- 172.21.15.71:0/1631365381 learned_addr learned my addr 172.21.15.71:0/1631365381 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:43.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.685+0000 7f2d2b7fe640 1 --2- 172.21.15.71:0/1631365381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d2c1003b0 0x7f2d2c197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.71:38626/0 (socket says 172.21.15.71:38626) 2024-03-04T08:56:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.685+0000 7f2d2affd640 1 --2- 172.21.15.71:0/1631365381 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2d2c1015a0 0x7f2d2c197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.685+0000 7f2d2bfff640 1 -- 172.21.15.71:0/1631365381 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2d2c1015a0 msgr2=0x7f2d2c197d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.685+0000 7f2d2bfff640 1 --2- 172.21.15.71:0/1631365381 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2d2c1015a0 0x7f2d2c197d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.685+0000 7f2d2bfff640 1 -- 172.21.15.71:0/1631365381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d2c1003b0 msgr2=0x7f2d2c197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.685+0000 7f2d2bfff640 1 --2- 172.21.15.71:0/1631365381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d2c1003b0 0x7f2d2c197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.685+0000 7f2d2bfff640 1 -- 172.21.15.71:0/1631365381 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c19f460 con 0x7f2d2c101f60 2024-03-04T08:56:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.685+0000 7f2d2b7fe640 1 --2- 172.21.15.71:0/1631365381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d2c1003b0 0x7f2d2c197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:56:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.685+0000 7f2d2bfff640 1 --2- 172.21.15.71:0/1631365381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d2c101f60 0x7f2d2c19edf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400efc0 tx=0x7f2d2400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.685+0000 7f2d28ff9640 1 -- 172.21.15.71:0/1631365381 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24003970 con 0x7f2d2c101f60 2024-03-04T08:56:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.685+0000 7f2d30ba0640 1 -- 172.21.15.71:0/1631365381 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d2c19f750 con 0x7f2d2c101f60 2024-03-04T08:56:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.685+0000 7f2d28ff9640 1 -- 172.21.15.71:0/1631365381 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d24004020 con 0x7f2d2c101f60 2024-03-04T08:56:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.685+0000 7f2d28ff9640 1 -- 172.21.15.71:0/1631365381 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2400b9f0 con 0x7f2d2c101f60 2024-03-04T08:56:43.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.685+0000 7f2d30ba0640 1 -- 172.21.15.71:0/1631365381 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2c19fc50 con 0x7f2d2c101f60 2024-03-04T08:56:43.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.686+0000 7f2d30ba0640 1 -- 172.21.15.71:0/1631365381 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d2c19feb0 con 0x7f2d2c101f60 2024-03-04T08:56:43.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.687+0000 7f2d28ff9640 1 -- 172.21.15.71:0/1631365381 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f2d2401d020 con 0x7f2d2c101f60 2024-03-04T08:56:43.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.688+0000 7f2d28ff9640 1 --2- 172.21.15.71:0/1631365381 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f2d140a1d80 0x7f2d140a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:43.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.688+0000 7f2d2b7fe640 1 --2- 172.21.15.71:0/1631365381 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f2d140a1d80 0x7f2d140a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:43.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.688+0000 7f2d28ff9640 1 -- 172.21.15.71:0/1631365381 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f2d240cde50 con 0x7f2d2c101f60 2024-03-04T08:56:43.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.688+0000 7f2d2b7fe640 1 --2- 172.21.15.71:0/1631365381 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f2d140a1d80 0x7f2d140a4240 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009830 tx=0x7f2d1c02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:43.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.692+0000 7f2d28ff9640 1 -- 172.21.15.71:0/1631365381 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2d2409b040 con 0x7f2d2c101f60 2024-03-04T08:56:43.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.866+0000 7f2d30ba0640 1 -- 172.21.15.71:0/1631365381 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2d2c1a01a0 con 0x7f2d2c101f60 2024-03-04T08:56:43.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.866+0000 7f2d28ff9640 1 -- 172.21.15.71:0/1631365381 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2d2409a940 con 0x7f2d2c101f60 2024-03-04T08:56:43.862 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-03-04T08:56:43.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.868+0000 7f2d30ba0640 1 -- 172.21.15.71:0/1631365381 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f2d140a1d80 msgr2=0x7f2d140a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:43.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.868+0000 7f2d30ba0640 1 --2- 172.21.15.71:0/1631365381 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f2d140a1d80 0x7f2d140a4240 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009830 tx=0x7f2d1c02e040 comp rx=0 tx=0).stop 2024-03-04T08:56:43.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.868+0000 7f2d30ba0640 1 -- 172.21.15.71:0/1631365381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d2c101f60 msgr2=0x7f2d2c19edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:43.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.868+0000 7f2d30ba0640 1 --2- 172.21.15.71:0/1631365381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d2c101f60 0x7f2d2c19edf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400efc0 tx=0x7f2d2400c800 comp rx=0 tx=0).stop 2024-03-04T08:56:43.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.868+0000 7f2d30ba0640 1 -- 172.21.15.71:0/1631365381 shutdown_connections 2024-03-04T08:56:43.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.868+0000 7f2d30ba0640 1 --2- 172.21.15.71:0/1631365381 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f2d140a1d80 0x7f2d140a4240 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.868+0000 7f2d30ba0640 1 --2- 172.21.15.71:0/1631365381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d2c101f60 0x7f2d2c19edf0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.868+0000 7f2d30ba0640 1 --2- 172.21.15.71:0/1631365381 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2d2c1015a0 0x7f2d2c197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.868+0000 7f2d30ba0640 1 --2- 172.21.15.71:0/1631365381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d2c1003b0 0x7f2d2c197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.868+0000 7f2d30ba0640 1 -- 172.21.15.71:0/1631365381 >> 172.21.15.71:0/1631365381 conn(0x7f2d2c0fb940 msgr2=0x7f2d2c068730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:43.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.868+0000 7f2d30ba0640 1 -- 172.21.15.71:0/1631365381 shutdown_connections 2024-03-04T08:56:43.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.868+0000 7f2d30ba0640 1 -- 172.21.15.71:0/1631365381 wait complete. 2024-03-04T08:56:43.874 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:43 smithi071 ceph-mon[32789]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-03-04T08:56:43.874 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:43 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/841603082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:56:43.874 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:43 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3107980408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T08:56:43.874 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:43 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2698868475' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T08:56:43.876 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-04T08:56:43.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.982+0000 7fd44a516640 1 -- 172.21.15.71:0/2674462057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4441004d0 msgr2=0x7fd4441008f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:43.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.982+0000 7fd44a516640 1 --2- 172.21.15.71:0/2674462057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4441004d0 0x7fd4441008f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009900 tx=0x7fd42c02bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:43.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.982+0000 7fd44a516640 1 -- 172.21.15.71:0/2674462057 shutdown_connections 2024-03-04T08:56:43.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.982+0000 7fd44a516640 1 --2- 172.21.15.71:0/2674462057 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd444102080 0x7fd444108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.982+0000 7fd44a516640 1 --2- 172.21.15.71:0/2674462057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4441016c0 0x7fd444101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.982+0000 7fd44a516640 1 --2- 172.21.15.71:0/2674462057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4441004d0 0x7fd4441008f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.982+0000 7fd44a516640 1 -- 172.21.15.71:0/2674462057 >> 172.21.15.71:0/2674462057 conn(0x7fd4440fba60 msgr2=0x7fd4440fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:43.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.982+0000 7fd44a516640 1 -- 172.21.15.71:0/2674462057 shutdown_connections 2024-03-04T08:56:43.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.982+0000 7fd44a516640 1 -- 172.21.15.71:0/2674462057 wait complete. 2024-03-04T08:56:43.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.982+0000 7fd44a516640 1 Processor -- start 2024-03-04T08:56:43.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.982+0000 7fd44a516640 1 -- start start 2024-03-04T08:56:43.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.982+0000 7fd44a516640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd4441004d0 0x7fd4440692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:43.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.983+0000 7fd44a516640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4441016c0 0x7fd44419c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:43.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.983+0000 7fd44a516640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd444102080 0x7fd44419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:43.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.983+0000 7fd44a516640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44410c1a0 con 0x7fd4441016c0 2024-03-04T08:56:43.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.983+0000 7fd44a516640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44410c020 con 0x7fd444102080 2024-03-04T08:56:43.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.983+0000 7fd44a516640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44410c320 con 0x7fd4441004d0 2024-03-04T08:56:43.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.983+0000 7fd4437fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4441016c0 0x7fd44419c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:43.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.983+0000 7fd4437fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4441016c0 0x7fd44419c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56746/0 (socket says 172.21.15.71:56746) 2024-03-04T08:56:43.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.983+0000 7fd4437fe640 1 -- 172.21.15.71:0/1460393871 learned_addr learned my addr 172.21.15.71:0/1460393871 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:43.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.983+0000 7fd448ab3640 1 --2- 172.21.15.71:0/1460393871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd444102080 0x7fd44419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:43.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.983+0000 7fd443fff640 1 --2- 172.21.15.71:0/1460393871 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd4441004d0 0x7fd4440692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:43.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.983+0000 7fd4437fe640 1 -- 172.21.15.71:0/1460393871 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd4441004d0 msgr2=0x7fd4440692c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:43.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.983+0000 7fd4437fe640 1 --2- 172.21.15.71:0/1460393871 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd4441004d0 0x7fd4440692c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.983+0000 7fd4437fe640 1 -- 172.21.15.71:0/1460393871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd444102080 msgr2=0x7fd44419ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:43.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.983+0000 7fd4437fe640 1 --2- 172.21.15.71:0/1460393871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd444102080 0x7fd44419ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:43.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.983+0000 7fd4437fe640 1 -- 172.21.15.71:0/1460393871 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44419f5b0 con 0x7fd4441016c0 2024-03-04T08:56:43.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.983+0000 7fd443fff640 1 --2- 172.21.15.71:0/1460393871 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd4441004d0 0x7fd4440692c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:56:43.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.983+0000 7fd4437fe640 1 --2- 172.21.15.71:0/1460393871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4441016c0 0x7fd44419c6b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd43400e9f0 tx=0x7fd43400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:43.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.983+0000 7fd4417fa640 1 -- 172.21.15.71:0/1460393871 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43400cd80 con 0x7fd4441016c0 2024-03-04T08:56:43.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.983+0000 7fd4417fa640 1 -- 172.21.15.71:0/1460393871 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd434010910 con 0x7fd4441016c0 2024-03-04T08:56:43.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.983+0000 7fd44a516640 1 -- 172.21.15.71:0/1460393871 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd44419f8a0 con 0x7fd4441016c0 2024-03-04T08:56:43.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.983+0000 7fd4417fa640 1 -- 172.21.15.71:0/1460393871 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434018990 con 0x7fd4441016c0 2024-03-04T08:56:43.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.983+0000 7fd44a516640 1 -- 172.21.15.71:0/1460393871 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd44419fdf0 con 0x7fd4441016c0 2024-03-04T08:56:43.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.984+0000 7fd44a516640 1 -- 172.21.15.71:0/1460393871 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd444107870 con 0x7fd4441016c0 2024-03-04T08:56:43.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.985+0000 7fd4417fa640 1 -- 172.21.15.71:0/1460393871 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fd434010ac0 con 0x7fd4441016c0 2024-03-04T08:56:43.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.986+0000 7fd4417fa640 1 --2- 172.21.15.71:0/1460393871 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd4300a1d80 0x7fd4300a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:43.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.986+0000 7fd443fff640 1 --2- 172.21.15.71:0/1460393871 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd4300a1d80 0x7fd4300a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:43.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.986+0000 7fd4417fa640 1 -- 172.21.15.71:0/1460393871 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd4340ce3b0 con 0x7fd4441016c0 2024-03-04T08:56:43.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.986+0000 7fd443fff640 1 --2- 172.21.15.71:0/1460393871 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd4300a1d80 0x7fd4300a4240 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd42c02c7b0 tx=0x7fd42c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:43.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:43.991+0000 7fd4417fa640 1 -- 172.21.15.71:0/1460393871 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd43409b5a0 con 0x7fd4441016c0 2024-03-04T08:56:44.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:43 smithi150 ceph-mon[36807]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-03-04T08:56:44.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:43 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/841603082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:56:44.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:43 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3107980408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T08:56:44.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:43 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2698868475' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T08:56:44.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:43 smithi158 ceph-mon[37241]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-03-04T08:56:44.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:43 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/841603082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:56:44.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:43 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3107980408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T08:56:44.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:43 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2698868475' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T08:56:44.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.177+0000 7fd44a516640 1 -- 172.21.15.71:0/1460393871 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd4440621d0 con 0x7fd4441016c0 2024-03-04T08:56:44.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.178+0000 7fd4417fa640 1 -- 172.21.15.71:0/1460393871 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) v1 ==== 81+0+591 (secure 0 0 0) 0x7fd43409aea0 con 0x7fd4441016c0 2024-03-04T08:56:44.174 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/1287712358 2024-03-05T08:48:22.593931+0000 2024-03-04T08:56:44.174 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/3770452860 2024-03-05T08:48:22.593931+0000 2024-03-04T08:56:44.174 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/718743094 2024-03-05T08:47:55.502914+0000 2024-03-04T08:56:44.174 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6801/1350134536 2024-03-05T08:47:55.502914+0000 2024-03-04T08:56:44.174 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/1904717532 2024-03-05T08:47:55.502914+0000 2024-03-04T08:56:44.174 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6801/2808156336 2024-03-05T08:48:22.593931+0000 2024-03-04T08:56:44.174 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6800/1350134536 2024-03-05T08:47:55.502914+0000 2024-03-04T08:56:44.174 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6800/2808156336 2024-03-05T08:48:22.593931+0000 2024-03-04T08:56:44.174 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/2862184313 2024-03-05T08:47:55.502914+0000 2024-03-04T08:56:44.174 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/3621754630 2024-03-05T08:48:22.593931+0000 2024-03-04T08:56:44.175 INFO:teuthology.orchestra.run.smithi071.stderr:listed 10 entries 2024-03-04T08:56:44.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.180+0000 7fd44a516640 1 -- 172.21.15.71:0/1460393871 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd4300a1d80 msgr2=0x7fd4300a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:44.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.180+0000 7fd44a516640 1 --2- 172.21.15.71:0/1460393871 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd4300a1d80 0x7fd4300a4240 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd42c02c7b0 tx=0x7fd42c005fd0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.180+0000 7fd44a516640 1 -- 172.21.15.71:0/1460393871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4441016c0 msgr2=0x7fd44419c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:44.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.180+0000 7fd44a516640 1 --2- 172.21.15.71:0/1460393871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4441016c0 0x7fd44419c6b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd43400e9f0 tx=0x7fd43400eeb0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.180+0000 7fd44a516640 1 -- 172.21.15.71:0/1460393871 shutdown_connections 2024-03-04T08:56:44.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.180+0000 7fd44a516640 1 --2- 172.21.15.71:0/1460393871 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd4300a1d80 0x7fd4300a4240 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.180+0000 7fd44a516640 1 --2- 172.21.15.71:0/1460393871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd444102080 0x7fd44419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.180+0000 7fd44a516640 1 --2- 172.21.15.71:0/1460393871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4441016c0 0x7fd44419c6b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.180+0000 7fd44a516640 1 --2- 172.21.15.71:0/1460393871 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd4441004d0 0x7fd4440692c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.180+0000 7fd44a516640 1 -- 172.21.15.71:0/1460393871 >> 172.21.15.71:0/1460393871 conn(0x7fd4440fba60 msgr2=0x7fd444104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:44.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.180+0000 7fd44a516640 1 -- 172.21.15.71:0/1460393871 shutdown_connections 2024-03-04T08:56:44.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.180+0000 7fd44a516640 1 -- 172.21.15.71:0/1460393871 wait complete. 2024-03-04T08:56:44.186 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-04T08:56:44.186 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T08:56:44.203 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T08:56:44.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.349+0000 7f69d59a0640 1 -- 172.21.15.71:0/2232005820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69d0101ed0 msgr2=0x7f69d0108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:44.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.349+0000 7f69d59a0640 1 --2- 172.21.15.71:0/2232005820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69d0101ed0 0x7f69d0108760 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f69b8009900 tx=0x7f69b802bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:44.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.349+0000 7f69d59a0640 1 -- 172.21.15.71:0/2232005820 shutdown_connections 2024-03-04T08:56:44.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.349+0000 7f69d59a0640 1 --2- 172.21.15.71:0/2232005820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69d0101ed0 0x7f69d0108760 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.349+0000 7f69d59a0640 1 --2- 172.21.15.71:0/2232005820 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f69d0101510 0x7f69d0101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.349+0000 7f69d59a0640 1 --2- 172.21.15.71:0/2232005820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69d0100320 0x7f69d0100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.349+0000 7f69d59a0640 1 -- 172.21.15.71:0/2232005820 >> 172.21.15.71:0/2232005820 conn(0x7f69d00fb8b0 msgr2=0x7f69d00fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:44.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.349+0000 7f69d59a0640 1 -- 172.21.15.71:0/2232005820 shutdown_connections 2024-03-04T08:56:44.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.349+0000 7f69d59a0640 1 -- 172.21.15.71:0/2232005820 wait complete. 2024-03-04T08:56:44.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.349+0000 7f69d59a0640 1 Processor -- start 2024-03-04T08:56:44.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.349+0000 7f69d59a0640 1 -- start start 2024-03-04T08:56:44.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.350+0000 7f69d59a0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69d0100320 0x7f69d006a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:44.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.350+0000 7f69d59a0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69d0101510 0x7f69d019c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:44.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.350+0000 7f69d59a0640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f69d0101ed0 0x7f69d019ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:44.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.350+0000 7f69d59a0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d010bfb0 con 0x7f69d0100320 2024-03-04T08:56:44.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.350+0000 7f69d59a0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d010be30 con 0x7f69d0101510 2024-03-04T08:56:44.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.350+0000 7f69d59a0640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d010c130 con 0x7f69d0101ed0 2024-03-04T08:56:44.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.350+0000 7f69d499e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69d0100320 0x7f69d006a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:44.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.350+0000 7f69d499e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69d0100320 0x7f69d006a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56762/0 (socket says 172.21.15.71:56762) 2024-03-04T08:56:44.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.350+0000 7f69d499e640 1 -- 172.21.15.71:0/394739350 learned_addr learned my addr 172.21.15.71:0/394739350 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:44.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.350+0000 7f69cffff640 1 --2- 172.21.15.71:0/394739350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69d0101510 0x7f69d019c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:44.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.350+0000 7f69d519f640 1 --2- 172.21.15.71:0/394739350 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f69d0101ed0 0x7f69d019ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:44.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.350+0000 7f69d499e640 1 -- 172.21.15.71:0/394739350 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f69d0101ed0 msgr2=0x7f69d019ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:44.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.350+0000 7f69d499e640 1 --2- 172.21.15.71:0/394739350 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f69d0101ed0 0x7f69d019ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.350+0000 7f69d499e640 1 -- 172.21.15.71:0/394739350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69d0101510 msgr2=0x7f69d019c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:44.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.350+0000 7f69d499e640 1 --2- 172.21.15.71:0/394739350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69d0101510 0x7f69d019c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.350+0000 7f69d499e640 1 -- 172.21.15.71:0/394739350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d019f2d0 con 0x7f69d0100320 2024-03-04T08:56:44.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.350+0000 7f69d499e640 1 --2- 172.21.15.71:0/394739350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69d0100320 0x7f69d006a690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f69c8009d20 tx=0x7f69c800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:44.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.350+0000 7f69cdffb640 1 -- 172.21.15.71:0/394739350 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c8010630 con 0x7f69d0100320 2024-03-04T08:56:44.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.350+0000 7f69cdffb640 1 -- 172.21.15.71:0/394739350 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f69c8010cb0 con 0x7f69d0100320 2024-03-04T08:56:44.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.350+0000 7f69d59a0640 1 -- 172.21.15.71:0/394739350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d019f5c0 con 0x7f69d0100320 2024-03-04T08:56:44.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.350+0000 7f69cdffb640 1 -- 172.21.15.71:0/394739350 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c8018de0 con 0x7f69d0100320 2024-03-04T08:56:44.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.351+0000 7f69d59a0640 1 -- 172.21.15.71:0/394739350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d019fb60 con 0x7f69d0100320 2024-03-04T08:56:44.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.352+0000 7f69d59a0640 1 -- 172.21.15.71:0/394739350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d0107640 con 0x7f69d0100320 2024-03-04T08:56:44.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.353+0000 7f69cdffb640 1 -- 172.21.15.71:0/394739350 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f69c801f020 con 0x7f69d0100320 2024-03-04T08:56:44.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.353+0000 7f69cdffb640 1 --2- 172.21.15.71:0/394739350 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f69bc0a1e00 0x7f69bc0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:44.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.353+0000 7f69cffff640 1 --2- 172.21.15.71:0/394739350 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f69bc0a1e00 0x7f69bc0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:44.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.353+0000 7f69cdffb640 1 -- 172.21.15.71:0/394739350 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f69c80ce480 con 0x7f69d0100320 2024-03-04T08:56:44.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.354+0000 7f69cffff640 1 --2- 172.21.15.71:0/394739350 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f69bc0a1e00 0x7f69bc0a42c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f69d006a400 tx=0x7f69c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:44.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.357+0000 7f69cdffb640 1 -- 172.21.15.71:0/394739350 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f69c809b670 con 0x7f69d0100320 2024-03-04T08:56:44.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.531+0000 7f69d59a0640 1 -- 172.21.15.71:0/394739350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69d00621d0 con 0x7f69d0100320 2024-03-04T08:56:44.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.533+0000 7f69cdffb640 1 -- 172.21.15.71:0/394739350 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24153 (secure 0 0 0) 0x7f69c809af70 con 0x7f69d0100320 2024-03-04T08:56:44.529 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:44.529 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":91,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","created":"2024-03-04T08:47:28.546102+0000","modified":"2024-03-04T08:56:35.007858+0000","last_up_change":"2024-03-04T08:55:30.744961+0000","last_in_change":"2024-03-04T08:55:11.570990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T08:51:14.933487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T08:56:28.253074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T08:56:29.543622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T08:56:32.884220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7e43b47d-8888-4631-bc96-8a96fcf8cd46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6803","nonce":1173641530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6805","nonce":1173641530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6809","nonce":1173641530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6807","nonce":1173641530}]},"public_addr":"172.21.15.71:6803/1173641530","cluster_addr":"172.21.15.71:6805/1173641530","heartbeat_back_addr":"172.21.15.71:6809/1173641530","heartbeat_front_addr":"172.21.15.71:6807/1173641530","state":["exists","up"]},{"osd":1,"uuid":"1209f214-f2e2-4209-bf69-4528a43a2123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6801","nonce":3383948110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6803","nonce":3383948110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6807","nonce":3383948110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6805","nonce":3383948110}]},"public_addr":"172.21.15.150:6801/3383948110","cluster_addr":"172.21.15.150:6803/3383948110","heartbeat_back_addr":"172.21.15.150:6807/3383948110","heartbeat_front_addr":"172.21.15.150:6805/3383948110","state":["exists","up"]},{"osd":2,"uuid":"b88c3082-93a9-4c9b-9221-a7105a44c91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6801","nonce":586944144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6803","nonce":586944144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6807","nonce":586944144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6805","nonce":586944144}]},"public_addr":"172.21.15.158:6801/586944144","cluster_addr":"172.21.15.158:6803/586944144","heartbeat_back_addr":"172.21.15.158:6807/586944144","heartbeat_front_addr":"172.21.15.158:6805/586944144","state":["exists","up"]},{"osd":3,"uuid":"e4fc7723-fd73-40b2-b9d0-7f9bda282ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6811","nonce":1311903262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6813","nonce":1311903262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6817","nonce":1311903262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6815","nonce":1311903262}]},"public_addr":"172.21.15.71:6811/1311903262","cluster_addr":"172.21.15.71:6813/1311903262","heartbeat_back_addr":"172.21.15.71:6817/1311903262","heartbeat_front_addr":"172.21.15.71:6815/1311903262","state":["exists","up"]},{"osd":4,"uuid":"e7eb72ac-f066-4ccd-b2bf-22a6a26daf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6809","nonce":2300837573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6811","nonce":2300837573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6815","nonce":2300837573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6813","nonce":2300837573}]},"public_addr":"172.21.15.150:6809/2300837573","cluster_addr":"172.21.15.150:6811/2300837573","heartbeat_back_addr":"172.21.15.150:6815/2300837573","heartbeat_front_addr":"172.21.15.150:6813/2300837573","state":["exists","up"]},{"osd":5,"uuid":"9b295840-e104-4a66-9cc4-7020627c1765","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6809","nonce":1922694045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6811","nonce":1922694045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6815","nonce":1922694045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6813","nonce":1922694045}]},"public_addr":"172.21.15.158:6809/1922694045","cluster_addr":"172.21.15.158:6811/1922694045","heartbeat_back_addr":"172.21.15.158:6815/1922694045","heartbeat_front_addr":"172.21.15.158:6813/1922694045","state":["exists","up"]},{"osd":6,"uuid":"6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6819","nonce":3848842707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6821","nonce":3848842707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6825","nonce":3848842707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6823","nonce":3848842707}]},"public_addr":"172.21.15.71:6819/3848842707","cluster_addr":"172.21.15.71:6821/3848842707","heartbeat_back_addr":"172.21.15.71:6825/3848842707","heartbeat_front_addr":"172.21.15.71:6823/3848842707","state":["exists","up"]},{"osd":7,"uuid":"6160a40a-d30f-4569-949c-0afb1bb3d133","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6817","nonce":1328330305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6819","nonce":1328330305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6823","nonce":1328330305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6821","nonce":1328330305}]},"public_addr":"172.21.15.150:6817/1328330305","cluster_addr":"172.21.15.150:6819/1328330305","heartbeat_back_addr":"172.21.15.150:6823/1328330305","heartbeat_front_addr":"172.21.15.150:6821/1328330305","state":["exists","up"]},{"osd":8,"uuid":"95ed2341-84a4-465c-9948-ac667680ce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6817","nonce":2195304577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6819","nonce":2195304577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6823","nonce":2195304577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6821","nonce":2195304577}]},"public_addr":"172.21.15.158:6817/2195304577","cluster_addr":"172.21.15.158:6819/2195304577","heartbeat_back_addr":"172.21.15.158:6823/2195304577","heartbeat_front_addr":"172.21.15.158:6821/2195304577","state":["exists","up"]},{"osd":9,"uuid":"088f6419-63a0-4768-afae-dbb8a5f5a565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6827","nonce":1913779347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6829","nonce":1913779347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6833","nonce":1913779347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6831","nonce":1913779347}]},"public_addr":"172.21.15.71:6827/1913779347","cluster_addr":"172.21.15.71:6829/1913779347","heartbeat_back_addr":"172.21.15.71:6833/1913779347","heartbeat_front_addr":"172.21.15.71:6831/1913779347","state":["exists","up"]},{"osd":10,"uuid":"18c85e24-068d-4f1e-a3b8-f7915b363234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6825","nonce":3295420996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6827","nonce":3295420996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6831","nonce":3295420996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6829","nonce":3295420996}]},"public_addr":"172.21.15.150:6825/3295420996","cluster_addr":"172.21.15.150:6827/3295420996","heartbeat_back_addr":"172.21.15.150:6831/3295420996","heartbeat_front_addr":"172.21.15.150:6829/3295420996","state":["exists","up"]},{"osd":11,"uuid":"4f6b3ddf-c482-4740-b1fd-de836e047c2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6825","nonce":3332083865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6827","nonce":3332083865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6831","nonce":3332083865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6829","nonce":3332083865}]},"public_addr":"172.21.15.158:6825/3332083865","cluster_addr":"172.21.15.158:6827/3332083865","heartbeat_back_addr":"172.21.15.158:6831/3332083865","heartbeat_front_addr":"172.21.15.158:6829/3332083865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:05.997832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:29.314235+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:08.819465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:41.194275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:06.032479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:38.814498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:13.067960+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:38.568479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:03.416713+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:38.349479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:04.096945+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:29.425250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1287712358":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/3770452860":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/718743094":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/1904717532":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:6800/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6800/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/2862184313":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/3621754630":"2024-03-05T08:48:22.593931+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T08:56:44.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.535+0000 7f69d59a0640 1 -- 172.21.15.71:0/394739350 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f69bc0a1e00 msgr2=0x7f69bc0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:44.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.535+0000 7f69d59a0640 1 --2- 172.21.15.71:0/394739350 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f69bc0a1e00 0x7f69bc0a42c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f69d006a400 tx=0x7f69c0005f90 comp rx=0 tx=0).stop 2024-03-04T08:56:44.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.535+0000 7f69d59a0640 1 -- 172.21.15.71:0/394739350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69d0100320 msgr2=0x7f69d006a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:44.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.535+0000 7f69d59a0640 1 --2- 172.21.15.71:0/394739350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69d0100320 0x7f69d006a690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f69c8009d20 tx=0x7f69c800c6a0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.535+0000 7f69d59a0640 1 -- 172.21.15.71:0/394739350 shutdown_connections 2024-03-04T08:56:44.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.535+0000 7f69d59a0640 1 --2- 172.21.15.71:0/394739350 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f69bc0a1e00 0x7f69bc0a42c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.535+0000 7f69d59a0640 1 --2- 172.21.15.71:0/394739350 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f69d0101ed0 0x7f69d019ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.535+0000 7f69d59a0640 1 --2- 172.21.15.71:0/394739350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69d0101510 0x7f69d019c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.535+0000 7f69d59a0640 1 --2- 172.21.15.71:0/394739350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69d0100320 0x7f69d006a690 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.535+0000 7f69d59a0640 1 -- 172.21.15.71:0/394739350 >> 172.21.15.71:0/394739350 conn(0x7f69d00fb8b0 msgr2=0x7f69d0104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:44.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.535+0000 7f69d59a0640 1 -- 172.21.15.71:0/394739350 shutdown_connections 2024-03-04T08:56:44.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.535+0000 7f69d59a0640 1 -- 172.21.15.71:0/394739350 wait complete. 2024-03-04T08:56:44.542 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T08:51:14.933487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T08:56:28.253074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T08:56:29.543622+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T08:56:32.884220+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T08:56:44.542 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T08:56:44.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.649+0000 7fc2ae3c7640 1 -- 172.21.15.71:0/653567721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2a8102110 msgr2=0x7fc2a8104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:44.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.649+0000 7fc2ae3c7640 1 --2- 172.21.15.71:0/653567721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2a8102110 0x7fc2a8104500 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc298009900 tx=0x7fc29802bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:44.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.649+0000 7fc2ae3c7640 1 -- 172.21.15.71:0/653567721 shutdown_connections 2024-03-04T08:56:44.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.649+0000 7fc2ae3c7640 1 --2- 172.21.15.71:0/653567721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2a8107370 0x7fc2a8109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.649+0000 7fc2ae3c7640 1 --2- 172.21.15.71:0/653567721 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2a8104a40 0x7fc2a8106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.649+0000 7fc2ae3c7640 1 --2- 172.21.15.71:0/653567721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2a8102110 0x7fc2a8104500 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.649+0000 7fc2ae3c7640 1 -- 172.21.15.71:0/653567721 >> 172.21.15.71:0/653567721 conn(0x7fc2a80fba60 msgr2=0x7fc2a80fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:44.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.649+0000 7fc2ae3c7640 1 -- 172.21.15.71:0/653567721 shutdown_connections 2024-03-04T08:56:44.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.650+0000 7fc2ae3c7640 1 -- 172.21.15.71:0/653567721 wait complete. 2024-03-04T08:56:44.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.650+0000 7fc2ae3c7640 1 Processor -- start 2024-03-04T08:56:44.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.650+0000 7fc2ae3c7640 1 -- start start 2024-03-04T08:56:44.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.650+0000 7fc2ae3c7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2a8102110 0x7fc2a8197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:44.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.650+0000 7fc2ae3c7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2a8104a40 0x7fc2a8197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:44.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.650+0000 7fc2ae3c7640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2a8107370 0x7fc2a819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:44.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.650+0000 7fc2ae3c7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a810d270 con 0x7fc2a8104a40 2024-03-04T08:56:44.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.650+0000 7fc2ae3c7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a810d0f0 con 0x7fc2a8102110 2024-03-04T08:56:44.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.650+0000 7fc2ae3c7640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a810d3f0 con 0x7fc2a8107370 2024-03-04T08:56:44.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.650+0000 7fc2a77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2a8104a40 0x7fc2a8197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:44.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.650+0000 7fc2a77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2a8104a40 0x7fc2a8197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56776/0 (socket says 172.21.15.71:56776) 2024-03-04T08:56:44.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.650+0000 7fc2a77fe640 1 -- 172.21.15.71:0/1292851059 learned_addr learned my addr 172.21.15.71:0/1292851059 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:44.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.651+0000 7fc2a7fff640 1 --2- 172.21.15.71:0/1292851059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2a8102110 0x7fc2a8197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:44.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.651+0000 7fc2ac964640 1 --2- 172.21.15.71:0/1292851059 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2a8107370 0x7fc2a819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:44.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.651+0000 7fc2a77fe640 1 -- 172.21.15.71:0/1292851059 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2a8107370 msgr2=0x7fc2a819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:44.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.651+0000 7fc2a77fe640 1 --2- 172.21.15.71:0/1292851059 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2a8107370 0x7fc2a819ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.651+0000 7fc2a77fe640 1 -- 172.21.15.71:0/1292851059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2a8102110 msgr2=0x7fc2a8197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:44.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.651+0000 7fc2a77fe640 1 --2- 172.21.15.71:0/1292851059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2a8102110 0x7fc2a8197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.651+0000 7fc2a77fe640 1 -- 172.21.15.71:0/1292851059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a819f5a0 con 0x7fc2a8104a40 2024-03-04T08:56:44.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.651+0000 7fc2a77fe640 1 --2- 172.21.15.71:0/1292851059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2a8104a40 0x7fc2a8197ec0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00d740 tx=0x7fc29c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:44.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.651+0000 7fc2a57fa640 1 -- 172.21.15.71:0/1292851059 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c017070 con 0x7fc2a8104a40 2024-03-04T08:56:44.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.651+0000 7fc2a57fa640 1 -- 172.21.15.71:0/1292851059 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc29c00de50 con 0x7fc2a8104a40 2024-03-04T08:56:44.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.651+0000 7fc2ae3c7640 1 -- 172.21.15.71:0/1292851059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a819f890 con 0x7fc2a8104a40 2024-03-04T08:56:44.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.651+0000 7fc2a57fa640 1 -- 172.21.15.71:0/1292851059 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c004e00 con 0x7fc2a8104a40 2024-03-04T08:56:44.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.651+0000 7fc2ae3c7640 1 -- 172.21.15.71:0/1292851059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a819fd90 con 0x7fc2a8104a40 2024-03-04T08:56:44.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.652+0000 7fc2ae3c7640 1 -- 172.21.15.71:0/1292851059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a80fd550 con 0x7fc2a8104a40 2024-03-04T08:56:44.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.653+0000 7fc2a57fa640 1 -- 172.21.15.71:0/1292851059 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fc29c004fa0 con 0x7fc2a8104a40 2024-03-04T08:56:44.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.654+0000 7fc2a57fa640 1 --2- 172.21.15.71:0/1292851059 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc2900a1d80 0x7fc2900a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:44.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.654+0000 7fc2a7fff640 1 --2- 172.21.15.71:0/1292851059 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc2900a1d80 0x7fc2900a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:44.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.654+0000 7fc2a57fa640 1 -- 172.21.15.71:0/1292851059 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fc29c0cd8e0 con 0x7fc2a8104a40 2024-03-04T08:56:44.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.654+0000 7fc2a7fff640 1 --2- 172.21.15.71:0/1292851059 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc2900a1d80 0x7fc2900a4240 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc29802c7b0 tx=0x7fc298005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:44.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.658+0000 7fc2a57fa640 1 -- 172.21.15.71:0/1292851059 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc29c09a480 con 0x7fc2a8104a40 2024-03-04T08:56:44.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.835+0000 7fc2ae3c7640 1 -- 172.21.15.71:0/1292851059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc2a80621d0 con 0x7fc2a8104a40 2024-03-04T08:56:44.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.835+0000 7fc2a57fa640 1 -- 172.21.15.71:0/1292851059 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc29c09e520 con 0x7fc2a8104a40 2024-03-04T08:56:44.832 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-03-04T08:56:44.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.837+0000 7fc2ae3c7640 1 -- 172.21.15.71:0/1292851059 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc2900a1d80 msgr2=0x7fc2900a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:44.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.837+0000 7fc2ae3c7640 1 --2- 172.21.15.71:0/1292851059 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc2900a1d80 0x7fc2900a4240 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc29802c7b0 tx=0x7fc298005fd0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.837+0000 7fc2ae3c7640 1 -- 172.21.15.71:0/1292851059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2a8104a40 msgr2=0x7fc2a8197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:44.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.837+0000 7fc2ae3c7640 1 --2- 172.21.15.71:0/1292851059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2a8104a40 0x7fc2a8197ec0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00d740 tx=0x7fc29c00dc00 comp rx=0 tx=0).stop 2024-03-04T08:56:44.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.838+0000 7fc2ae3c7640 1 -- 172.21.15.71:0/1292851059 shutdown_connections 2024-03-04T08:56:44.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.838+0000 7fc2ae3c7640 1 --2- 172.21.15.71:0/1292851059 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc2900a1d80 0x7fc2900a4240 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.838+0000 7fc2ae3c7640 1 --2- 172.21.15.71:0/1292851059 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2a8107370 0x7fc2a819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.838+0000 7fc2ae3c7640 1 --2- 172.21.15.71:0/1292851059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2a8104a40 0x7fc2a8197ec0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.838+0000 7fc2ae3c7640 1 --2- 172.21.15.71:0/1292851059 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2a8102110 0x7fc2a8197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.838+0000 7fc2ae3c7640 1 -- 172.21.15.71:0/1292851059 >> 172.21.15.71:0/1292851059 conn(0x7fc2a80fba60 msgr2=0x7fc2a8102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:44.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.838+0000 7fc2ae3c7640 1 -- 172.21.15.71:0/1292851059 shutdown_connections 2024-03-04T08:56:44.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.838+0000 7fc2ae3c7640 1 -- 172.21.15.71:0/1292851059 wait complete. 2024-03-04T08:56:44.843 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:44 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1631365381' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T08:56:44.843 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:44 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1460393871' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T08:56:44.843 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:44 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/394739350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:44.844 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T08:56:44.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.952+0000 7fe197d4b640 1 -- 172.21.15.71:0/3250584326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1901015e0 msgr2=0x7fe190101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:44.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.952+0000 7fe197d4b640 1 --2- 172.21.15.71:0/3250584326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1901015e0 0x7fe190101a00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe18c009900 tx=0x7fe18c02bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:44.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.953+0000 7fe197d4b640 1 -- 172.21.15.71:0/3250584326 shutdown_connections 2024-03-04T08:56:44.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.953+0000 7fe197d4b640 1 --2- 172.21.15.71:0/3250584326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe190103190 0x7fe190109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.953+0000 7fe197d4b640 1 --2- 172.21.15.71:0/3250584326 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe1901027d0 0x7fe190102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.953+0000 7fe197d4b640 1 --2- 172.21.15.71:0/3250584326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1901015e0 0x7fe190101a00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.953+0000 7fe197d4b640 1 -- 172.21.15.71:0/3250584326 >> 172.21.15.71:0/3250584326 conn(0x7fe1900fcb70 msgr2=0x7fe1900fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:44.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.953+0000 7fe197d4b640 1 -- 172.21.15.71:0/3250584326 shutdown_connections 2024-03-04T08:56:44.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.953+0000 7fe197d4b640 1 -- 172.21.15.71:0/3250584326 wait complete. 2024-03-04T08:56:44.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.953+0000 7fe197d4b640 1 Processor -- start 2024-03-04T08:56:44.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.953+0000 7fe197d4b640 1 -- start start 2024-03-04T08:56:44.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.953+0000 7fe197d4b640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe1901015e0 0x7fe190197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:44.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.953+0000 7fe197d4b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1901027d0 0x7fe190197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:44.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.953+0000 7fe197d4b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe190103190 0x7fe19019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:44.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.953+0000 7fe197d4b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19010d250 con 0x7fe1901027d0 2024-03-04T08:56:44.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.953+0000 7fe197d4b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19010d0d0 con 0x7fe190103190 2024-03-04T08:56:44.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.953+0000 7fe197d4b640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19010d3d0 con 0x7fe1901015e0 2024-03-04T08:56:44.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.953+0000 7fe1952e6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1901027d0 0x7fe190197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:44.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.953+0000 7fe1952e6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1901027d0 0x7fe190197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56790/0 (socket says 172.21.15.71:56790) 2024-03-04T08:56:44.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.953+0000 7fe1952e6640 1 -- 172.21.15.71:0/4036360341 learned_addr learned my addr 172.21.15.71:0/4036360341 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:44.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.954+0000 7fe1962e8640 1 --2- 172.21.15.71:0/4036360341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe190103190 0x7fe19019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:44.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.954+0000 7fe195ae7640 1 --2- 172.21.15.71:0/4036360341 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe1901015e0 0x7fe190197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:44.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.954+0000 7fe1952e6640 1 -- 172.21.15.71:0/4036360341 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe1901015e0 msgr2=0x7fe190197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:44.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.954+0000 7fe1952e6640 1 --2- 172.21.15.71:0/4036360341 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe1901015e0 0x7fe190197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.954+0000 7fe1952e6640 1 -- 172.21.15.71:0/4036360341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe190103190 msgr2=0x7fe19019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:44.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.954+0000 7fe1952e6640 1 --2- 172.21.15.71:0/4036360341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe190103190 0x7fe19019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:44.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.954+0000 7fe1952e6640 1 -- 172.21.15.71:0/4036360341 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19019f520 con 0x7fe1901027d0 2024-03-04T08:56:44.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.954+0000 7fe195ae7640 1 --2- 172.21.15.71:0/4036360341 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe1901015e0 0x7fe190197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:56:44.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.954+0000 7fe1952e6640 1 --2- 172.21.15.71:0/4036360341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1901027d0 0x7fe190197e40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe18000e9f0 tx=0x7fe18000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:44.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.954+0000 7fe186ffd640 1 -- 172.21.15.71:0/4036360341 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18000cd80 con 0x7fe1901027d0 2024-03-04T08:56:44.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.954+0000 7fe186ffd640 1 -- 172.21.15.71:0/4036360341 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe180005500 con 0x7fe1901027d0 2024-03-04T08:56:44.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.954+0000 7fe186ffd640 1 -- 172.21.15.71:0/4036360341 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe180010690 con 0x7fe1901027d0 2024-03-04T08:56:44.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.954+0000 7fe197d4b640 1 -- 172.21.15.71:0/4036360341 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe19019f810 con 0x7fe1901027d0 2024-03-04T08:56:44.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.954+0000 7fe197d4b640 1 -- 172.21.15.71:0/4036360341 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe19019fd10 con 0x7fe1901027d0 2024-03-04T08:56:44.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.955+0000 7fe197d4b640 1 -- 172.21.15.71:0/4036360341 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe190108980 con 0x7fe1901027d0 2024-03-04T08:56:44.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.956+0000 7fe186ffd640 1 -- 172.21.15.71:0/4036360341 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fe1800046c0 con 0x7fe1901027d0 2024-03-04T08:56:44.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.957+0000 7fe186ffd640 1 --2- 172.21.15.71:0/4036360341 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe17c0a1d80 0x7fe17c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:44.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.957+0000 7fe195ae7640 1 --2- 172.21.15.71:0/4036360341 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe17c0a1d80 0x7fe17c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:44.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.957+0000 7fe186ffd640 1 -- 172.21.15.71:0/4036360341 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe1800ce490 con 0x7fe1901027d0 2024-03-04T08:56:44.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.957+0000 7fe195ae7640 1 --2- 172.21.15.71:0/4036360341 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe17c0a1d80 0x7fe17c0a4240 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe18c036040 tx=0x7fe18c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:44.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:44.961+0000 7fe186ffd640 1 -- 172.21.15.71:0/4036360341 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe18009b680 con 0x7fe1901027d0 2024-03-04T08:56:45.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:44 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1631365381' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T08:56:45.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:44 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1460393871' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T08:56:45.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:44 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/394739350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:45.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:44 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1631365381' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T08:56:45.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:44 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1460393871' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T08:56:45.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:44 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/394739350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:45.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.135+0000 7fe197d4b640 1 -- 172.21.15.71:0/4036360341 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe1900621d0 con 0x7fe1901027d0 2024-03-04T08:56:45.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.135+0000 7fe186ffd640 1 -- 172.21.15.71:0/4036360341 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe18009af80 con 0x7fe1901027d0 2024-03-04T08:56:45.131 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-03-04T08:56:45.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.137+0000 7fe197d4b640 1 -- 172.21.15.71:0/4036360341 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe17c0a1d80 msgr2=0x7fe17c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:45.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.137+0000 7fe197d4b640 1 --2- 172.21.15.71:0/4036360341 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe17c0a1d80 0x7fe17c0a4240 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe18c036040 tx=0x7fe18c000bc0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.137+0000 7fe197d4b640 1 -- 172.21.15.71:0/4036360341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1901027d0 msgr2=0x7fe190197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:45.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.137+0000 7fe197d4b640 1 --2- 172.21.15.71:0/4036360341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1901027d0 0x7fe190197e40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe18000e9f0 tx=0x7fe18000edf0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.137+0000 7fe197d4b640 1 -- 172.21.15.71:0/4036360341 shutdown_connections 2024-03-04T08:56:45.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.137+0000 7fe197d4b640 1 --2- 172.21.15.71:0/4036360341 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe17c0a1d80 0x7fe17c0a4240 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.137+0000 7fe197d4b640 1 --2- 172.21.15.71:0/4036360341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe190103190 0x7fe19019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.137+0000 7fe197d4b640 1 --2- 172.21.15.71:0/4036360341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1901027d0 0x7fe190197e40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.137+0000 7fe197d4b640 1 --2- 172.21.15.71:0/4036360341 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe1901015e0 0x7fe190197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.137+0000 7fe197d4b640 1 -- 172.21.15.71:0/4036360341 >> 172.21.15.71:0/4036360341 conn(0x7fe1900fcb70 msgr2=0x7fe190105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:45.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.137+0000 7fe197d4b640 1 -- 172.21.15.71:0/4036360341 shutdown_connections 2024-03-04T08:56:45.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.137+0000 7fe197d4b640 1 -- 172.21.15.71:0/4036360341 wait complete. 2024-03-04T08:56:45.144 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T08:56:45.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.252+0000 7f7a205bc640 1 -- 172.21.15.71:0/3750046135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a180ffc80 msgr2=0x7f7a181000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:45.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.252+0000 7f7a205bc640 1 --2- 172.21.15.71:0/3750046135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a180ffc80 0x7f7a181000a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7a1400aa10 tx=0x7f7a1402cf70 comp rx=0 tx=0).stop 2024-03-04T08:56:45.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.252+0000 7f7a205bc640 1 -- 172.21.15.71:0/3750046135 shutdown_connections 2024-03-04T08:56:45.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.252+0000 7f7a205bc640 1 --2- 172.21.15.71:0/3750046135 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7a18101830 0x7f7a181080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.252+0000 7f7a205bc640 1 --2- 172.21.15.71:0/3750046135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a18100e70 0x7f7a181012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.252+0000 7f7a205bc640 1 --2- 172.21.15.71:0/3750046135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a180ffc80 0x7f7a181000a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.252+0000 7f7a205bc640 1 -- 172.21.15.71:0/3750046135 >> 172.21.15.71:0/3750046135 conn(0x7f7a180fb1d0 msgr2=0x7f7a180fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:45.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.252+0000 7f7a205bc640 1 -- 172.21.15.71:0/3750046135 shutdown_connections 2024-03-04T08:56:45.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.252+0000 7f7a205bc640 1 -- 172.21.15.71:0/3750046135 wait complete. 2024-03-04T08:56:45.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.252+0000 7f7a205bc640 1 Processor -- start 2024-03-04T08:56:45.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.252+0000 7f7a205bc640 1 -- start start 2024-03-04T08:56:45.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.252+0000 7f7a205bc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a180ffc80 0x7f7a18197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:45.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.252+0000 7f7a205bc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a18100e70 0x7f7a18197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:45.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.252+0000 7f7a205bc640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7a18101830 0x7f7a1819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:45.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.252+0000 7f7a205bc640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1810b090 con 0x7f7a18100e70 2024-03-04T08:56:45.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.252+0000 7f7a205bc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1810af10 con 0x7f7a180ffc80 2024-03-04T08:56:45.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.252+0000 7f7a205bc640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1810b210 con 0x7f7a18101830 2024-03-04T08:56:45.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.253+0000 7f7a1db57640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a18100e70 0x7f7a18197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:45.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.253+0000 7f7a1db57640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a18100e70 0x7f7a18197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56812/0 (socket says 172.21.15.71:56812) 2024-03-04T08:56:45.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.253+0000 7f7a1db57640 1 -- 172.21.15.71:0/1131425068 learned_addr learned my addr 172.21.15.71:0/1131425068 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:45.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.253+0000 7f7a1e358640 1 --2- 172.21.15.71:0/1131425068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a180ffc80 0x7f7a18197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:45.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.253+0000 7f7a1eb59640 1 --2- 172.21.15.71:0/1131425068 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7a18101830 0x7f7a1819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:45.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.253+0000 7f7a1db57640 1 -- 172.21.15.71:0/1131425068 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7a18101830 msgr2=0x7f7a1819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:45.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.253+0000 7f7a1db57640 1 --2- 172.21.15.71:0/1131425068 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7a18101830 0x7f7a1819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.253+0000 7f7a1db57640 1 -- 172.21.15.71:0/1131425068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a180ffc80 msgr2=0x7f7a18197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:45.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.253+0000 7f7a1db57640 1 --2- 172.21.15.71:0/1131425068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a180ffc80 0x7f7a18197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.253+0000 7f7a1db57640 1 -- 172.21.15.71:0/1131425068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1819f540 con 0x7f7a18100e70 2024-03-04T08:56:45.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.253+0000 7f7a1db57640 1 --2- 172.21.15.71:0/1131425068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a18100e70 0x7f7a18197e60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800ea60 tx=0x7f7a0800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:45.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.253+0000 7f7a0f7fe640 1 -- 172.21.15.71:0/1131425068 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0800cde0 con 0x7f7a18100e70 2024-03-04T08:56:45.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.253+0000 7f7a0f7fe640 1 -- 172.21.15.71:0/1131425068 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a080035f0 con 0x7f7a18100e70 2024-03-04T08:56:45.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.253+0000 7f7a205bc640 1 -- 172.21.15.71:0/1131425068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a1819f830 con 0x7f7a18100e70 2024-03-04T08:56:45.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.253+0000 7f7a0f7fe640 1 -- 172.21.15.71:0/1131425068 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a08010670 con 0x7f7a18100e70 2024-03-04T08:56:45.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.253+0000 7f7a205bc640 1 -- 172.21.15.71:0/1131425068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a1819fd80 con 0x7f7a18100e70 2024-03-04T08:56:45.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.254+0000 7f7a205bc640 1 -- 172.21.15.71:0/1131425068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a1819ffe0 con 0x7f7a18100e70 2024-03-04T08:56:45.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.255+0000 7f7a0f7fe640 1 -- 172.21.15.71:0/1131425068 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f7a080037a0 con 0x7f7a18100e70 2024-03-04T08:56:45.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.256+0000 7f7a0f7fe640 1 --2- 172.21.15.71:0/1131425068 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f7a040a1d80 0x7f7a040a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:45.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.256+0000 7f7a1e358640 1 --2- 172.21.15.71:0/1131425068 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f7a040a1d80 0x7f7a040a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:45.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.256+0000 7f7a0f7fe640 1 -- 172.21.15.71:0/1131425068 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f7a080ce340 con 0x7f7a18100e70 2024-03-04T08:56:45.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.256+0000 7f7a1e358640 1 --2- 172.21.15.71:0/1131425068 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f7a040a1d80 0x7f7a040a4240 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7a14000c00 tx=0x7f7a14005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:45.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.259+0000 7f7a0f7fe640 1 -- 172.21.15.71:0/1131425068 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7a0809b5c0 con 0x7f7a18100e70 2024-03-04T08:56:45.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.432+0000 7f7a205bc640 1 -- 172.21.15.71:0/1131425068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7a181a01b0 con 0x7f7a18100e70 2024-03-04T08:56:45.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.432+0000 7f7a0f7fe640 1 -- 172.21.15.71:0/1131425068 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7a0809aec0 con 0x7f7a18100e70 2024-03-04T08:56:45.429 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2024-03-04T08:56:45.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.434+0000 7f7a205bc640 1 -- 172.21.15.71:0/1131425068 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f7a040a1d80 msgr2=0x7f7a040a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:45.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.434+0000 7f7a205bc640 1 --2- 172.21.15.71:0/1131425068 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f7a040a1d80 0x7f7a040a4240 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7a14000c00 tx=0x7f7a14005fd0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.434+0000 7f7a205bc640 1 -- 172.21.15.71:0/1131425068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a18100e70 msgr2=0x7f7a18197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:45.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.434+0000 7f7a205bc640 1 --2- 172.21.15.71:0/1131425068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a18100e70 0x7f7a18197e60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800ea60 tx=0x7f7a0800ef20 comp rx=0 tx=0).stop 2024-03-04T08:56:45.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.434+0000 7f7a205bc640 1 -- 172.21.15.71:0/1131425068 shutdown_connections 2024-03-04T08:56:45.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.434+0000 7f7a205bc640 1 --2- 172.21.15.71:0/1131425068 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f7a040a1d80 0x7f7a040a4240 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.434+0000 7f7a205bc640 1 --2- 172.21.15.71:0/1131425068 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7a18101830 0x7f7a1819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.434+0000 7f7a205bc640 1 --2- 172.21.15.71:0/1131425068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a18100e70 0x7f7a18197e60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.434+0000 7f7a205bc640 1 --2- 172.21.15.71:0/1131425068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a180ffc80 0x7f7a18197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.434+0000 7f7a205bc640 1 -- 172.21.15.71:0/1131425068 >> 172.21.15.71:0/1131425068 conn(0x7f7a180fb1d0 msgr2=0x7f7a18106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:45.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.435+0000 7f7a205bc640 1 -- 172.21.15.71:0/1131425068 shutdown_connections 2024-03-04T08:56:45.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.435+0000 7f7a205bc640 1 -- 172.21.15.71:0/1131425068 wait complete. 2024-03-04T08:56:45.441 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T08:56:45.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.550+0000 7fc9512c3640 1 -- 172.21.15.71:0/2048780503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc94c102050 msgr2=0x7fc94c104440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:45.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.550+0000 7fc9512c3640 1 --2- 172.21.15.71:0/2048780503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc94c102050 0x7fc94c104440 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009900 tx=0x7fc93c02bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:45.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.550+0000 7fc9512c3640 1 -- 172.21.15.71:0/2048780503 shutdown_connections 2024-03-04T08:56:45.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.550+0000 7fc9512c3640 1 --2- 172.21.15.71:0/2048780503 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc94c1072b0 0x7fc94c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.550+0000 7fc9512c3640 1 --2- 172.21.15.71:0/2048780503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc94c104980 0x7fc94c106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.550+0000 7fc9512c3640 1 --2- 172.21.15.71:0/2048780503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc94c102050 0x7fc94c104440 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.550+0000 7fc9512c3640 1 -- 172.21.15.71:0/2048780503 >> 172.21.15.71:0/2048780503 conn(0x7fc94c0fba60 msgr2=0x7fc94c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:45.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.550+0000 7fc9512c3640 1 -- 172.21.15.71:0/2048780503 shutdown_connections 2024-03-04T08:56:45.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.550+0000 7fc9512c3640 1 -- 172.21.15.71:0/2048780503 wait complete. 2024-03-04T08:56:45.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.550+0000 7fc9512c3640 1 Processor -- start 2024-03-04T08:56:45.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.550+0000 7fc9512c3640 1 -- start start 2024-03-04T08:56:45.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.550+0000 7fc9512c3640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc94c102050 0x7fc94c1019c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:45.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.550+0000 7fc9512c3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc94c104980 0x7fc94c100000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:45.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.550+0000 7fc9512c3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc94c1072b0 0x7fc94c100540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:45.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.550+0000 7fc9512c3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c10d260 con 0x7fc94c104980 2024-03-04T08:56:45.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.550+0000 7fc9512c3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c10d0e0 con 0x7fc94c1072b0 2024-03-04T08:56:45.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.550+0000 7fc9512c3640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c10d3e0 con 0x7fc94c102050 2024-03-04T08:56:45.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.551+0000 7fc94a7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc94c104980 0x7fc94c100000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:45.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.551+0000 7fc94a7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc94c104980 0x7fc94c100000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56830/0 (socket says 172.21.15.71:56830) 2024-03-04T08:56:45.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.551+0000 7fc94a7fc640 1 -- 172.21.15.71:0/1117043731 learned_addr learned my addr 172.21.15.71:0/1117043731 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:45.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.551+0000 7fc94b7fe640 1 --2- 172.21.15.71:0/1117043731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc94c1072b0 0x7fc94c100540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:45.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.551+0000 7fc94affd640 1 --2- 172.21.15.71:0/1117043731 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc94c102050 0x7fc94c1019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:45.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.551+0000 7fc94a7fc640 1 -- 172.21.15.71:0/1117043731 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc94c102050 msgr2=0x7fc94c1019c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:45.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.551+0000 7fc94a7fc640 1 --2- 172.21.15.71:0/1117043731 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc94c102050 0x7fc94c1019c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.551+0000 7fc94a7fc640 1 -- 172.21.15.71:0/1117043731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc94c1072b0 msgr2=0x7fc94c100540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:45.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.551+0000 7fc94a7fc640 1 --2- 172.21.15.71:0/1117043731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc94c1072b0 0x7fc94c100540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.551+0000 7fc94a7fc640 1 -- 172.21.15.71:0/1117043731 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94c100dd0 con 0x7fc94c104980 2024-03-04T08:56:45.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.551+0000 7fc94a7fc640 1 --2- 172.21.15.71:0/1117043731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc94c104980 0x7fc94c100000 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc94000e9a0 tx=0x7fc94000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:45.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.551+0000 7fc933fff640 1 -- 172.21.15.71:0/1117043731 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94000cd80 con 0x7fc94c104980 2024-03-04T08:56:45.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.551+0000 7fc9512c3640 1 -- 172.21.15.71:0/1117043731 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc94c1a1a50 con 0x7fc94c104980 2024-03-04T08:56:45.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.551+0000 7fc933fff640 1 -- 172.21.15.71:0/1117043731 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc9400035f0 con 0x7fc94c104980 2024-03-04T08:56:45.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.551+0000 7fc933fff640 1 -- 172.21.15.71:0/1117043731 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940010670 con 0x7fc94c104980 2024-03-04T08:56:45.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.551+0000 7fc9512c3640 1 -- 172.21.15.71:0/1117043731 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc94c1a1fa0 con 0x7fc94c104980 2024-03-04T08:56:45.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.552+0000 7fc9512c3640 1 -- 172.21.15.71:0/1117043731 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc94c0fd550 con 0x7fc94c104980 2024-03-04T08:56:45.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.553+0000 7fc933fff640 1 -- 172.21.15.71:0/1117043731 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fc9400037a0 con 0x7fc94c104980 2024-03-04T08:56:45.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.554+0000 7fc933fff640 1 --2- 172.21.15.71:0/1117043731 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc9340a1d80 0x7fc9340a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:45.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.554+0000 7fc94affd640 1 --2- 172.21.15.71:0/1117043731 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc9340a1d80 0x7fc9340a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:45.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.554+0000 7fc933fff640 1 -- 172.21.15.71:0/1117043731 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fc9400ce340 con 0x7fc94c104980 2024-03-04T08:56:45.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.554+0000 7fc94affd640 1 --2- 172.21.15.71:0/1117043731 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc9340a1d80 0x7fc9340a4240 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc93c005c20 tx=0x7fc93c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:45.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.558+0000 7fc933fff640 1 -- 172.21.15.71:0/1117043731 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc94009b530 con 0x7fc94c104980 2024-03-04T08:56:45.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.735+0000 7fc9512c3640 1 -- 172.21.15.71:0/1117043731 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc94c0621d0 con 0x7fc94c104980 2024-03-04T08:56:45.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.736+0000 7fc933fff640 1 -- 172.21.15.71:0/1117043731 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc94009ae30 con 0x7fc94c104980 2024-03-04T08:56:45.732 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-03-04T08:56:45.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.738+0000 7fc9512c3640 1 -- 172.21.15.71:0/1117043731 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc9340a1d80 msgr2=0x7fc9340a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:45.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.738+0000 7fc9512c3640 1 --2- 172.21.15.71:0/1117043731 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc9340a1d80 0x7fc9340a4240 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc93c005c20 tx=0x7fc93c005fd0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.738+0000 7fc9512c3640 1 -- 172.21.15.71:0/1117043731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc94c104980 msgr2=0x7fc94c100000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:45.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.738+0000 7fc9512c3640 1 --2- 172.21.15.71:0/1117043731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc94c104980 0x7fc94c100000 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc94000e9a0 tx=0x7fc94000ee60 comp rx=0 tx=0).stop 2024-03-04T08:56:45.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.738+0000 7fc9512c3640 1 -- 172.21.15.71:0/1117043731 shutdown_connections 2024-03-04T08:56:45.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.738+0000 7fc9512c3640 1 --2- 172.21.15.71:0/1117043731 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc9340a1d80 0x7fc9340a4240 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.738+0000 7fc9512c3640 1 --2- 172.21.15.71:0/1117043731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc94c1072b0 0x7fc94c100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.738+0000 7fc9512c3640 1 --2- 172.21.15.71:0/1117043731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc94c104980 0x7fc94c100000 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.738+0000 7fc9512c3640 1 --2- 172.21.15.71:0/1117043731 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc94c102050 0x7fc94c1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.738+0000 7fc9512c3640 1 -- 172.21.15.71:0/1117043731 >> 172.21.15.71:0/1117043731 conn(0x7fc94c0fba60 msgr2=0x7fc94c1029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:45.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.738+0000 7fc9512c3640 1 -- 172.21.15.71:0/1117043731 shutdown_connections 2024-03-04T08:56:45.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.738+0000 7fc9512c3640 1 -- 172.21.15.71:0/1117043731 wait complete. 2024-03-04T08:56:45.745 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-04T08:56:45.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.850+0000 7f7f37bb6640 1 -- 172.21.15.71:0/4235853841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f30104980 msgr2=0x7f7f30106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:45.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.850+0000 7f7f37bb6640 1 --2- 172.21.15.71:0/4235853841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f30104980 0x7f7f30106d70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7f20009900 tx=0x7f7f2002bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:45.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.850+0000 7f7f37bb6640 1 -- 172.21.15.71:0/4235853841 shutdown_connections 2024-03-04T08:56:45.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.850+0000 7f7f37bb6640 1 --2- 172.21.15.71:0/4235853841 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f301072b0 0x7f7f30109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.850+0000 7f7f37bb6640 1 --2- 172.21.15.71:0/4235853841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f30104980 0x7f7f30106d70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.850+0000 7f7f37bb6640 1 --2- 172.21.15.71:0/4235853841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f30102050 0x7f7f30104440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.850+0000 7f7f37bb6640 1 -- 172.21.15.71:0/4235853841 >> 172.21.15.71:0/4235853841 conn(0x7f7f300fba60 msgr2=0x7f7f300fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:45.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.851+0000 7f7f37bb6640 1 -- 172.21.15.71:0/4235853841 shutdown_connections 2024-03-04T08:56:45.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.851+0000 7f7f37bb6640 1 -- 172.21.15.71:0/4235853841 wait complete. 2024-03-04T08:56:45.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.851+0000 7f7f37bb6640 1 Processor -- start 2024-03-04T08:56:45.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.851+0000 7f7f37bb6640 1 -- start start 2024-03-04T08:56:45.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.851+0000 7f7f37bb6640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f30102050 0x7f7f30197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:45.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.851+0000 7f7f37bb6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f30104980 0x7f7f30197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:45.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.851+0000 7f7f37bb6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f301072b0 0x7f7f3019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:45.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.851+0000 7f7f37bb6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3010d260 con 0x7f7f301072b0 2024-03-04T08:56:45.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.851+0000 7f7f37bb6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3010d0e0 con 0x7f7f30104980 2024-03-04T08:56:45.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.851+0000 7f7f37bb6640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3010d3e0 con 0x7f7f30102050 2024-03-04T08:56:45.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.851+0000 7f7f36153640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f301072b0 0x7f7f3019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:45.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.851+0000 7f7f36153640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f301072b0 0x7f7f3019eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52742/0 (socket says 172.21.15.71:52742) 2024-03-04T08:56:45.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.851+0000 7f7f36153640 1 -- 172.21.15.71:0/595825270 learned_addr learned my addr 172.21.15.71:0/595825270 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:45.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.851+0000 7f7f35151640 1 --2- 172.21.15.71:0/595825270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f30104980 0x7f7f30197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:45.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.852+0000 7f7f35952640 1 --2- 172.21.15.71:0/595825270 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f30102050 0x7f7f30197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:45.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.852+0000 7f7f36153640 1 -- 172.21.15.71:0/595825270 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f30102050 msgr2=0x7f7f30197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:45.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.852+0000 7f7f36153640 1 --2- 172.21.15.71:0/595825270 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f30102050 0x7f7f30197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.852+0000 7f7f36153640 1 -- 172.21.15.71:0/595825270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f30104980 msgr2=0x7f7f30197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:45.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.852+0000 7f7f36153640 1 --2- 172.21.15.71:0/595825270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f30104980 0x7f7f30197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:45.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.852+0000 7f7f36153640 1 -- 172.21.15.71:0/595825270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3019f560 con 0x7f7f301072b0 2024-03-04T08:56:45.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.852+0000 7f7f35952640 1 --2- 172.21.15.71:0/595825270 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f30102050 0x7f7f30197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:56:45.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.852+0000 7f7f36153640 1 --2- 172.21.15.71:0/595825270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f301072b0 0x7f7f3019eef0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7f2800eaa0 tx=0x7f7f2800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:45.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.852+0000 7f7f26ffd640 1 -- 172.21.15.71:0/595825270 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2800cdb0 con 0x7f7f301072b0 2024-03-04T08:56:45.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.852+0000 7f7f26ffd640 1 -- 172.21.15.71:0/595825270 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f28005020 con 0x7f7f301072b0 2024-03-04T08:56:45.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.852+0000 7f7f37bb6640 1 -- 172.21.15.71:0/595825270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f3019f850 con 0x7f7f301072b0 2024-03-04T08:56:45.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.852+0000 7f7f26ffd640 1 -- 172.21.15.71:0/595825270 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f280116b0 con 0x7f7f301072b0 2024-03-04T08:56:45.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.852+0000 7f7f37bb6640 1 -- 172.21.15.71:0/595825270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f3019fd70 con 0x7f7f301072b0 2024-03-04T08:56:45.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.853+0000 7f7f37bb6640 1 -- 172.21.15.71:0/595825270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f300fd550 con 0x7f7f301072b0 2024-03-04T08:56:45.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.854+0000 7f7f26ffd640 1 -- 172.21.15.71:0/595825270 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f7f28005560 con 0x7f7f301072b0 2024-03-04T08:56:45.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.854+0000 7f7f26ffd640 1 --2- 172.21.15.71:0/595825270 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f7f1c0a1d80 0x7f7f1c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:45.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.854+0000 7f7f35952640 1 --2- 172.21.15.71:0/595825270 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f7f1c0a1d80 0x7f7f1c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:45.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.855+0000 7f7f26ffd640 1 -- 172.21.15.71:0/595825270 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f7f280ce240 con 0x7f7f301072b0 2024-03-04T08:56:45.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.855+0000 7f7f35952640 1 --2- 172.21.15.71:0/595825270 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f7f1c0a1d80 0x7f7f1c0a4240 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c009b50 tx=0x7f7f2c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:45.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:45.859+0000 7f7f26ffd640 1 -- 172.21.15.71:0/595825270 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7f28016030 con 0x7f7f301072b0 2024-03-04T08:56:46.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:45 smithi150 ceph-mon[36807]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 0 op/s 2024-03-04T08:56:46.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:45 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1292851059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:56:46.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:45 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/4036360341' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T08:56:46.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:45 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1131425068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T08:56:46.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:46.048+0000 7f7f37bb6640 1 -- 172.21.15.71:0/595825270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7f3002cff0 con 0x7f7f301072b0 2024-03-04T08:56:46.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:46.048+0000 7f7f26ffd640 1 -- 172.21.15.71:0/595825270 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) v1 ==== 81+0+591 (secure 0 0 0) 0x7f7f2809b430 con 0x7f7f301072b0 2024-03-04T08:56:46.045 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/1287712358 2024-03-05T08:48:22.593931+0000 2024-03-04T08:56:46.045 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/3770452860 2024-03-05T08:48:22.593931+0000 2024-03-04T08:56:46.045 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/718743094 2024-03-05T08:47:55.502914+0000 2024-03-04T08:56:46.045 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6801/1350134536 2024-03-05T08:47:55.502914+0000 2024-03-04T08:56:46.045 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/1904717532 2024-03-05T08:47:55.502914+0000 2024-03-04T08:56:46.045 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6801/2808156336 2024-03-05T08:48:22.593931+0000 2024-03-04T08:56:46.045 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6800/1350134536 2024-03-05T08:47:55.502914+0000 2024-03-04T08:56:46.045 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6800/2808156336 2024-03-05T08:48:22.593931+0000 2024-03-04T08:56:46.045 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/2862184313 2024-03-05T08:47:55.502914+0000 2024-03-04T08:56:46.045 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/3621754630 2024-03-05T08:48:22.593931+0000 2024-03-04T08:56:46.046 INFO:teuthology.orchestra.run.smithi071.stderr:listed 10 entries 2024-03-04T08:56:46.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:46.050+0000 7f7f37bb6640 1 -- 172.21.15.71:0/595825270 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f7f1c0a1d80 msgr2=0x7f7f1c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:46.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:46.050+0000 7f7f37bb6640 1 --2- 172.21.15.71:0/595825270 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f7f1c0a1d80 0x7f7f1c0a4240 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c009b50 tx=0x7f7f2c005f90 comp rx=0 tx=0).stop 2024-03-04T08:56:46.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:46.050+0000 7f7f37bb6640 1 -- 172.21.15.71:0/595825270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f301072b0 msgr2=0x7f7f3019eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:46.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:46.050+0000 7f7f37bb6640 1 --2- 172.21.15.71:0/595825270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f301072b0 0x7f7f3019eef0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7f2800eaa0 tx=0x7f7f2800ef60 comp rx=0 tx=0).stop 2024-03-04T08:56:46.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:46.050+0000 7f7f37bb6640 1 -- 172.21.15.71:0/595825270 shutdown_connections 2024-03-04T08:56:46.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:46.050+0000 7f7f37bb6640 1 --2- 172.21.15.71:0/595825270 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f7f1c0a1d80 0x7f7f1c0a4240 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:46.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:46.050+0000 7f7f37bb6640 1 --2- 172.21.15.71:0/595825270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f301072b0 0x7f7f3019eef0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:46.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:46.050+0000 7f7f37bb6640 1 --2- 172.21.15.71:0/595825270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f30104980 0x7f7f30197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:46.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:46.050+0000 7f7f37bb6640 1 --2- 172.21.15.71:0/595825270 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f30102050 0x7f7f30197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:46.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:46.050+0000 7f7f37bb6640 1 -- 172.21.15.71:0/595825270 >> 172.21.15.71:0/595825270 conn(0x7f7f300fba60 msgr2=0x7f7f30102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:46.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:46.050+0000 7f7f37bb6640 1 -- 172.21.15.71:0/595825270 shutdown_connections 2024-03-04T08:56:46.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:46.050+0000 7f7f37bb6640 1 -- 172.21.15.71:0/595825270 wait complete. 2024-03-04T08:56:46.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:45 smithi158 ceph-mon[37241]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 0 op/s 2024-03-04T08:56:46.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:45 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1292851059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:56:46.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:45 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/4036360341' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T08:56:46.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:45 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1131425068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T08:56:46.056 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:45 smithi071 ceph-mon[32789]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 0 op/s 2024-03-04T08:56:46.056 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:45 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1292851059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:56:46.056 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:45 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/4036360341' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T08:56:46.056 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:45 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1131425068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T08:56:46.057 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi071.front.sepia.ceph.com... 2024-03-04T08:56:46.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:56:46.057 DEBUG:teuthology.orchestra.run.smithi071:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-04T08:56:46.082 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi071.stderr:ceph-fuse[76854]: fuse finished with error 0 and tester_r 0 2024-03-04T08:56:46.084 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:56:47.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:46 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1117043731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T08:56:47.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:46 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/595825270' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T08:56:47.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:46 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1117043731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T08:56:47.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:46 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/595825270' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T08:56:47.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:46 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1117043731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T08:56:47.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:46 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/595825270' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T08:56:48.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:47 smithi150 ceph-mon[36807]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 371 B/s wr, 0 op/s 2024-03-04T08:56:48.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:47 smithi158 ceph-mon[37241]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 371 B/s wr, 0 op/s 2024-03-04T08:56:48.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:47 smithi071 ceph-mon[32789]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 371 B/s wr, 0 op/s 2024-03-04T08:56:50.033 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:49 smithi150 ceph-mon[36807]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-04T08:56:50.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:49 smithi158 ceph-mon[37241]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-04T08:56:50.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:49 smithi071 ceph-mon[32789]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-04T08:56:52.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:51 smithi150 ceph-mon[36807]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-04T08:56:52.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:51 smithi158 ceph-mon[37241]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-04T08:56:52.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:51 smithi071 ceph-mon[32789]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-04T08:56:52.186 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2024-03-04T08:56:52.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:56:52.186 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-04T08:56:52.214 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T08:56:52.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.365+0000 7febdd165640 1 -- 172.21.15.71:0/661871056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febd81027e0 msgr2=0x7febd8102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:52.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.365+0000 7febdd165640 1 --2- 172.21.15.71:0/661871056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febd81027e0 0x7febd8102c60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7febc0009900 tx=0x7febc002bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:52.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.365+0000 7febdd165640 1 -- 172.21.15.71:0/661871056 shutdown_connections 2024-03-04T08:56:52.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.365+0000 7febdd165640 1 --2- 172.21.15.71:0/661871056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febd81031a0 0x7febd8109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:52.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.365+0000 7febdd165640 1 --2- 172.21.15.71:0/661871056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febd81027e0 0x7febd8102c60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:52.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.366+0000 7febdd165640 1 --2- 172.21.15.71:0/661871056 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7febd81015f0 0x7febd8101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:52.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.366+0000 7febdd165640 1 -- 172.21.15.71:0/661871056 >> 172.21.15.71:0/661871056 conn(0x7febd80fcb80 msgr2=0x7febd80fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:52.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.366+0000 7febdd165640 1 -- 172.21.15.71:0/661871056 shutdown_connections 2024-03-04T08:56:52.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.366+0000 7febdd165640 1 -- 172.21.15.71:0/661871056 wait complete. 2024-03-04T08:56:52.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.366+0000 7febdd165640 1 Processor -- start 2024-03-04T08:56:52.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.366+0000 7febdd165640 1 -- start start 2024-03-04T08:56:52.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.367+0000 7febdd165640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febd81015f0 0x7febd8197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:52.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.367+0000 7febdd165640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7febd81027e0 0x7febd8197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:52.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.367+0000 7febdd165640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febd81031a0 0x7febd819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:52.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.367+0000 7febdd165640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd810d2e0 con 0x7febd81031a0 2024-03-04T08:56:52.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.367+0000 7febdd165640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd810d160 con 0x7febd81015f0 2024-03-04T08:56:52.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.367+0000 7febdd165640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd810d460 con 0x7febd81027e0 2024-03-04T08:56:52.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.367+0000 7febd759e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febd81031a0 0x7febd819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:52.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.367+0000 7febd759e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febd81031a0 0x7febd819ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52766/0 (socket says 172.21.15.71:52766) 2024-03-04T08:56:52.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.367+0000 7febd759e640 1 -- 172.21.15.71:0/2022870266 learned_addr learned my addr 172.21.15.71:0/2022870266 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:52.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.367+0000 7febd6d9d640 1 --2- 172.21.15.71:0/2022870266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febd81015f0 0x7febd8197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:52.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.367+0000 7febd659c640 1 --2- 172.21.15.71:0/2022870266 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7febd81027e0 0x7febd8197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:52.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.367+0000 7febd759e640 1 -- 172.21.15.71:0/2022870266 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7febd81027e0 msgr2=0x7febd8197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:52.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.367+0000 7febd759e640 1 --2- 172.21.15.71:0/2022870266 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7febd81027e0 0x7febd8197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:52.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.367+0000 7febd759e640 1 -- 172.21.15.71:0/2022870266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febd81015f0 msgr2=0x7febd8197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:52.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.367+0000 7febd759e640 1 --2- 172.21.15.71:0/2022870266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febd81015f0 0x7febd8197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:52.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.367+0000 7febd759e640 1 -- 172.21.15.71:0/2022870266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd819f570 con 0x7febd81031a0 2024-03-04T08:56:52.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.367+0000 7febd659c640 1 --2- 172.21.15.71:0/2022870266 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7febd81027e0 0x7febd8197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:56:52.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.367+0000 7febd6d9d640 1 --2- 172.21.15.71:0/2022870266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febd81015f0 0x7febd8197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:56:52.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.367+0000 7febd759e640 1 --2- 172.21.15.71:0/2022870266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febd81031a0 0x7febd819ef00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7febc800e970 tx=0x7febc800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:52.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.367+0000 7febcffff640 1 -- 172.21.15.71:0/2022870266 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc800ccc0 con 0x7febd81031a0 2024-03-04T08:56:52.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.367+0000 7febcffff640 1 -- 172.21.15.71:0/2022870266 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febc800ce60 con 0x7febd81031a0 2024-03-04T08:56:52.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.367+0000 7febdd165640 1 -- 172.21.15.71:0/2022870266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd819f860 con 0x7febd81031a0 2024-03-04T08:56:52.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.367+0000 7febcffff640 1 -- 172.21.15.71:0/2022870266 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc8010490 con 0x7febd81031a0 2024-03-04T08:56:52.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.367+0000 7febdd165640 1 -- 172.21.15.71:0/2022870266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd819fd60 con 0x7febd81031a0 2024-03-04T08:56:52.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.368+0000 7febdd165640 1 -- 172.21.15.71:0/2022870266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd8108910 con 0x7febd81031a0 2024-03-04T08:56:52.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.370+0000 7febcffff640 1 -- 172.21.15.71:0/2022870266 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7febc8005020 con 0x7febd81031a0 2024-03-04T08:56:52.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.370+0000 7febcffff640 1 --2- 172.21.15.71:0/2022870266 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7febc40a1d80 0x7febc40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:52.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.370+0000 7febd6d9d640 1 --2- 172.21.15.71:0/2022870266 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7febc40a1d80 0x7febc40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:52.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.371+0000 7febcffff640 1 -- 172.21.15.71:0/2022870266 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7febc80ce500 con 0x7febd81031a0 2024-03-04T08:56:52.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.371+0000 7febd6d9d640 1 --2- 172.21.15.71:0/2022870266 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7febc40a1d80 0x7febc40a4240 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7febd8102640 tx=0x7febd000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:52.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.375+0000 7febcffff640 1 -- 172.21.15.71:0/2022870266 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7febc809b6f0 con 0x7febd81031a0 2024-03-04T08:56:52.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.552+0000 7febdd165640 1 -- 172.21.15.71:0/2022870266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7febd80621d0 con 0x7febd81031a0 2024-03-04T08:56:52.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.554+0000 7febcffff640 1 -- 172.21.15.71:0/2022870266 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24153 (secure 0 0 0) 0x7febc809aff0 con 0x7febd81031a0 2024-03-04T08:56:52.550 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:52.550 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":91,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","created":"2024-03-04T08:47:28.546102+0000","modified":"2024-03-04T08:56:35.007858+0000","last_up_change":"2024-03-04T08:55:30.744961+0000","last_in_change":"2024-03-04T08:55:11.570990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T08:51:14.933487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T08:56:28.253074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T08:56:29.543622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T08:56:32.884220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7e43b47d-8888-4631-bc96-8a96fcf8cd46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6803","nonce":1173641530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6805","nonce":1173641530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6809","nonce":1173641530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6807","nonce":1173641530}]},"public_addr":"172.21.15.71:6803/1173641530","cluster_addr":"172.21.15.71:6805/1173641530","heartbeat_back_addr":"172.21.15.71:6809/1173641530","heartbeat_front_addr":"172.21.15.71:6807/1173641530","state":["exists","up"]},{"osd":1,"uuid":"1209f214-f2e2-4209-bf69-4528a43a2123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6801","nonce":3383948110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6803","nonce":3383948110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6807","nonce":3383948110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6805","nonce":3383948110}]},"public_addr":"172.21.15.150:6801/3383948110","cluster_addr":"172.21.15.150:6803/3383948110","heartbeat_back_addr":"172.21.15.150:6807/3383948110","heartbeat_front_addr":"172.21.15.150:6805/3383948110","state":["exists","up"]},{"osd":2,"uuid":"b88c3082-93a9-4c9b-9221-a7105a44c91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6801","nonce":586944144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6803","nonce":586944144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6807","nonce":586944144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6805","nonce":586944144}]},"public_addr":"172.21.15.158:6801/586944144","cluster_addr":"172.21.15.158:6803/586944144","heartbeat_back_addr":"172.21.15.158:6807/586944144","heartbeat_front_addr":"172.21.15.158:6805/586944144","state":["exists","up"]},{"osd":3,"uuid":"e4fc7723-fd73-40b2-b9d0-7f9bda282ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6811","nonce":1311903262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6813","nonce":1311903262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6817","nonce":1311903262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6815","nonce":1311903262}]},"public_addr":"172.21.15.71:6811/1311903262","cluster_addr":"172.21.15.71:6813/1311903262","heartbeat_back_addr":"172.21.15.71:6817/1311903262","heartbeat_front_addr":"172.21.15.71:6815/1311903262","state":["exists","up"]},{"osd":4,"uuid":"e7eb72ac-f066-4ccd-b2bf-22a6a26daf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6809","nonce":2300837573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6811","nonce":2300837573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6815","nonce":2300837573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6813","nonce":2300837573}]},"public_addr":"172.21.15.150:6809/2300837573","cluster_addr":"172.21.15.150:6811/2300837573","heartbeat_back_addr":"172.21.15.150:6815/2300837573","heartbeat_front_addr":"172.21.15.150:6813/2300837573","state":["exists","up"]},{"osd":5,"uuid":"9b295840-e104-4a66-9cc4-7020627c1765","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6809","nonce":1922694045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6811","nonce":1922694045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6815","nonce":1922694045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6813","nonce":1922694045}]},"public_addr":"172.21.15.158:6809/1922694045","cluster_addr":"172.21.15.158:6811/1922694045","heartbeat_back_addr":"172.21.15.158:6815/1922694045","heartbeat_front_addr":"172.21.15.158:6813/1922694045","state":["exists","up"]},{"osd":6,"uuid":"6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6819","nonce":3848842707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6821","nonce":3848842707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6825","nonce":3848842707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6823","nonce":3848842707}]},"public_addr":"172.21.15.71:6819/3848842707","cluster_addr":"172.21.15.71:6821/3848842707","heartbeat_back_addr":"172.21.15.71:6825/3848842707","heartbeat_front_addr":"172.21.15.71:6823/3848842707","state":["exists","up"]},{"osd":7,"uuid":"6160a40a-d30f-4569-949c-0afb1bb3d133","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6817","nonce":1328330305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6819","nonce":1328330305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6823","nonce":1328330305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6821","nonce":1328330305}]},"public_addr":"172.21.15.150:6817/1328330305","cluster_addr":"172.21.15.150:6819/1328330305","heartbeat_back_addr":"172.21.15.150:6823/1328330305","heartbeat_front_addr":"172.21.15.150:6821/1328330305","state":["exists","up"]},{"osd":8,"uuid":"95ed2341-84a4-465c-9948-ac667680ce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6817","nonce":2195304577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6819","nonce":2195304577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6823","nonce":2195304577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6821","nonce":2195304577}]},"public_addr":"172.21.15.158:6817/2195304577","cluster_addr":"172.21.15.158:6819/2195304577","heartbeat_back_addr":"172.21.15.158:6823/2195304577","heartbeat_front_addr":"172.21.15.158:6821/2195304577","state":["exists","up"]},{"osd":9,"uuid":"088f6419-63a0-4768-afae-dbb8a5f5a565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6827","nonce":1913779347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6829","nonce":1913779347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6833","nonce":1913779347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6831","nonce":1913779347}]},"public_addr":"172.21.15.71:6827/1913779347","cluster_addr":"172.21.15.71:6829/1913779347","heartbeat_back_addr":"172.21.15.71:6833/1913779347","heartbeat_front_addr":"172.21.15.71:6831/1913779347","state":["exists","up"]},{"osd":10,"uuid":"18c85e24-068d-4f1e-a3b8-f7915b363234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6825","nonce":3295420996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6827","nonce":3295420996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6831","nonce":3295420996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6829","nonce":3295420996}]},"public_addr":"172.21.15.150:6825/3295420996","cluster_addr":"172.21.15.150:6827/3295420996","heartbeat_back_addr":"172.21.15.150:6831/3295420996","heartbeat_front_addr":"172.21.15.150:6829/3295420996","state":["exists","up"]},{"osd":11,"uuid":"4f6b3ddf-c482-4740-b1fd-de836e047c2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6825","nonce":3332083865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6827","nonce":3332083865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6831","nonce":3332083865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6829","nonce":3332083865}]},"public_addr":"172.21.15.158:6825/3332083865","cluster_addr":"172.21.15.158:6827/3332083865","heartbeat_back_addr":"172.21.15.158:6831/3332083865","heartbeat_front_addr":"172.21.15.158:6829/3332083865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:05.997832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:29.314235+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:08.819465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:41.194275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:06.032479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:38.814498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:13.067960+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:38.568479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:03.416713+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:38.349479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:04.096945+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:29.425250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1287712358":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/3770452860":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/718743094":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/1904717532":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:6800/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6800/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/2862184313":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/3621754630":"2024-03-05T08:48:22.593931+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T08:56:52.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.556+0000 7febdd165640 1 -- 172.21.15.71:0/2022870266 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7febc40a1d80 msgr2=0x7febc40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:52.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.556+0000 7febdd165640 1 --2- 172.21.15.71:0/2022870266 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7febc40a1d80 0x7febc40a4240 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7febd8102640 tx=0x7febd000d040 comp rx=0 tx=0).stop 2024-03-04T08:56:52.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.556+0000 7febdd165640 1 -- 172.21.15.71:0/2022870266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febd81031a0 msgr2=0x7febd819ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:52.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.556+0000 7febdd165640 1 --2- 172.21.15.71:0/2022870266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febd81031a0 0x7febd819ef00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7febc800e970 tx=0x7febc800ed70 comp rx=0 tx=0).stop 2024-03-04T08:56:52.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.556+0000 7febdd165640 1 -- 172.21.15.71:0/2022870266 shutdown_connections 2024-03-04T08:56:52.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.556+0000 7febdd165640 1 --2- 172.21.15.71:0/2022870266 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7febc40a1d80 0x7febc40a4240 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:52.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.556+0000 7febdd165640 1 --2- 172.21.15.71:0/2022870266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febd81031a0 0x7febd819ef00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:52.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.556+0000 7febdd165640 1 --2- 172.21.15.71:0/2022870266 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7febd81027e0 0x7febd8197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:52.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.556+0000 7febdd165640 1 --2- 172.21.15.71:0/2022870266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febd81015f0 0x7febd8197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:52.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.556+0000 7febdd165640 1 -- 172.21.15.71:0/2022870266 >> 172.21.15.71:0/2022870266 conn(0x7febd80fcb80 msgr2=0x7febd8105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:52.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.556+0000 7febdd165640 1 -- 172.21.15.71:0/2022870266 shutdown_connections 2024-03-04T08:56:52.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.556+0000 7febdd165640 1 -- 172.21.15.71:0/2022870266 wait complete. 2024-03-04T08:56:52.563 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T08:56:52.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.668+0000 7ff3fd335640 1 -- 172.21.15.71:0/3203198154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3f81020a0 msgr2=0x7ff3f8108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:52.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.668+0000 7ff3fd335640 1 --2- 172.21.15.71:0/3203198154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3f81020a0 0x7ff3f8108930 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0009900 tx=0x7ff3f002bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:52.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.668+0000 7ff3fd335640 1 -- 172.21.15.71:0/3203198154 shutdown_connections 2024-03-04T08:56:52.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.668+0000 7ff3fd335640 1 --2- 172.21.15.71:0/3203198154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3f81020a0 0x7ff3f8108930 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:52.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.668+0000 7ff3fd335640 1 --2- 172.21.15.71:0/3203198154 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff3f81016e0 0x7ff3f8101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:52.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.668+0000 7ff3fd335640 1 --2- 172.21.15.71:0/3203198154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3f81004f0 0x7ff3f8100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:52.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.668+0000 7ff3fd335640 1 -- 172.21.15.71:0/3203198154 >> 172.21.15.71:0/3203198154 conn(0x7ff3f80fba60 msgr2=0x7ff3f80fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:52.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.668+0000 7ff3fd335640 1 -- 172.21.15.71:0/3203198154 shutdown_connections 2024-03-04T08:56:52.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.668+0000 7ff3fd335640 1 -- 172.21.15.71:0/3203198154 wait complete. 2024-03-04T08:56:52.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.669+0000 7ff3fd335640 1 Processor -- start 2024-03-04T08:56:52.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.669+0000 7ff3fd335640 1 -- start start 2024-03-04T08:56:52.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.669+0000 7ff3fd335640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3f81004f0 0x7ff3f8069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:52.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.669+0000 7ff3fd335640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3f81016e0 0x7ff3f819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:52.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.669+0000 7ff3fd335640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff3f81020a0 0x7ff3f819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:52.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.669+0000 7ff3fd335640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f810c150 con 0x7ff3f81016e0 2024-03-04T08:56:52.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.669+0000 7ff3fd335640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f810bfd0 con 0x7ff3f81004f0 2024-03-04T08:56:52.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.669+0000 7ff3fd335640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f810c2d0 con 0x7ff3f81020a0 2024-03-04T08:56:52.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.669+0000 7ff3f67fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3f81016e0 0x7ff3f819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:52.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.669+0000 7ff3f6ffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3f81004f0 0x7ff3f8069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:52.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.669+0000 7ff3f67fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3f81016e0 0x7ff3f819c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52776/0 (socket says 172.21.15.71:52776) 2024-03-04T08:56:52.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.670+0000 7ff3f67fc640 1 -- 172.21.15.71:0/2497229463 learned_addr learned my addr 172.21.15.71:0/2497229463 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:52.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.670+0000 7ff3f77fe640 1 --2- 172.21.15.71:0/2497229463 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff3f81020a0 0x7ff3f819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:52.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.670+0000 7ff3f67fc640 1 -- 172.21.15.71:0/2497229463 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff3f81020a0 msgr2=0x7ff3f819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:52.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.670+0000 7ff3f67fc640 1 --2- 172.21.15.71:0/2497229463 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff3f81020a0 0x7ff3f819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:52.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.670+0000 7ff3f67fc640 1 -- 172.21.15.71:0/2497229463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3f81004f0 msgr2=0x7ff3f8069300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:52.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.670+0000 7ff3f67fc640 1 --2- 172.21.15.71:0/2497229463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3f81004f0 0x7ff3f8069300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:52.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.670+0000 7ff3f67fc640 1 -- 172.21.15.71:0/2497229463 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f819f4b0 con 0x7ff3f81016e0 2024-03-04T08:56:52.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.670+0000 7ff3f6ffd640 1 --2- 172.21.15.71:0/2497229463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3f81004f0 0x7ff3f8069300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:56:52.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.670+0000 7ff3f67fc640 1 --2- 172.21.15.71:0/2497229463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3f81016e0 0x7ff3f819c580 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00d950 tx=0x7ff3ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:52.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.670+0000 7ff3dffff640 1 -- 172.21.15.71:0/2497229463 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec017070 con 0x7ff3f81016e0 2024-03-04T08:56:52.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.670+0000 7ff3fd335640 1 -- 172.21.15.71:0/2497229463 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f819f7a0 con 0x7ff3f81016e0 2024-03-04T08:56:52.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.670+0000 7ff3dffff640 1 -- 172.21.15.71:0/2497229463 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3ec009c30 con 0x7ff3f81016e0 2024-03-04T08:56:52.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.670+0000 7ff3dffff640 1 -- 172.21.15.71:0/2497229463 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec00f6b0 con 0x7ff3f81016e0 2024-03-04T08:56:52.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.670+0000 7ff3fd335640 1 -- 172.21.15.71:0/2497229463 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f819fd40 con 0x7ff3f81016e0 2024-03-04T08:56:52.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.671+0000 7ff3fd335640 1 -- 172.21.15.71:0/2497229463 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f81077b0 con 0x7ff3f81016e0 2024-03-04T08:56:52.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.672+0000 7ff3dffff640 1 -- 172.21.15.71:0/2497229463 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7ff3ec0109d0 con 0x7ff3f81016e0 2024-03-04T08:56:52.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.673+0000 7ff3dffff640 1 --2- 172.21.15.71:0/2497229463 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff3e00a1e00 0x7ff3e00a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:52.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.673+0000 7ff3f6ffd640 1 --2- 172.21.15.71:0/2497229463 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff3e00a1e00 0x7ff3e00a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:52.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.673+0000 7ff3dffff640 1 -- 172.21.15.71:0/2497229463 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff3ec0cdc20 con 0x7ff3f81016e0 2024-03-04T08:56:52.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.673+0000 7ff3f6ffd640 1 --2- 172.21.15.71:0/2497229463 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff3e00a1e00 0x7ff3e00a42c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80099a0 tx=0x7ff3e8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:52.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.677+0000 7ff3dffff640 1 -- 172.21.15.71:0/2497229463 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff3ec009de0 con 0x7ff3f81016e0 2024-03-04T08:56:52.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.854+0000 7ff3fd335640 1 -- 172.21.15.71:0/2497229463 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3f80621d0 con 0x7ff3f81016e0 2024-03-04T08:56:52.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.855+0000 7ff3dffff640 1 -- 172.21.15.71:0/2497229463 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24153 (secure 0 0 0) 0x7ff3ec0b8e20 con 0x7ff3f81016e0 2024-03-04T08:56:52.852 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:52.852 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":91,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","created":"2024-03-04T08:47:28.546102+0000","modified":"2024-03-04T08:56:35.007858+0000","last_up_change":"2024-03-04T08:55:30.744961+0000","last_in_change":"2024-03-04T08:55:11.570990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T08:51:14.933487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T08:56:28.253074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T08:56:29.543622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T08:56:32.884220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7e43b47d-8888-4631-bc96-8a96fcf8cd46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6803","nonce":1173641530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6805","nonce":1173641530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6809","nonce":1173641530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6807","nonce":1173641530}]},"public_addr":"172.21.15.71:6803/1173641530","cluster_addr":"172.21.15.71:6805/1173641530","heartbeat_back_addr":"172.21.15.71:6809/1173641530","heartbeat_front_addr":"172.21.15.71:6807/1173641530","state":["exists","up"]},{"osd":1,"uuid":"1209f214-f2e2-4209-bf69-4528a43a2123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6801","nonce":3383948110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6803","nonce":3383948110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6807","nonce":3383948110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6805","nonce":3383948110}]},"public_addr":"172.21.15.150:6801/3383948110","cluster_addr":"172.21.15.150:6803/3383948110","heartbeat_back_addr":"172.21.15.150:6807/3383948110","heartbeat_front_addr":"172.21.15.150:6805/3383948110","state":["exists","up"]},{"osd":2,"uuid":"b88c3082-93a9-4c9b-9221-a7105a44c91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6801","nonce":586944144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6803","nonce":586944144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6807","nonce":586944144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6805","nonce":586944144}]},"public_addr":"172.21.15.158:6801/586944144","cluster_addr":"172.21.15.158:6803/586944144","heartbeat_back_addr":"172.21.15.158:6807/586944144","heartbeat_front_addr":"172.21.15.158:6805/586944144","state":["exists","up"]},{"osd":3,"uuid":"e4fc7723-fd73-40b2-b9d0-7f9bda282ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6811","nonce":1311903262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6813","nonce":1311903262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6817","nonce":1311903262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6815","nonce":1311903262}]},"public_addr":"172.21.15.71:6811/1311903262","cluster_addr":"172.21.15.71:6813/1311903262","heartbeat_back_addr":"172.21.15.71:6817/1311903262","heartbeat_front_addr":"172.21.15.71:6815/1311903262","state":["exists","up"]},{"osd":4,"uuid":"e7eb72ac-f066-4ccd-b2bf-22a6a26daf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6809","nonce":2300837573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6811","nonce":2300837573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6815","nonce":2300837573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6813","nonce":2300837573}]},"public_addr":"172.21.15.150:6809/2300837573","cluster_addr":"172.21.15.150:6811/2300837573","heartbeat_back_addr":"172.21.15.150:6815/2300837573","heartbeat_front_addr":"172.21.15.150:6813/2300837573","state":["exists","up"]},{"osd":5,"uuid":"9b295840-e104-4a66-9cc4-7020627c1765","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6809","nonce":1922694045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6811","nonce":1922694045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6815","nonce":1922694045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6813","nonce":1922694045}]},"public_addr":"172.21.15.158:6809/1922694045","cluster_addr":"172.21.15.158:6811/1922694045","heartbeat_back_addr":"172.21.15.158:6815/1922694045","heartbeat_front_addr":"172.21.15.158:6813/1922694045","state":["exists","up"]},{"osd":6,"uuid":"6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6819","nonce":3848842707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6821","nonce":3848842707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6825","nonce":3848842707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6823","nonce":3848842707}]},"public_addr":"172.21.15.71:6819/3848842707","cluster_addr":"172.21.15.71:6821/3848842707","heartbeat_back_addr":"172.21.15.71:6825/3848842707","heartbeat_front_addr":"172.21.15.71:6823/3848842707","state":["exists","up"]},{"osd":7,"uuid":"6160a40a-d30f-4569-949c-0afb1bb3d133","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6817","nonce":1328330305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6819","nonce":1328330305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6823","nonce":1328330305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6821","nonce":1328330305}]},"public_addr":"172.21.15.150:6817/1328330305","cluster_addr":"172.21.15.150:6819/1328330305","heartbeat_back_addr":"172.21.15.150:6823/1328330305","heartbeat_front_addr":"172.21.15.150:6821/1328330305","state":["exists","up"]},{"osd":8,"uuid":"95ed2341-84a4-465c-9948-ac667680ce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6817","nonce":2195304577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6819","nonce":2195304577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6823","nonce":2195304577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6821","nonce":2195304577}]},"public_addr":"172.21.15.158:6817/2195304577","cluster_addr":"172.21.15.158:6819/2195304577","heartbeat_back_addr":"172.21.15.158:6823/2195304577","heartbeat_front_addr":"172.21.15.158:6821/2195304577","state":["exists","up"]},{"osd":9,"uuid":"088f6419-63a0-4768-afae-dbb8a5f5a565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6827","nonce":1913779347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6829","nonce":1913779347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6833","nonce":1913779347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6831","nonce":1913779347}]},"public_addr":"172.21.15.71:6827/1913779347","cluster_addr":"172.21.15.71:6829/1913779347","heartbeat_back_addr":"172.21.15.71:6833/1913779347","heartbeat_front_addr":"172.21.15.71:6831/1913779347","state":["exists","up"]},{"osd":10,"uuid":"18c85e24-068d-4f1e-a3b8-f7915b363234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6825","nonce":3295420996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6827","nonce":3295420996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6831","nonce":3295420996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6829","nonce":3295420996}]},"public_addr":"172.21.15.150:6825/3295420996","cluster_addr":"172.21.15.150:6827/3295420996","heartbeat_back_addr":"172.21.15.150:6831/3295420996","heartbeat_front_addr":"172.21.15.150:6829/3295420996","state":["exists","up"]},{"osd":11,"uuid":"4f6b3ddf-c482-4740-b1fd-de836e047c2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6825","nonce":3332083865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6827","nonce":3332083865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6831","nonce":3332083865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6829","nonce":3332083865}]},"public_addr":"172.21.15.158:6825/3332083865","cluster_addr":"172.21.15.158:6827/3332083865","heartbeat_back_addr":"172.21.15.158:6831/3332083865","heartbeat_front_addr":"172.21.15.158:6829/3332083865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:05.997832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:29.314235+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:08.819465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:41.194275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:06.032479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:38.814498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:13.067960+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:38.568479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:03.416713+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:38.349479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:04.096945+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:29.425250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1287712358":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/3770452860":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/718743094":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/1904717532":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:6800/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6800/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/2862184313":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/3621754630":"2024-03-05T08:48:22.593931+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T08:56:52.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.857+0000 7ff3fd335640 1 -- 172.21.15.71:0/2497229463 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff3e00a1e00 msgr2=0x7ff3e00a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:52.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.857+0000 7ff3fd335640 1 --2- 172.21.15.71:0/2497229463 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff3e00a1e00 0x7ff3e00a42c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80099a0 tx=0x7ff3e8009500 comp rx=0 tx=0).stop 2024-03-04T08:56:52.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.858+0000 7ff3fd335640 1 -- 172.21.15.71:0/2497229463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3f81016e0 msgr2=0x7ff3f819c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:52.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.858+0000 7ff3fd335640 1 --2- 172.21.15.71:0/2497229463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3f81016e0 0x7ff3f819c580 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00d950 tx=0x7ff3ec00de10 comp rx=0 tx=0).stop 2024-03-04T08:56:52.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.858+0000 7ff3fd335640 1 -- 172.21.15.71:0/2497229463 shutdown_connections 2024-03-04T08:56:52.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.858+0000 7ff3fd335640 1 --2- 172.21.15.71:0/2497229463 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff3e00a1e00 0x7ff3e00a42c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:52.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.858+0000 7ff3fd335640 1 --2- 172.21.15.71:0/2497229463 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff3f81020a0 0x7ff3f819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:52.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.858+0000 7ff3fd335640 1 --2- 172.21.15.71:0/2497229463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3f81016e0 0x7ff3f819c580 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:52.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.858+0000 7ff3fd335640 1 --2- 172.21.15.71:0/2497229463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3f81004f0 0x7ff3f8069300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:52.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.858+0000 7ff3fd335640 1 -- 172.21.15.71:0/2497229463 >> 172.21.15.71:0/2497229463 conn(0x7ff3f80fba60 msgr2=0x7ff3f8104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:52.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.858+0000 7ff3fd335640 1 -- 172.21.15.71:0/2497229463 shutdown_connections 2024-03-04T08:56:52.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.858+0000 7ff3fd335640 1 -- 172.21.15.71:0/2497229463 wait complete. 2024-03-04T08:56:52.864 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:52 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2022870266' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:52.865 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-04T08:56:52.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.974+0000 7f4c102b5640 1 -- 172.21.15.71:0/1298363737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c081004d0 msgr2=0x7f4c081008f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:52.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.974+0000 7f4c102b5640 1 --2- 172.21.15.71:0/1298363737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c081004d0 0x7f4c081008f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc009860 tx=0x7f4bfc02bb70 comp rx=0 tx=0).stop 2024-03-04T08:56:52.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.974+0000 7f4c102b5640 1 -- 172.21.15.71:0/1298363737 shutdown_connections 2024-03-04T08:56:52.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.974+0000 7f4c102b5640 1 --2- 172.21.15.71:0/1298363737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c08102080 0x7f4c08108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:52.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.974+0000 7f4c102b5640 1 --2- 172.21.15.71:0/1298363737 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c081016c0 0x7f4c08101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:52.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.974+0000 7f4c102b5640 1 --2- 172.21.15.71:0/1298363737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c081004d0 0x7f4c081008f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:52.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.974+0000 7f4c102b5640 1 -- 172.21.15.71:0/1298363737 >> 172.21.15.71:0/1298363737 conn(0x7f4c080fba60 msgr2=0x7f4c080fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:52.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.974+0000 7f4c102b5640 1 -- 172.21.15.71:0/1298363737 shutdown_connections 2024-03-04T08:56:52.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.974+0000 7f4c102b5640 1 -- 172.21.15.71:0/1298363737 wait complete. 2024-03-04T08:56:52.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.974+0000 7f4c102b5640 1 Processor -- start 2024-03-04T08:56:52.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.974+0000 7f4c102b5640 1 -- start start 2024-03-04T08:56:52.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.975+0000 7f4c102b5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c081004d0 0x7f4c081978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:52.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.975+0000 7f4c102b5640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c081016c0 0x7f4c08197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:52.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.975+0000 7f4c102b5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c08102080 0x7f4c0819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:52.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.975+0000 7f4c102b5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0810d270 con 0x7f4c08102080 2024-03-04T08:56:52.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.975+0000 7f4c102b5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0810d0f0 con 0x7f4c081004d0 2024-03-04T08:56:52.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.975+0000 7f4c102b5640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0810d3f0 con 0x7f4c081016c0 2024-03-04T08:56:52.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.975+0000 7f4c0e852640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c08102080 0x7f4c0819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:52.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.975+0000 7f4c0e852640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c08102080 0x7f4c0819ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52790/0 (socket says 172.21.15.71:52790) 2024-03-04T08:56:52.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.975+0000 7f4c0e852640 1 -- 172.21.15.71:0/1144598329 learned_addr learned my addr 172.21.15.71:0/1144598329 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:52.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.975+0000 7f4c0e051640 1 --2- 172.21.15.71:0/1144598329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c081004d0 0x7f4c081978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:52.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.975+0000 7f4c0d850640 1 --2- 172.21.15.71:0/1144598329 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c081016c0 0x7f4c08197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:52.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.975+0000 7f4c0e852640 1 -- 172.21.15.71:0/1144598329 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c081016c0 msgr2=0x7f4c08197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:52.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.975+0000 7f4c0e852640 1 --2- 172.21.15.71:0/1144598329 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c081016c0 0x7f4c08197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:52.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.975+0000 7f4c0e852640 1 -- 172.21.15.71:0/1144598329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c081004d0 msgr2=0x7f4c081978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:52.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.975+0000 7f4c0e852640 1 --2- 172.21.15.71:0/1144598329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c081004d0 0x7f4c081978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:52.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.975+0000 7f4c0e852640 1 -- 172.21.15.71:0/1144598329 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0819f4f0 con 0x7f4c08102080 2024-03-04T08:56:52.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.975+0000 7f4c0e852640 1 --2- 172.21.15.71:0/1144598329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c08102080 0x7f4c0819ee80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009fd0 tx=0x7f4c0000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:52.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.975+0000 7f4bfb7fe640 1 -- 172.21.15.71:0/1144598329 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c00004b20 con 0x7f4c08102080 2024-03-04T08:56:52.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.975+0000 7f4c102b5640 1 -- 172.21.15.71:0/1144598329 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0819f7e0 con 0x7f4c08102080 2024-03-04T08:56:52.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.975+0000 7f4bfb7fe640 1 -- 172.21.15.71:0/1144598329 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c00003e50 con 0x7f4c08102080 2024-03-04T08:56:52.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.975+0000 7f4bfb7fe640 1 -- 172.21.15.71:0/1144598329 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0000b630 con 0x7f4c08102080 2024-03-04T08:56:52.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.975+0000 7f4c102b5640 1 -- 172.21.15.71:0/1144598329 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0819fd30 con 0x7f4c08102080 2024-03-04T08:56:52.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.976+0000 7f4c102b5640 1 -- 172.21.15.71:0/1144598329 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c081077f0 con 0x7f4c08102080 2024-03-04T08:56:52.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.977+0000 7f4bfb7fe640 1 -- 172.21.15.71:0/1144598329 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f4c0001d020 con 0x7f4c08102080 2024-03-04T08:56:52.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.978+0000 7f4bfb7fe640 1 --2- 172.21.15.71:0/1144598329 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f4bf40a1d80 0x7f4bf40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:52.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.978+0000 7f4c0e051640 1 --2- 172.21.15.71:0/1144598329 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f4bf40a1d80 0x7f4bf40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:52.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.978+0000 7f4bfb7fe640 1 -- 172.21.15.71:0/1144598329 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f4c000cea40 con 0x7f4c08102080 2024-03-04T08:56:52.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.978+0000 7f4c0e051640 1 --2- 172.21.15.71:0/1144598329 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f4bf40a1d80 0x7f4bf40a4240 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc009830 tx=0x7f4bfc034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:52.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:52.981+0000 7f4bfb7fe640 1 -- 172.21.15.71:0/1144598329 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4c00024030 con 0x7f4c08102080 2024-03-04T08:56:53.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:52 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2022870266' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:53.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:52 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2022870266' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:53.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.193+0000 7f4c102b5640 1 -- 172.21.15.71:0/1144598329 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4c080621d0 con 0x7f4c08102080 2024-03-04T08:56:53.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.737+0000 7f4bfb7fe640 1 -- 172.21.15.71:0/1144598329 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4c0009bc30 con 0x7f4c08102080 2024-03-04T08:56:53.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.739+0000 7f4c102b5640 1 -- 172.21.15.71:0/1144598329 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f4bf40a1d80 msgr2=0x7f4bf40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:53.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.739+0000 7f4c102b5640 1 --2- 172.21.15.71:0/1144598329 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f4bf40a1d80 0x7f4bf40a4240 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc009830 tx=0x7f4bfc034ad0 comp rx=0 tx=0).stop 2024-03-04T08:56:53.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.739+0000 7f4c102b5640 1 -- 172.21.15.71:0/1144598329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c08102080 msgr2=0x7f4c0819ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:53.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.739+0000 7f4c102b5640 1 --2- 172.21.15.71:0/1144598329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c08102080 0x7f4c0819ee80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009fd0 tx=0x7f4c0000c800 comp rx=0 tx=0).stop 2024-03-04T08:56:53.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.739+0000 7f4c102b5640 1 -- 172.21.15.71:0/1144598329 shutdown_connections 2024-03-04T08:56:53.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.739+0000 7f4c102b5640 1 --2- 172.21.15.71:0/1144598329 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f4bf40a1d80 0x7f4bf40a4240 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:53.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.739+0000 7f4c102b5640 1 --2- 172.21.15.71:0/1144598329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c08102080 0x7f4c0819ee80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:53.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.739+0000 7f4c102b5640 1 --2- 172.21.15.71:0/1144598329 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4c081016c0 0x7f4c08197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:53.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.739+0000 7f4c102b5640 1 --2- 172.21.15.71:0/1144598329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c081004d0 0x7f4c081978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:53.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.739+0000 7f4c102b5640 1 -- 172.21.15.71:0/1144598329 >> 172.21.15.71:0/1144598329 conn(0x7f4c080fba60 msgr2=0x7f4c081049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:53.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.739+0000 7f4c102b5640 1 -- 172.21.15.71:0/1144598329 shutdown_connections 2024-03-04T08:56:53.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.739+0000 7f4c102b5640 1 -- 172.21.15.71:0/1144598329 wait complete. 2024-03-04T08:56:53.748 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-04T08:56:53.748 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-04T08:56:53.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.859+0000 7f528c356640 1 -- 172.21.15.71:0/651597394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52841004e0 msgr2=0x7f5284100900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:53.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.859+0000 7f528c356640 1 --2- 172.21.15.71:0/651597394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52841004e0 0x7f5284100900 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5280009900 tx=0x7f528002bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:53.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.859+0000 7f528c356640 1 -- 172.21.15.71:0/651597394 shutdown_connections 2024-03-04T08:56:53.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.859+0000 7f528c356640 1 --2- 172.21.15.71:0/651597394 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5284102090 0x7f5284108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:53.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.859+0000 7f528c356640 1 --2- 172.21.15.71:0/651597394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52841016d0 0x7f5284101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:53.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.859+0000 7f528c356640 1 --2- 172.21.15.71:0/651597394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52841004e0 0x7f5284100900 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:53.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.859+0000 7f528c356640 1 -- 172.21.15.71:0/651597394 >> 172.21.15.71:0/651597394 conn(0x7f52840fba70 msgr2=0x7f52840fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:53.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.859+0000 7f528c356640 1 -- 172.21.15.71:0/651597394 shutdown_connections 2024-03-04T08:56:53.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.859+0000 7f528c356640 1 -- 172.21.15.71:0/651597394 wait complete. 2024-03-04T08:56:53.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.859+0000 7f528c356640 1 Processor -- start 2024-03-04T08:56:53.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.859+0000 7f528c356640 1 -- start start 2024-03-04T08:56:53.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.860+0000 7f528c356640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f52841004e0 0x7f5284197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:53.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.860+0000 7f528c356640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52841016d0 0x7f5284197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:53.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.860+0000 7f528c356640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5284102090 0x7f528419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:53.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.860+0000 7f528c356640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528410d2b0 con 0x7f5284102090 2024-03-04T08:56:53.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.860+0000 7f528c356640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528410d130 con 0x7f52841016d0 2024-03-04T08:56:53.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.860+0000 7f528c356640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528410d430 con 0x7f52841004e0 2024-03-04T08:56:53.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.860+0000 7f528a8f3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5284102090 0x7f528419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:53.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.860+0000 7f52898f1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52841016d0 0x7f5284197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:53.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.860+0000 7f52898f1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52841016d0 0x7f5284197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.71:59526/0 (socket says 172.21.15.71:59526) 2024-03-04T08:56:53.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.860+0000 7f52898f1640 1 -- 172.21.15.71:0/170099365 learned_addr learned my addr 172.21.15.71:0/170099365 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:53.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.860+0000 7f528a8f3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5284102090 0x7f528419ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52804/0 (socket says 172.21.15.71:52804) 2024-03-04T08:56:53.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.860+0000 7f528a0f2640 1 --2- 172.21.15.71:0/170099365 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f52841004e0 0x7f5284197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:53.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.860+0000 7f52898f1640 1 -- 172.21.15.71:0/170099365 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f52841004e0 msgr2=0x7f5284197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:53.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.860+0000 7f52898f1640 1 --2- 172.21.15.71:0/170099365 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f52841004e0 0x7f5284197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:53.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.860+0000 7f52898f1640 1 -- 172.21.15.71:0/170099365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5284102090 msgr2=0x7f528419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:53.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.860+0000 7f52898f1640 1 --2- 172.21.15.71:0/170099365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5284102090 0x7f528419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:53.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.860+0000 7f52898f1640 1 -- 172.21.15.71:0/170099365 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528419f5b0 con 0x7f52841016d0 2024-03-04T08:56:53.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.860+0000 7f528a8f3640 1 --2- 172.21.15.71:0/170099365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5284102090 0x7f528419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:56:53.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.860+0000 7f52898f1640 1 --2- 172.21.15.71:0/170099365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52841016d0 0x7f5284197ed0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f527400e9f0 tx=0x7f527400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:53.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.860+0000 7f527b7fe640 1 -- 172.21.15.71:0/170099365 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527400cd80 con 0x7f52841016d0 2024-03-04T08:56:53.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.860+0000 7f527b7fe640 1 -- 172.21.15.71:0/170099365 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5274010910 con 0x7f52841016d0 2024-03-04T08:56:53.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.860+0000 7f528c356640 1 -- 172.21.15.71:0/170099365 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f528419f8a0 con 0x7f52841016d0 2024-03-04T08:56:53.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.861+0000 7f527b7fe640 1 -- 172.21.15.71:0/170099365 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274018990 con 0x7f52841016d0 2024-03-04T08:56:53.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.861+0000 7f528c356640 1 -- 172.21.15.71:0/170099365 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528419fdf0 con 0x7f52841016d0 2024-03-04T08:56:53.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.861+0000 7f528c356640 1 -- 172.21.15.71:0/170099365 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5284107880 con 0x7f52841016d0 2024-03-04T08:56:53.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.862+0000 7f527b7fe640 1 -- 172.21.15.71:0/170099365 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f5274010ac0 con 0x7f52841016d0 2024-03-04T08:56:53.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.863+0000 7f527b7fe640 1 --2- 172.21.15.71:0/170099365 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f52700a1d80 0x7f52700a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:53.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.863+0000 7f528a0f2640 1 --2- 172.21.15.71:0/170099365 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f52700a1d80 0x7f52700a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:53.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.863+0000 7f527b7fe640 1 -- 172.21.15.71:0/170099365 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f52740ce300 con 0x7f52841016d0 2024-03-04T08:56:53.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.863+0000 7f528a0f2640 1 --2- 172.21.15.71:0/170099365 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f52700a1d80 0x7f52700a4240 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5280000c00 tx=0x7f5280005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:53.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:53.868+0000 7f527b7fe640 1 -- 172.21.15.71:0/170099365 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f527409aea0 con 0x7f52841016d0 2024-03-04T08:56:54.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:53 smithi150 ceph-mon[36807]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-04T08:56:54.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:53 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2497229463' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:54.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:53 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1144598329' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-04T08:56:54.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:53 smithi158 ceph-mon[37241]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-04T08:56:54.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:53 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2497229463' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:54.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:53 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1144598329' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-04T08:56:54.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.081+0000 7f528c356640 1 -- 172.21.15.71:0/170099365 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f52840621d0 con 0x7f52841016d0 2024-03-04T08:56:54.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:53 smithi071 ceph-mon[32789]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-04T08:56:54.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:53 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2497229463' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:54.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:53 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1144598329' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-04T08:56:54.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.553+0000 7f527b7fe640 1 -- 172.21.15.71:0/170099365 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) v1 ==== 121+0+0 (secure 0 0 0) 0x7f527409ef40 con 0x7f52841016d0 2024-03-04T08:56:54.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.556+0000 7f528c356640 1 -- 172.21.15.71:0/170099365 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f52700a1d80 msgr2=0x7f52700a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:54.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.556+0000 7f528c356640 1 --2- 172.21.15.71:0/170099365 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f52700a1d80 0x7f52700a4240 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5280000c00 tx=0x7f5280005fd0 comp rx=0 tx=0).stop 2024-03-04T08:56:54.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.556+0000 7f528c356640 1 -- 172.21.15.71:0/170099365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52841016d0 msgr2=0x7f5284197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:54.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.556+0000 7f528c356640 1 --2- 172.21.15.71:0/170099365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52841016d0 0x7f5284197ed0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f527400e9f0 tx=0x7f527400eeb0 comp rx=0 tx=0).stop 2024-03-04T08:56:54.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.556+0000 7f528c356640 1 -- 172.21.15.71:0/170099365 shutdown_connections 2024-03-04T08:56:54.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.556+0000 7f528c356640 1 --2- 172.21.15.71:0/170099365 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f52700a1d80 0x7f52700a4240 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:54.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.556+0000 7f528c356640 1 --2- 172.21.15.71:0/170099365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5284102090 0x7f528419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:54.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.556+0000 7f528c356640 1 --2- 172.21.15.71:0/170099365 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52841016d0 0x7f5284197ed0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:54.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.556+0000 7f528c356640 1 --2- 172.21.15.71:0/170099365 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f52841004e0 0x7f5284197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:54.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.556+0000 7f528c356640 1 -- 172.21.15.71:0/170099365 >> 172.21.15.71:0/170099365 conn(0x7f52840fba70 msgr2=0x7f5284104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:54.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.556+0000 7f528c356640 1 -- 172.21.15.71:0/170099365 shutdown_connections 2024-03-04T08:56:54.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.556+0000 7f528c356640 1 -- 172.21.15.71:0/170099365 wait complete. 2024-03-04T08:56:54.564 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-04T08:56:54.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.677+0000 7f3f20ac2640 1 -- 172.21.15.71:0/3853558905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f1c1020b0 msgr2=0x7f3f1c1044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:54.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.677+0000 7f3f20ac2640 1 --2- 172.21.15.71:0/3853558905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f1c1020b0 0x7f3f1c1044a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009ca0 tx=0x7f3f0c02c0f0 comp rx=0 tx=0).stop 2024-03-04T08:56:54.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.678+0000 7f3f20ac2640 1 -- 172.21.15.71:0/3853558905 shutdown_connections 2024-03-04T08:56:54.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.678+0000 7f3f20ac2640 1 --2- 172.21.15.71:0/3853558905 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f1c107310 0x7f3f1c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:54.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.678+0000 7f3f20ac2640 1 --2- 172.21.15.71:0/3853558905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f1c1049e0 0x7f3f1c106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:54.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.678+0000 7f3f20ac2640 1 --2- 172.21.15.71:0/3853558905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f1c1020b0 0x7f3f1c1044a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:54.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.678+0000 7f3f20ac2640 1 -- 172.21.15.71:0/3853558905 >> 172.21.15.71:0/3853558905 conn(0x7f3f1c0fba60 msgr2=0x7f3f1c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:54.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.678+0000 7f3f20ac2640 1 -- 172.21.15.71:0/3853558905 shutdown_connections 2024-03-04T08:56:54.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.678+0000 7f3f20ac2640 1 -- 172.21.15.71:0/3853558905 wait complete. 2024-03-04T08:56:54.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.678+0000 7f3f20ac2640 1 Processor -- start 2024-03-04T08:56:54.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.678+0000 7f3f20ac2640 1 -- start start 2024-03-04T08:56:54.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.678+0000 7f3f20ac2640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f1c1020b0 0x7f3f1c197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:54.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.678+0000 7f3f20ac2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f1c1049e0 0x7f3f1c197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:54.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.678+0000 7f3f20ac2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f1c107310 0x7f3f1c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:54.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.679+0000 7f3f20ac2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c10d260 con 0x7f3f1c1049e0 2024-03-04T08:56:54.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.679+0000 7f3f20ac2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c10d0e0 con 0x7f3f1c107310 2024-03-04T08:56:54.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.679+0000 7f3f20ac2640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c10d3e0 con 0x7f3f1c1020b0 2024-03-04T08:56:54.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.679+0000 7f3f19d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f1c1049e0 0x7f3f1c197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:54.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.679+0000 7f3f19d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f1c1049e0 0x7f3f1c197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52828/0 (socket says 172.21.15.71:52828) 2024-03-04T08:56:54.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.679+0000 7f3f1a59c640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f1c1020b0 0x7f3f1c197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:54.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.679+0000 7f3f19d9b640 1 -- 172.21.15.71:0/4060558525 learned_addr learned my addr 172.21.15.71:0/4060558525 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:54.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.679+0000 7f3f1ad9d640 1 --2- 172.21.15.71:0/4060558525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f1c107310 0x7f3f1c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:54.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.679+0000 7f3f19d9b640 1 -- 172.21.15.71:0/4060558525 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f1c1020b0 msgr2=0x7f3f1c197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:54.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.679+0000 7f3f19d9b640 1 --2- 172.21.15.71:0/4060558525 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f1c1020b0 0x7f3f1c197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:54.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.679+0000 7f3f19d9b640 1 -- 172.21.15.71:0/4060558525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f1c107310 msgr2=0x7f3f1c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:54.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.679+0000 7f3f19d9b640 1 --2- 172.21.15.71:0/4060558525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f1c107310 0x7f3f1c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:54.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.679+0000 7f3f19d9b640 1 -- 172.21.15.71:0/4060558525 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c19f550 con 0x7f3f1c1049e0 2024-03-04T08:56:54.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.679+0000 7f3f19d9b640 1 --2- 172.21.15.71:0/4060558525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f1c1049e0 0x7f3f1c197e70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3f1000e9c0 tx=0x7f3f1000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:54.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.679+0000 7f3f0b7fe640 1 -- 172.21.15.71:0/4060558525 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1000cd80 con 0x7f3f1c1049e0 2024-03-04T08:56:54.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.679+0000 7f3f0b7fe640 1 -- 172.21.15.71:0/4060558525 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f10005500 con 0x7f3f1c1049e0 2024-03-04T08:56:54.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.679+0000 7f3f20ac2640 1 -- 172.21.15.71:0/4060558525 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1c19f840 con 0x7f3f1c1049e0 2024-03-04T08:56:54.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.679+0000 7f3f0b7fe640 1 -- 172.21.15.71:0/4060558525 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10010690 con 0x7f3f1c1049e0 2024-03-04T08:56:54.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.679+0000 7f3f20ac2640 1 -- 172.21.15.71:0/4060558525 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1c19fd90 con 0x7f3f1c1049e0 2024-03-04T08:56:54.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.680+0000 7f3f20ac2640 1 -- 172.21.15.71:0/4060558525 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f1c0fd550 con 0x7f3f1c1049e0 2024-03-04T08:56:54.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.681+0000 7f3f0b7fe640 1 -- 172.21.15.71:0/4060558525 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f3f100046c0 con 0x7f3f1c1049e0 2024-03-04T08:56:54.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.682+0000 7f3f0b7fe640 1 --2- 172.21.15.71:0/4060558525 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3f040a1d80 0x7f3f040a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:54.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.682+0000 7f3f1a59c640 1 --2- 172.21.15.71:0/4060558525 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3f040a1d80 0x7f3f040a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:54.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.682+0000 7f3f0b7fe640 1 -- 172.21.15.71:0/4060558525 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f3f100ce360 con 0x7f3f1c1049e0 2024-03-04T08:56:54.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.682+0000 7f3f1a59c640 1 --2- 172.21.15.71:0/4060558525 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3f040a1d80 0x7f3f040a4240 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c006010 tx=0x7f3f0c032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:54.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.687+0000 7f3f0b7fe640 1 -- 172.21.15.71:0/4060558525 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3f1009b550 con 0x7f3f1c1049e0 2024-03-04T08:56:54.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:54.899+0000 7f3f20ac2640 1 -- 172.21.15.71:0/4060558525 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f3f1c0621d0 con 0x7f3f1c1049e0 2024-03-04T08:56:54.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:54 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1144598329' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-04T08:56:54.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:54 smithi071 ceph-mon[32789]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T08:56:54.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:54 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/170099365' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-04T08:56:54.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:54 smithi071 ceph-mon[32789]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-04T08:56:54.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:54 smithi071 ceph-mon[32789]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-04T08:56:54.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:54 smithi071 ceph-mon[32789]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-03-04T08:56:55.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:54 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1144598329' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-04T08:56:55.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:54 smithi150 ceph-mon[36807]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T08:56:55.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:54 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/170099365' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-04T08:56:55.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:54 smithi150 ceph-mon[36807]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-04T08:56:55.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:54 smithi150 ceph-mon[36807]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-04T08:56:55.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:54 smithi150 ceph-mon[36807]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-03-04T08:56:55.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:54 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1144598329' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-04T08:56:55.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:54 smithi158 ceph-mon[37241]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T08:56:55.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:54 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/170099365' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-04T08:56:55.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:54 smithi158 ceph-mon[37241]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-04T08:56:55.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:54 smithi158 ceph-mon[37241]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-04T08:56:55.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:54 smithi158 ceph-mon[37241]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-03-04T08:56:55.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.744+0000 7f3f0b7fe640 1 -- 172.21.15.71:0/4060558525 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) v1 ==== 115+0+0 (secure 0 0 0) 0x7f3f1009ae50 con 0x7f3f1c1049e0 2024-03-04T08:56:55.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.748+0000 7f3f20ac2640 1 -- 172.21.15.71:0/4060558525 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3f040a1d80 msgr2=0x7f3f040a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:55.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.748+0000 7f3f20ac2640 1 --2- 172.21.15.71:0/4060558525 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3f040a1d80 0x7f3f040a4240 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c006010 tx=0x7f3f0c032590 comp rx=0 tx=0).stop 2024-03-04T08:56:55.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.748+0000 7f3f20ac2640 1 -- 172.21.15.71:0/4060558525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f1c1049e0 msgr2=0x7f3f1c197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:55.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.748+0000 7f3f20ac2640 1 --2- 172.21.15.71:0/4060558525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f1c1049e0 0x7f3f1c197e70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3f1000e9c0 tx=0x7f3f1000ee80 comp rx=0 tx=0).stop 2024-03-04T08:56:55.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.748+0000 7f3f20ac2640 1 -- 172.21.15.71:0/4060558525 shutdown_connections 2024-03-04T08:56:55.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.748+0000 7f3f20ac2640 1 --2- 172.21.15.71:0/4060558525 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3f040a1d80 0x7f3f040a4240 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:55.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.748+0000 7f3f20ac2640 1 --2- 172.21.15.71:0/4060558525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f1c107310 0x7f3f1c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:55.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.748+0000 7f3f20ac2640 1 --2- 172.21.15.71:0/4060558525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f1c1049e0 0x7f3f1c197e70 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:55.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.748+0000 7f3f20ac2640 1 --2- 172.21.15.71:0/4060558525 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f1c1020b0 0x7f3f1c197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:55.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.748+0000 7f3f20ac2640 1 -- 172.21.15.71:0/4060558525 >> 172.21.15.71:0/4060558525 conn(0x7f3f1c0fba60 msgr2=0x7f3f1c102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:55.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.748+0000 7f3f20ac2640 1 -- 172.21.15.71:0/4060558525 shutdown_connections 2024-03-04T08:56:55.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.748+0000 7f3f20ac2640 1 -- 172.21.15.71:0/4060558525 wait complete. 2024-03-04T08:56:55.760 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-03-04T08:56:55.761 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-04T08:56:55.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.874+0000 7fb18f8ec640 1 -- 172.21.15.71:0/1253896851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb188102080 msgr2=0x7fb188108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:55.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.874+0000 7fb18f8ec640 1 --2- 172.21.15.71:0/1253896851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb188102080 0x7fb188108910 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb180009ca0 tx=0x7fb18002c0f0 comp rx=0 tx=0).stop 2024-03-04T08:56:55.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.874+0000 7fb18f8ec640 1 -- 172.21.15.71:0/1253896851 shutdown_connections 2024-03-04T08:56:55.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.874+0000 7fb18f8ec640 1 --2- 172.21.15.71:0/1253896851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb188102080 0x7fb188108910 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:55.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.874+0000 7fb18f8ec640 1 --2- 172.21.15.71:0/1253896851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1881016c0 0x7fb188101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:55.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.874+0000 7fb18f8ec640 1 --2- 172.21.15.71:0/1253896851 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb1881004d0 0x7fb1881008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:55.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.874+0000 7fb18f8ec640 1 -- 172.21.15.71:0/1253896851 >> 172.21.15.71:0/1253896851 conn(0x7fb1880fba60 msgr2=0x7fb1880fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:55.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.874+0000 7fb18f8ec640 1 -- 172.21.15.71:0/1253896851 shutdown_connections 2024-03-04T08:56:55.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.874+0000 7fb18f8ec640 1 -- 172.21.15.71:0/1253896851 wait complete. 2024-03-04T08:56:55.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.875+0000 7fb18f8ec640 1 Processor -- start 2024-03-04T08:56:55.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.875+0000 7fb18f8ec640 1 -- start start 2024-03-04T08:56:55.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.875+0000 7fb18f8ec640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1881004d0 0x7fb1880692a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:55.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.875+0000 7fb18f8ec640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb1881016c0 0x7fb18819c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:55.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.875+0000 7fb18f8ec640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb188102080 0x7fb18819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:55.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.875+0000 7fb18f8ec640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18810c130 con 0x7fb188102080 2024-03-04T08:56:55.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.875+0000 7fb18f8ec640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18810bfb0 con 0x7fb1881004d0 2024-03-04T08:56:55.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.875+0000 7fb18f8ec640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18810c2b0 con 0x7fb1881016c0 2024-03-04T08:56:55.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.875+0000 7fb18de89640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb188102080 0x7fb18819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:55.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.875+0000 7fb18de89640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb188102080 0x7fb18819ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52176/0 (socket says 172.21.15.71:52176) 2024-03-04T08:56:55.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.875+0000 7fb18de89640 1 -- 172.21.15.71:0/3934998806 learned_addr learned my addr 172.21.15.71:0/3934998806 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:55.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.875+0000 7fb18d688640 1 --2- 172.21.15.71:0/3934998806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1881004d0 0x7fb1880692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:55.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.875+0000 7fb18de89640 1 -- 172.21.15.71:0/3934998806 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb1881016c0 msgr2=0x7fb18819c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:55.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.875+0000 7fb18ce87640 1 --2- 172.21.15.71:0/3934998806 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb1881016c0 0x7fb18819c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:55.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.875+0000 7fb18de89640 1 --2- 172.21.15.71:0/3934998806 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb1881016c0 0x7fb18819c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:55.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.875+0000 7fb18de89640 1 -- 172.21.15.71:0/3934998806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1881004d0 msgr2=0x7fb1880692a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:55.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.875+0000 7fb18de89640 1 --2- 172.21.15.71:0/3934998806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1881004d0 0x7fb1880692a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:55.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.875+0000 7fb18de89640 1 -- 172.21.15.71:0/3934998806 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18819f4c0 con 0x7fb188102080 2024-03-04T08:56:55.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.875+0000 7fb18ce87640 1 --2- 172.21.15.71:0/3934998806 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb1881016c0 0x7fb18819c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:56:55.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.875+0000 7fb18de89640 1 --2- 172.21.15.71:0/3934998806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb188102080 0x7fb18819ebb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb180037040 tx=0x7fb180032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:55.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.876+0000 7fb17e7fc640 1 -- 172.21.15.71:0/3934998806 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb180041020 con 0x7fb188102080 2024-03-04T08:56:55.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.876+0000 7fb17e7fc640 1 -- 172.21.15.71:0/3934998806 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb180034d90 con 0x7fb188102080 2024-03-04T08:56:55.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.876+0000 7fb18f8ec640 1 -- 172.21.15.71:0/3934998806 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb18819f6f0 con 0x7fb188102080 2024-03-04T08:56:55.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.876+0000 7fb17e7fc640 1 -- 172.21.15.71:0/3934998806 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb180039040 con 0x7fb188102080 2024-03-04T08:56:55.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.876+0000 7fb18f8ec640 1 -- 172.21.15.71:0/3934998806 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb18819fc30 con 0x7fb188102080 2024-03-04T08:56:55.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.877+0000 7fb18f8ec640 1 -- 172.21.15.71:0/3934998806 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb188107790 con 0x7fb188102080 2024-03-04T08:56:55.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.878+0000 7fb17e7fc640 1 -- 172.21.15.71:0/3934998806 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fb180044020 con 0x7fb188102080 2024-03-04T08:56:55.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.879+0000 7fb17e7fc640 1 --2- 172.21.15.71:0/3934998806 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb1740a1e00 0x7fb1740a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:55.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.879+0000 7fb18d688640 1 --2- 172.21.15.71:0/3934998806 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb1740a1e00 0x7fb1740a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:55.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.879+0000 7fb17e7fc640 1 -- 172.21.15.71:0/3934998806 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb1800efef0 con 0x7fb188102080 2024-03-04T08:56:55.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.879+0000 7fb18d688640 1 --2- 172.21.15.71:0/3934998806 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb1740a1e00 0x7fb1740a42c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb188101520 tx=0x7fb184005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:55.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:55.883+0000 7fb17e7fc640 1 -- 172.21.15.71:0/3934998806 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb18004b020 con 0x7fb188102080 2024-03-04T08:56:56.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:55 smithi150 ceph-mon[36807]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-04T08:56:56.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:55 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/4060558525' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-04T08:56:56.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:55 smithi158 ceph-mon[37241]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-04T08:56:56.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:55 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/4060558525' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-04T08:56:56.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.082+0000 7fb18f8ec640 1 -- 172.21.15.71:0/3934998806 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fb188061a10 con 0x7fb1740a1e00 2024-03-04T08:56:56.084 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:55 smithi071 ceph-mon[32789]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-04T08:56:56.084 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:55 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/4060558525' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-04T08:56:56.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.099+0000 7fb17e7fc640 1 -- 172.21.15.71:0/3934998806 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb188061a10 con 0x7fb1740a1e00 2024-03-04T08:56:56.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.101+0000 7fb18f8ec640 1 -- 172.21.15.71:0/3934998806 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb1740a1e00 msgr2=0x7fb1740a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:56.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.101+0000 7fb18f8ec640 1 --2- 172.21.15.71:0/3934998806 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb1740a1e00 0x7fb1740a42c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb188101520 tx=0x7fb184005f90 comp rx=0 tx=0).stop 2024-03-04T08:56:56.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.101+0000 7fb18f8ec640 1 -- 172.21.15.71:0/3934998806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb188102080 msgr2=0x7fb18819ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:56.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.101+0000 7fb18f8ec640 1 --2- 172.21.15.71:0/3934998806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb188102080 0x7fb18819ebb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb180037040 tx=0x7fb180032f20 comp rx=0 tx=0).stop 2024-03-04T08:56:56.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.101+0000 7fb18f8ec640 1 -- 172.21.15.71:0/3934998806 shutdown_connections 2024-03-04T08:56:56.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.101+0000 7fb18f8ec640 1 --2- 172.21.15.71:0/3934998806 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb1740a1e00 0x7fb1740a42c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.101+0000 7fb18f8ec640 1 --2- 172.21.15.71:0/3934998806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb188102080 0x7fb18819ebb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.101+0000 7fb18f8ec640 1 --2- 172.21.15.71:0/3934998806 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb1881016c0 0x7fb18819c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.101+0000 7fb18f8ec640 1 --2- 172.21.15.71:0/3934998806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1881004d0 0x7fb1880692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.101+0000 7fb18f8ec640 1 -- 172.21.15.71:0/3934998806 >> 172.21.15.71:0/3934998806 conn(0x7fb1880fba60 msgr2=0x7fb188104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:56.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.101+0000 7fb18f8ec640 1 -- 172.21.15.71:0/3934998806 shutdown_connections 2024-03-04T08:56:56.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.101+0000 7fb18f8ec640 1 -- 172.21.15.71:0/3934998806 wait complete. 2024-03-04T08:56:56.108 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-04T08:56:56.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.218+0000 7fc0c1bbc640 1 -- 172.21.15.71:0/665038378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0bc0ff8b0 msgr2=0x7fc0bc0ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:56.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.218+0000 7fc0c1bbc640 1 --2- 172.21.15.71:0/665038378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0bc0ff8b0 0x7fc0bc0ffcd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac009930 tx=0x7fc0ac02bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:56.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.218+0000 7fc0c1bbc640 1 -- 172.21.15.71:0/665038378 shutdown_connections 2024-03-04T08:56:56.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.218+0000 7fc0c1bbc640 1 --2- 172.21.15.71:0/665038378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0bc103240 0x7fc0bc109ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.218+0000 7fc0c1bbc640 1 --2- 172.21.15.71:0/665038378 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0bc102880 0x7fc0bc102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.218+0000 7fc0c1bbc640 1 --2- 172.21.15.71:0/665038378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0bc0ff8b0 0x7fc0bc0ffcd0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.218+0000 7fc0c1bbc640 1 -- 172.21.15.71:0/665038378 >> 172.21.15.71:0/665038378 conn(0x7fc0bc0fb1e0 msgr2=0x7fc0bc0fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:56.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.218+0000 7fc0c1bbc640 1 -- 172.21.15.71:0/665038378 shutdown_connections 2024-03-04T08:56:56.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.218+0000 7fc0c1bbc640 1 -- 172.21.15.71:0/665038378 wait complete. 2024-03-04T08:56:56.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.219+0000 7fc0c1bbc640 1 Processor -- start 2024-03-04T08:56:56.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.219+0000 7fc0c1bbc640 1 -- start start 2024-03-04T08:56:56.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.219+0000 7fc0c1bbc640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0bc0ff8b0 0x7fc0bc197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:56.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.219+0000 7fc0c1bbc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0bc102880 0x7fc0bc197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:56.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.219+0000 7fc0c1bbc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0bc103240 0x7fc0bc19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:56.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.219+0000 7fc0c1bbc640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc10d260 con 0x7fc0bc103240 2024-03-04T08:56:56.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.219+0000 7fc0c1bbc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc10d0e0 con 0x7fc0bc102880 2024-03-04T08:56:56.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.219+0000 7fc0c1bbc640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc10d3e0 con 0x7fc0bc0ff8b0 2024-03-04T08:56:56.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.219+0000 7fc0bbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0bc103240 0x7fc0bc19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:56.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.220+0000 7fc0bbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0bc103240 0x7fc0bc19eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52206/0 (socket says 172.21.15.71:52206) 2024-03-04T08:56:56.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.220+0000 7fc0bb7fe640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0bc0ff8b0 0x7fc0bc197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:56.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.220+0000 7fc0bbfff640 1 -- 172.21.15.71:0/2941068313 learned_addr learned my addr 172.21.15.71:0/2941068313 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:56.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.220+0000 7fc0baffd640 1 --2- 172.21.15.71:0/2941068313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0bc102880 0x7fc0bc197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:56.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.220+0000 7fc0bbfff640 1 -- 172.21.15.71:0/2941068313 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0bc0ff8b0 msgr2=0x7fc0bc197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:56.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.220+0000 7fc0bbfff640 1 --2- 172.21.15.71:0/2941068313 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0bc0ff8b0 0x7fc0bc197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.220+0000 7fc0bbfff640 1 -- 172.21.15.71:0/2941068313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0bc102880 msgr2=0x7fc0bc197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:56.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.220+0000 7fc0bbfff640 1 --2- 172.21.15.71:0/2941068313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0bc102880 0x7fc0bc197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.220+0000 7fc0bbfff640 1 -- 172.21.15.71:0/2941068313 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0bc19f540 con 0x7fc0bc103240 2024-03-04T08:56:56.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.220+0000 7fc0baffd640 1 --2- 172.21.15.71:0/2941068313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0bc102880 0x7fc0bc197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:56:56.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.220+0000 7fc0bbfff640 1 --2- 172.21.15.71:0/2941068313 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0bc103240 0x7fc0bc19eed0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400eb10 tx=0x7fc0b400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:56.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.220+0000 7fc0b8ff9640 1 -- 172.21.15.71:0/2941068313 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4004b20 con 0x7fc0bc103240 2024-03-04T08:56:56.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.220+0000 7fc0b8ff9640 1 -- 172.21.15.71:0/2941068313 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc0b4004cc0 con 0x7fc0bc103240 2024-03-04T08:56:56.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.220+0000 7fc0c1bbc640 1 -- 172.21.15.71:0/2941068313 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0bc19f830 con 0x7fc0bc103240 2024-03-04T08:56:56.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.220+0000 7fc0b8ff9640 1 -- 172.21.15.71:0/2941068313 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4010760 con 0x7fc0bc103240 2024-03-04T08:56:56.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.220+0000 7fc0c1bbc640 1 -- 172.21.15.71:0/2941068313 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0bc19fd30 con 0x7fc0bc103240 2024-03-04T08:56:56.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.221+0000 7fc0c1bbc640 1 -- 172.21.15.71:0/2941068313 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0bc108a80 con 0x7fc0bc103240 2024-03-04T08:56:56.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.222+0000 7fc0b8ff9640 1 -- 172.21.15.71:0/2941068313 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fc0b4005020 con 0x7fc0bc103240 2024-03-04T08:56:56.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.223+0000 7fc0b8ff9640 1 --2- 172.21.15.71:0/2941068313 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc0a40a1d80 0x7fc0a40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:56.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.223+0000 7fc0bb7fe640 1 --2- 172.21.15.71:0/2941068313 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc0a40a1d80 0x7fc0a40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:56.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.223+0000 7fc0b8ff9640 1 -- 172.21.15.71:0/2941068313 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fc0b40ce5a0 con 0x7fc0bc103240 2024-03-04T08:56:56.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.223+0000 7fc0bb7fe640 1 --2- 172.21.15.71:0/2941068313 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc0a40a1d80 0x7fc0a40a4240 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac02c7b0 tx=0x7fc0ac000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:56.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.227+0000 7fc0b8ff9640 1 -- 172.21.15.71:0/2941068313 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc0b409b790 con 0x7fc0bc103240 2024-03-04T08:56:56.421 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:56:56 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:56:56.087+0000 7f04a58f8700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T08:56:56.421 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:56:56 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:56:56.087+0000 7f04a58f8700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T08:56:56.422 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:56:56 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:56:56.087+0000 7f04a58f8700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T08:56:56.422 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:56:56 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:56:56.087+0000 7f04a58f8700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T08:56:56.422 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:56:56 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:56:56.087+0000 7f04a58f8700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T08:56:56.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.426+0000 7fc0c1bbc640 1 -- 172.21.15.71:0/2941068313 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0bc061a10 con 0x7fc0a40a1d80 2024-03-04T08:56:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.447+0000 7fc0b8ff9640 1 -- 172.21.15.71:0/2941068313 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc0bc061a10 con 0x7fc0a40a1d80 2024-03-04T08:56:56.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.449+0000 7fc0c1bbc640 1 -- 172.21.15.71:0/2941068313 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc0a40a1d80 msgr2=0x7fc0a40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:56.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.449+0000 7fc0c1bbc640 1 --2- 172.21.15.71:0/2941068313 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc0a40a1d80 0x7fc0a40a4240 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac02c7b0 tx=0x7fc0ac000bc0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.449+0000 7fc0c1bbc640 1 -- 172.21.15.71:0/2941068313 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0bc103240 msgr2=0x7fc0bc19eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:56.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.449+0000 7fc0c1bbc640 1 --2- 172.21.15.71:0/2941068313 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0bc103240 0x7fc0bc19eed0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400eb10 tx=0x7fc0b400c2d0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.449+0000 7fc0c1bbc640 1 -- 172.21.15.71:0/2941068313 shutdown_connections 2024-03-04T08:56:56.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.449+0000 7fc0c1bbc640 1 --2- 172.21.15.71:0/2941068313 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fc0a40a1d80 0x7fc0a40a4240 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.449+0000 7fc0c1bbc640 1 --2- 172.21.15.71:0/2941068313 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0bc103240 0x7fc0bc19eed0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.449+0000 7fc0c1bbc640 1 --2- 172.21.15.71:0/2941068313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0bc102880 0x7fc0bc197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.449+0000 7fc0c1bbc640 1 --2- 172.21.15.71:0/2941068313 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc0bc0ff8b0 0x7fc0bc197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.449+0000 7fc0c1bbc640 1 -- 172.21.15.71:0/2941068313 >> 172.21.15.71:0/2941068313 conn(0x7fc0bc0fb1e0 msgr2=0x7fc0bc105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:56.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.449+0000 7fc0c1bbc640 1 -- 172.21.15.71:0/2941068313 shutdown_connections 2024-03-04T08:56:56.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.449+0000 7fc0c1bbc640 1 -- 172.21.15.71:0/2941068313 wait complete. 2024-03-04T08:56:56.456 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-04T08:56:56.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.566+0000 7fbd6c5d6640 1 -- 172.21.15.71:0/527933106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd641004d0 msgr2=0x7fbd641008f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:56.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.566+0000 7fbd6c5d6640 1 --2- 172.21.15.71:0/527933106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd641004d0 0x7fbd641008f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbd60009ca0 tx=0x7fbd6002c0f0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.566+0000 7fbd6c5d6640 1 -- 172.21.15.71:0/527933106 shutdown_connections 2024-03-04T08:56:56.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.566+0000 7fbd6c5d6640 1 --2- 172.21.15.71:0/527933106 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbd64102080 0x7fbd64108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.566+0000 7fbd6c5d6640 1 --2- 172.21.15.71:0/527933106 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd641016c0 0x7fbd64101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.566+0000 7fbd6c5d6640 1 --2- 172.21.15.71:0/527933106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd641004d0 0x7fbd641008f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.566+0000 7fbd6c5d6640 1 -- 172.21.15.71:0/527933106 >> 172.21.15.71:0/527933106 conn(0x7fbd640fba60 msgr2=0x7fbd640fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:56.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.567+0000 7fbd6c5d6640 1 -- 172.21.15.71:0/527933106 shutdown_connections 2024-03-04T08:56:56.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.567+0000 7fbd6c5d6640 1 -- 172.21.15.71:0/527933106 wait complete. 2024-03-04T08:56:56.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.567+0000 7fbd6c5d6640 1 Processor -- start 2024-03-04T08:56:56.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.567+0000 7fbd6c5d6640 1 -- start start 2024-03-04T08:56:56.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.567+0000 7fbd6c5d6640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbd641004d0 0x7fbd64197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:56.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.567+0000 7fbd6c5d6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd641016c0 0x7fbd64197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:56.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.567+0000 7fbd6c5d6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd64102080 0x7fbd6419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:56.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.567+0000 7fbd6c5d6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6410d270 con 0x7fbd64102080 2024-03-04T08:56:56.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.567+0000 7fbd6c5d6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6410d0f0 con 0x7fbd641016c0 2024-03-04T08:56:56.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.567+0000 7fbd6c5d6640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6410d3f0 con 0x7fbd641004d0 2024-03-04T08:56:56.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.567+0000 7fbd6ab73640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd64102080 0x7fbd6419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:56.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.567+0000 7fbd6ab73640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd64102080 0x7fbd6419eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52220/0 (socket says 172.21.15.71:52220) 2024-03-04T08:56:56.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.567+0000 7fbd6ab73640 1 -- 172.21.15.71:0/4054976251 learned_addr learned my addr 172.21.15.71:0/4054976251 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:56.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.567+0000 7fbd69b71640 1 --2- 172.21.15.71:0/4054976251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd641016c0 0x7fbd64197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:56.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.567+0000 7fbd6a372640 1 --2- 172.21.15.71:0/4054976251 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbd641004d0 0x7fbd64197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:56.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.568+0000 7fbd6ab73640 1 -- 172.21.15.71:0/4054976251 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbd641004d0 msgr2=0x7fbd64197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:56.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.568+0000 7fbd6ab73640 1 --2- 172.21.15.71:0/4054976251 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbd641004d0 0x7fbd64197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.568+0000 7fbd6ab73640 1 -- 172.21.15.71:0/4054976251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd641016c0 msgr2=0x7fbd64197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:56.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.568+0000 7fbd6ab73640 1 --2- 172.21.15.71:0/4054976251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd641016c0 0x7fbd64197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.568+0000 7fbd6ab73640 1 -- 172.21.15.71:0/4054976251 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6419f560 con 0x7fbd64102080 2024-03-04T08:56:56.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.568+0000 7fbd6ab73640 1 --2- 172.21.15.71:0/4054976251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd64102080 0x7fbd6419eef0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c00e970 tx=0x7fbd5c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:56.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.568+0000 7fbd5b7fe640 1 -- 172.21.15.71:0/4054976251 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c00ccc0 con 0x7fbd64102080 2024-03-04T08:56:56.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.568+0000 7fbd6c5d6640 1 -- 172.21.15.71:0/4054976251 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd6419f850 con 0x7fbd64102080 2024-03-04T08:56:56.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.568+0000 7fbd5b7fe640 1 -- 172.21.15.71:0/4054976251 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd5c00ce60 con 0x7fbd64102080 2024-03-04T08:56:56.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.568+0000 7fbd5b7fe640 1 -- 172.21.15.71:0/4054976251 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c010450 con 0x7fbd64102080 2024-03-04T08:56:56.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.568+0000 7fbd6c5d6640 1 -- 172.21.15.71:0/4054976251 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6419fda0 con 0x7fbd64102080 2024-03-04T08:56:56.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.569+0000 7fbd6c5d6640 1 -- 172.21.15.71:0/4054976251 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd64107910 con 0x7fbd64102080 2024-03-04T08:56:56.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.570+0000 7fbd5b7fe640 1 -- 172.21.15.71:0/4054976251 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fbd5c005020 con 0x7fbd64102080 2024-03-04T08:56:56.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.571+0000 7fbd5b7fe640 1 --2- 172.21.15.71:0/4054976251 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fbd500a1d80 0x7fbd500a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:56.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.571+0000 7fbd6a372640 1 --2- 172.21.15.71:0/4054976251 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fbd500a1d80 0x7fbd500a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:56.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.571+0000 7fbd5b7fe640 1 -- 172.21.15.71:0/4054976251 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fbd5c0ce3f0 con 0x7fbd64102080 2024-03-04T08:56:56.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.572+0000 7fbd6a372640 1 --2- 172.21.15.71:0/4054976251 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fbd500a1d80 0x7fbd500a4240 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd6002c700 tx=0x7fbd60032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:56.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.576+0000 7fbd5b7fe640 1 -- 172.21.15.71:0/4054976251 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbd5c09b5e0 con 0x7fbd64102080 2024-03-04T08:56:56.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.778+0000 7fbd6c5d6640 1 -- 172.21.15.71:0/4054976251 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd64061a10 con 0x7fbd500a1d80 2024-03-04T08:56:56.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.781+0000 7fbd5b7fe640 1 -- 172.21.15.71:0/4054976251 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fbd64061a10 con 0x7fbd500a1d80 2024-03-04T08:56:56.777 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/qa/sv_0/32636d6c-8d7e-4c32-9e7f-d042f2f7e6dc 2024-03-04T08:56:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.783+0000 7fbd6c5d6640 1 -- 172.21.15.71:0/4054976251 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fbd500a1d80 msgr2=0x7fbd500a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.783+0000 7fbd6c5d6640 1 --2- 172.21.15.71:0/4054976251 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fbd500a1d80 0x7fbd500a4240 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd6002c700 tx=0x7fbd60032590 comp rx=0 tx=0).stop 2024-03-04T08:56:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.783+0000 7fbd6c5d6640 1 -- 172.21.15.71:0/4054976251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd64102080 msgr2=0x7fbd6419eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.783+0000 7fbd6c5d6640 1 --2- 172.21.15.71:0/4054976251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd64102080 0x7fbd6419eef0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c00e970 tx=0x7fbd5c00ed70 comp rx=0 tx=0).stop 2024-03-04T08:56:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.783+0000 7fbd6c5d6640 1 -- 172.21.15.71:0/4054976251 shutdown_connections 2024-03-04T08:56:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.783+0000 7fbd6c5d6640 1 --2- 172.21.15.71:0/4054976251 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fbd500a1d80 0x7fbd500a4240 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.783+0000 7fbd6c5d6640 1 --2- 172.21.15.71:0/4054976251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd64102080 0x7fbd6419eef0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.783+0000 7fbd6c5d6640 1 --2- 172.21.15.71:0/4054976251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd641016c0 0x7fbd64197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.783+0000 7fbd6c5d6640 1 --2- 172.21.15.71:0/4054976251 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fbd641004d0 0x7fbd64197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.783+0000 7fbd6c5d6640 1 -- 172.21.15.71:0/4054976251 >> 172.21.15.71:0/4054976251 conn(0x7fbd640fba60 msgr2=0x7fbd64104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.784+0000 7fbd6c5d6640 1 -- 172.21.15.71:0/4054976251 shutdown_connections 2024-03-04T08:56:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.784+0000 7fbd6c5d6640 1 -- 172.21.15.71:0/4054976251 wait complete. 2024-03-04T08:56:56.790 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-04T08:56:56.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.900+0000 7f84ff19a640 1 -- 172.21.15.71:0/3852128793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84f8107360 msgr2=0x7f84f8109750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:56.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.900+0000 7f84ff19a640 1 --2- 172.21.15.71:0/3852128793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84f8107360 0x7f84f8109750 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009ca0 tx=0x7f84f002c0f0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.900+0000 7f84ff19a640 1 -- 172.21.15.71:0/3852128793 shutdown_connections 2024-03-04T08:56:56.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.900+0000 7f84ff19a640 1 --2- 172.21.15.71:0/3852128793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84f8107360 0x7f84f8109750 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.900+0000 7f84ff19a640 1 --2- 172.21.15.71:0/3852128793 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f84f8104a30 0x7f84f8106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.900+0000 7f84ff19a640 1 --2- 172.21.15.71:0/3852128793 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84f8102100 0x7f84f81044f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.900+0000 7f84ff19a640 1 -- 172.21.15.71:0/3852128793 >> 172.21.15.71:0/3852128793 conn(0x7f84f80fba70 msgr2=0x7f84f80fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:56.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.900+0000 7f84ff19a640 1 -- 172.21.15.71:0/3852128793 shutdown_connections 2024-03-04T08:56:56.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.900+0000 7f84ff19a640 1 -- 172.21.15.71:0/3852128793 wait complete. 2024-03-04T08:56:56.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.901+0000 7f84ff19a640 1 Processor -- start 2024-03-04T08:56:56.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.901+0000 7f84ff19a640 1 -- start start 2024-03-04T08:56:56.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.901+0000 7f84ff19a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84f8102100 0x7f84f8197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:56.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.901+0000 7f84ff19a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84f8104a30 0x7f84f8197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:56.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.901+0000 7f84ff19a640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f84f8107360 0x7f84f819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:56.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.901+0000 7f84ff19a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f810d270 con 0x7f84f8104a30 2024-03-04T08:56:56.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.901+0000 7f84ff19a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f810d0f0 con 0x7f84f8102100 2024-03-04T08:56:56.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.901+0000 7f84ff19a640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f810d3f0 con 0x7f84f8107360 2024-03-04T08:56:56.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.901+0000 7f84f7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84f8104a30 0x7f84f8197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:56.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.901+0000 7f84f7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84f8104a30 0x7f84f8197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52246/0 (socket says 172.21.15.71:52246) 2024-03-04T08:56:56.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.901+0000 7f84f7fff640 1 -- 172.21.15.71:0/987565276 learned_addr learned my addr 172.21.15.71:0/987565276 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:56.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.901+0000 7f84fcf36640 1 --2- 172.21.15.71:0/987565276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84f8102100 0x7f84f8197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:56.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.901+0000 7f84f7fff640 1 -- 172.21.15.71:0/987565276 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f84f8107360 msgr2=0x7f84f819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:56.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.901+0000 7f84f7fff640 1 --2- 172.21.15.71:0/987565276 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f84f8107360 0x7f84f819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.901+0000 7f84f7fff640 1 -- 172.21.15.71:0/987565276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84f8102100 msgr2=0x7f84f8197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:56.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.901+0000 7f84f7fff640 1 --2- 172.21.15.71:0/987565276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84f8102100 0x7f84f8197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:56.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.901+0000 7f84f7fff640 1 -- 172.21.15.71:0/987565276 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f819f530 con 0x7f84f8104a30 2024-03-04T08:56:56.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.902+0000 7f84f7fff640 1 --2- 172.21.15.71:0/987565276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84f8104a30 0x7f84f8197e50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f84e400d740 tx=0x7f84e400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:56.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.902+0000 7f84f5ffb640 1 -- 172.21.15.71:0/987565276 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4017070 con 0x7f84f8104a30 2024-03-04T08:56:56.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.902+0000 7f84ff19a640 1 -- 172.21.15.71:0/987565276 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f819f820 con 0x7f84f8104a30 2024-03-04T08:56:56.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.902+0000 7f84f5ffb640 1 -- 172.21.15.71:0/987565276 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84e400de50 con 0x7f84f8104a30 2024-03-04T08:56:56.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.902+0000 7f84f5ffb640 1 -- 172.21.15.71:0/987565276 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4004e00 con 0x7f84f8104a30 2024-03-04T08:56:56.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.902+0000 7f84ff19a640 1 -- 172.21.15.71:0/987565276 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f819fd70 con 0x7f84f8104a30 2024-03-04T08:56:56.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.903+0000 7f84ff19a640 1 -- 172.21.15.71:0/987565276 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f80fd560 con 0x7f84f8104a30 2024-03-04T08:56:56.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.904+0000 7f84f5ffb640 1 -- 172.21.15.71:0/987565276 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f84e4004fa0 con 0x7f84f8104a30 2024-03-04T08:56:56.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.904+0000 7f84f5ffb640 1 --2- 172.21.15.71:0/987565276 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f84e80a1d80 0x7f84e80a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:56.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.904+0000 7f84fcf36640 1 --2- 172.21.15.71:0/987565276 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f84e80a1d80 0x7f84e80a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:56.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.904+0000 7f84f5ffb640 1 -- 172.21.15.71:0/987565276 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f84e40cd990 con 0x7f84f8104a30 2024-03-04T08:56:56.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.905+0000 7f84fcf36640 1 --2- 172.21.15.71:0/987565276 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f84e80a1d80 0x7f84e80a4240 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f84ec009bd0 tx=0x7f84ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:56.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:56.909+0000 7f84f5ffb640 1 -- 172.21.15.71:0/987565276 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f84e409ab80 con 0x7f84f8104a30 2024-03-04T08:56:57.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:56 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/4060558525' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-04T08:56:57.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:56 smithi150 ceph-mon[36807]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-03-04T08:56:57.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:56 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/4060558525' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-04T08:56:57.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:56 smithi158 ceph-mon[37241]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-03-04T08:56:57.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.109+0000 7f84ff19a640 1 -- 172.21.15.71:0/987565276 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f84f8061aa0 con 0x7f84e80a1d80 2024-03-04T08:56:57.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.130+0000 7f84f5ffb640 1 -- 172.21.15.71:0/987565276 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f84f8061aa0 con 0x7f84e80a1d80 2024-03-04T08:56:57.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.132+0000 7f84ff19a640 1 -- 172.21.15.71:0/987565276 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f84e80a1d80 msgr2=0x7f84e80a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:57.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.132+0000 7f84ff19a640 1 --2- 172.21.15.71:0/987565276 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f84e80a1d80 0x7f84e80a4240 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f84ec009bd0 tx=0x7f84ec005f90 comp rx=0 tx=0).stop 2024-03-04T08:56:57.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.132+0000 7f84ff19a640 1 -- 172.21.15.71:0/987565276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84f8104a30 msgr2=0x7f84f8197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:57.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.132+0000 7f84ff19a640 1 --2- 172.21.15.71:0/987565276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84f8104a30 0x7f84f8197e50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f84e400d740 tx=0x7f84e400dc00 comp rx=0 tx=0).stop 2024-03-04T08:56:57.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.132+0000 7f84ff19a640 1 -- 172.21.15.71:0/987565276 shutdown_connections 2024-03-04T08:56:57.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.132+0000 7f84ff19a640 1 --2- 172.21.15.71:0/987565276 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f84e80a1d80 0x7f84e80a4240 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.132+0000 7f84ff19a640 1 --2- 172.21.15.71:0/987565276 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f84f8107360 0x7f84f819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.132+0000 7f84ff19a640 1 --2- 172.21.15.71:0/987565276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84f8104a30 0x7f84f8197e50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.132+0000 7f84ff19a640 1 --2- 172.21.15.71:0/987565276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84f8102100 0x7f84f8197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.132+0000 7f84ff19a640 1 -- 172.21.15.71:0/987565276 >> 172.21.15.71:0/987565276 conn(0x7f84f80fba70 msgr2=0x7f84f8102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:57.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.133+0000 7f84ff19a640 1 -- 172.21.15.71:0/987565276 shutdown_connections 2024-03-04T08:56:57.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.133+0000 7f84ff19a640 1 -- 172.21.15.71:0/987565276 wait complete. 2024-03-04T08:56:57.138 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:56 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/4060558525' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-04T08:56:57.138 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:56 smithi071 ceph-mon[32789]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-03-04T08:56:57.140 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-04T08:56:57.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.250+0000 7efe399f6640 1 -- 172.21.15.71:0/3992889789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe34102090 msgr2=0x7efe34108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:57.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.250+0000 7efe399f6640 1 --2- 172.21.15.71:0/3992889789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe34102090 0x7efe34108920 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009ca0 tx=0x7efe2c02c0f0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.250+0000 7efe399f6640 1 -- 172.21.15.71:0/3992889789 shutdown_connections 2024-03-04T08:56:57.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.250+0000 7efe399f6640 1 --2- 172.21.15.71:0/3992889789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe34102090 0x7efe34108920 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.250+0000 7efe399f6640 1 --2- 172.21.15.71:0/3992889789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe341016d0 0x7efe34101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.250+0000 7efe399f6640 1 --2- 172.21.15.71:0/3992889789 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efe341004e0 0x7efe34100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.250+0000 7efe399f6640 1 -- 172.21.15.71:0/3992889789 >> 172.21.15.71:0/3992889789 conn(0x7efe340fba70 msgr2=0x7efe340fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:57.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.250+0000 7efe399f6640 1 -- 172.21.15.71:0/3992889789 shutdown_connections 2024-03-04T08:56:57.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.250+0000 7efe399f6640 1 -- 172.21.15.71:0/3992889789 wait complete. 2024-03-04T08:56:57.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.251+0000 7efe399f6640 1 Processor -- start 2024-03-04T08:56:57.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.251+0000 7efe399f6640 1 -- start start 2024-03-04T08:56:57.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.251+0000 7efe399f6640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efe341004e0 0x7efe341967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:57.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.251+0000 7efe399f6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe341016d0 0x7efe34196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:57.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.251+0000 7efe399f6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe34102090 0x7efe3419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:57.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.251+0000 7efe399f6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3410c170 con 0x7efe341016d0 2024-03-04T08:56:57.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.251+0000 7efe399f6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3410bff0 con 0x7efe34102090 2024-03-04T08:56:57.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.251+0000 7efe399f6640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3410c2f0 con 0x7efe341004e0 2024-03-04T08:56:57.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.251+0000 7efe2bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe341016d0 0x7efe34196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:57.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.251+0000 7efe2bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe341016d0 0x7efe34196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52254/0 (socket says 172.21.15.71:52254) 2024-03-04T08:56:57.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.251+0000 7efe2bfff640 1 -- 172.21.15.71:0/514939248 learned_addr learned my addr 172.21.15.71:0/514939248 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:57.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.251+0000 7efe337fe640 1 --2- 172.21.15.71:0/514939248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe34102090 0x7efe3419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:57.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.251+0000 7efe32ffd640 1 --2- 172.21.15.71:0/514939248 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efe341004e0 0x7efe341967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:57.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.251+0000 7efe2bfff640 1 -- 172.21.15.71:0/514939248 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efe341004e0 msgr2=0x7efe341967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:57.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.251+0000 7efe2bfff640 1 --2- 172.21.15.71:0/514939248 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efe341004e0 0x7efe341967f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.251+0000 7efe2bfff640 1 -- 172.21.15.71:0/514939248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe34102090 msgr2=0x7efe3419dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:57.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.251+0000 7efe2bfff640 1 --2- 172.21.15.71:0/514939248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe34102090 0x7efe3419dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.251+0000 7efe2bfff640 1 -- 172.21.15.71:0/514939248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3419e410 con 0x7efe341016d0 2024-03-04T08:56:57.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.251+0000 7efe2bfff640 1 --2- 172.21.15.71:0/514939248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe341016d0 0x7efe34196d30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00e9f0 tx=0x7efe1c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:57.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.252+0000 7efe389f4640 1 -- 172.21.15.71:0/514939248 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c00cd80 con 0x7efe341016d0 2024-03-04T08:56:57.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.252+0000 7efe399f6640 1 -- 172.21.15.71:0/514939248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3419e700 con 0x7efe341016d0 2024-03-04T08:56:57.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.252+0000 7efe389f4640 1 -- 172.21.15.71:0/514939248 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe1c010910 con 0x7efe341016d0 2024-03-04T08:56:57.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.252+0000 7efe389f4640 1 -- 172.21.15.71:0/514939248 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c0188f0 con 0x7efe341016d0 2024-03-04T08:56:57.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.252+0000 7efe399f6640 1 -- 172.21.15.71:0/514939248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3419eca0 con 0x7efe341016d0 2024-03-04T08:56:57.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.253+0000 7efe399f6640 1 -- 172.21.15.71:0/514939248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe34107800 con 0x7efe341016d0 2024-03-04T08:56:57.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.254+0000 7efe389f4640 1 -- 172.21.15.71:0/514939248 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7efe1c010ac0 con 0x7efe341016d0 2024-03-04T08:56:57.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.254+0000 7efe389f4640 1 --2- 172.21.15.71:0/514939248 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7efe200a1d80 0x7efe200a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:57.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.254+0000 7efe32ffd640 1 --2- 172.21.15.71:0/514939248 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7efe200a1d80 0x7efe200a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:57.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.254+0000 7efe389f4640 1 -- 172.21.15.71:0/514939248 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7efe1c0ce400 con 0x7efe341016d0 2024-03-04T08:56:57.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.255+0000 7efe32ffd640 1 --2- 172.21.15.71:0/514939248 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7efe200a1d80 0x7efe200a4240 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efe34101530 tx=0x7efe24005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:57.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.259+0000 7efe389f4640 1 -- 172.21.15.71:0/514939248 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7efe1c09b5f0 con 0x7efe341016d0 2024-03-04T08:56:57.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.462+0000 7efe399f6640 1 -- 172.21.15.71:0/514939248 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7efe34061aa0 con 0x7efe200a1d80 2024-03-04T08:56:57.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.466+0000 7efe389f4640 1 -- 172.21.15.71:0/514939248 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7efe34061aa0 con 0x7efe200a1d80 2024-03-04T08:56:57.462 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/qa/sv_1/60c26083-5e79-4991-b9a9-8838ea2e9acd 2024-03-04T08:56:57.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.468+0000 7efe399f6640 1 -- 172.21.15.71:0/514939248 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7efe200a1d80 msgr2=0x7efe200a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:57.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.468+0000 7efe399f6640 1 --2- 172.21.15.71:0/514939248 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7efe200a1d80 0x7efe200a4240 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efe34101530 tx=0x7efe24005f90 comp rx=0 tx=0).stop 2024-03-04T08:56:57.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.468+0000 7efe399f6640 1 -- 172.21.15.71:0/514939248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe341016d0 msgr2=0x7efe34196d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:57.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.468+0000 7efe399f6640 1 --2- 172.21.15.71:0/514939248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe341016d0 0x7efe34196d30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00e9f0 tx=0x7efe1c00eeb0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.468+0000 7efe399f6640 1 -- 172.21.15.71:0/514939248 shutdown_connections 2024-03-04T08:56:57.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.468+0000 7efe399f6640 1 --2- 172.21.15.71:0/514939248 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7efe200a1d80 0x7efe200a4240 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.468+0000 7efe399f6640 1 --2- 172.21.15.71:0/514939248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe34102090 0x7efe3419dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.468+0000 7efe399f6640 1 --2- 172.21.15.71:0/514939248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe341016d0 0x7efe34196d30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.468+0000 7efe399f6640 1 --2- 172.21.15.71:0/514939248 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7efe341004e0 0x7efe341967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.468+0000 7efe399f6640 1 -- 172.21.15.71:0/514939248 >> 172.21.15.71:0/514939248 conn(0x7efe340fba70 msgr2=0x7efe341049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:57.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.468+0000 7efe399f6640 1 -- 172.21.15.71:0/514939248 shutdown_connections 2024-03-04T08:56:57.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.468+0000 7efe399f6640 1 -- 172.21.15.71:0/514939248 wait complete. 2024-03-04T08:56:57.475 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:56:57.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.581+0000 7fd1c0781640 1 -- 172.21.15.71:0/4142428566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1b8103270 msgr2=0x7fd1b8109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:57.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.581+0000 7fd1c0781640 1 --2- 172.21.15.71:0/4142428566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1b8103270 0x7fd1b8109b00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009cd0 tx=0x7fd1b002c0f0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.581+0000 7fd1c0781640 1 -- 172.21.15.71:0/4142428566 shutdown_connections 2024-03-04T08:56:57.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.581+0000 7fd1c0781640 1 --2- 172.21.15.71:0/4142428566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1b8103270 0x7fd1b8109b00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.581+0000 7fd1c0781640 1 --2- 172.21.15.71:0/4142428566 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd1b81028b0 0x7fd1b8102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.581+0000 7fd1c0781640 1 --2- 172.21.15.71:0/4142428566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1b80ff8e0 0x7fd1b80ffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.581+0000 7fd1c0781640 1 -- 172.21.15.71:0/4142428566 >> 172.21.15.71:0/4142428566 conn(0x7fd1b80fb210 msgr2=0x7fd1b80fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:57.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.581+0000 7fd1c0781640 1 -- 172.21.15.71:0/4142428566 shutdown_connections 2024-03-04T08:56:57.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.581+0000 7fd1c0781640 1 -- 172.21.15.71:0/4142428566 wait complete. 2024-03-04T08:56:57.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.582+0000 7fd1c0781640 1 Processor -- start 2024-03-04T08:56:57.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.582+0000 7fd1c0781640 1 -- start start 2024-03-04T08:56:57.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.582+0000 7fd1c0781640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1b80ff8e0 0x7fd1b81978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:57.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.582+0000 7fd1c0781640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd1b81028b0 0x7fd1b8197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:57.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.582+0000 7fd1c0781640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1b8103270 0x7fd1b819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:57.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.582+0000 7fd1c0781640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b810d2c0 con 0x7fd1b8103270 2024-03-04T08:56:57.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.582+0000 7fd1c0781640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b810d140 con 0x7fd1b80ff8e0 2024-03-04T08:56:57.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.582+0000 7fd1c0781640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b810d440 con 0x7fd1b81028b0 2024-03-04T08:56:57.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.582+0000 7fd1bed1e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1b8103270 0x7fd1b819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:57.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.582+0000 7fd1be51d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1b80ff8e0 0x7fd1b81978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:57.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.582+0000 7fd1be51d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1b80ff8e0 0x7fd1b81978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.71:60562/0 (socket says 172.21.15.71:60562) 2024-03-04T08:56:57.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.582+0000 7fd1be51d640 1 -- 172.21.15.71:0/1193363184 learned_addr learned my addr 172.21.15.71:0/1193363184 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:57.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.582+0000 7fd1bed1e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1b8103270 0x7fd1b819ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52272/0 (socket says 172.21.15.71:52272) 2024-03-04T08:56:57.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.582+0000 7fd1bdd1c640 1 --2- 172.21.15.71:0/1193363184 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd1b81028b0 0x7fd1b8197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:57.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.582+0000 7fd1bed1e640 1 -- 172.21.15.71:0/1193363184 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd1b81028b0 msgr2=0x7fd1b8197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:57.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.582+0000 7fd1bed1e640 1 --2- 172.21.15.71:0/1193363184 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd1b81028b0 0x7fd1b8197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.582+0000 7fd1bed1e640 1 -- 172.21.15.71:0/1193363184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1b80ff8e0 msgr2=0x7fd1b81978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:57.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.582+0000 7fd1bed1e640 1 --2- 172.21.15.71:0/1193363184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1b80ff8e0 0x7fd1b81978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.582+0000 7fd1bed1e640 1 -- 172.21.15.71:0/1193363184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b819f4f0 con 0x7fd1b8103270 2024-03-04T08:56:57.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.582+0000 7fd1be51d640 1 --2- 172.21.15.71:0/1193363184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1b80ff8e0 0x7fd1b81978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:56:57.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.583+0000 7fd1bed1e640 1 --2- 172.21.15.71:0/1193363184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1b8103270 0x7fd1b819ee80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009bf0 tx=0x7fd1b0032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:57.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.583+0000 7fd1af7fe640 1 -- 172.21.15.71:0/1193363184 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0041020 con 0x7fd1b8103270 2024-03-04T08:56:57.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.583+0000 7fd1af7fe640 1 -- 172.21.15.71:0/1193363184 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1b0034cd0 con 0x7fd1b8103270 2024-03-04T08:56:57.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.583+0000 7fd1c0781640 1 -- 172.21.15.71:0/1193363184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b819f780 con 0x7fd1b8103270 2024-03-04T08:56:57.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.583+0000 7fd1af7fe640 1 -- 172.21.15.71:0/1193363184 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0039040 con 0x7fd1b8103270 2024-03-04T08:56:57.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.583+0000 7fd1c0781640 1 -- 172.21.15.71:0/1193363184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b819fc40 con 0x7fd1b8103270 2024-03-04T08:56:57.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.584+0000 7fd1c0781640 1 -- 172.21.15.71:0/1193363184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b81089e0 con 0x7fd1b8103270 2024-03-04T08:56:57.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.585+0000 7fd1af7fe640 1 -- 172.21.15.71:0/1193363184 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fd1b0044020 con 0x7fd1b8103270 2024-03-04T08:56:57.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.585+0000 7fd1af7fe640 1 --2- 172.21.15.71:0/1193363184 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd1a40a1d80 0x7fd1a40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:57.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.585+0000 7fd1be51d640 1 --2- 172.21.15.71:0/1193363184 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd1a40a1d80 0x7fd1a40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:57.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.586+0000 7fd1af7fe640 1 -- 172.21.15.71:0/1193363184 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd1b00efd20 con 0x7fd1b8103270 2024-03-04T08:56:57.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.586+0000 7fd1be51d640 1 --2- 172.21.15.71:0/1193363184 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd1a40a1d80 0x7fd1a40a4240 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80058c0 tx=0x7fd1a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:57.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.590+0000 7fd1af7fe640 1 -- 172.21.15.71:0/1193363184 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd1b004b020 con 0x7fd1b8103270 2024-03-04T08:56:57.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.765+0000 7fd1af7fe640 1 -- 172.21.15.71:0/1193363184 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fd1b00b8f10 con 0x7fd1b8103270 2024-03-04T08:56:57.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.809+0000 7fd1c0781640 1 -- 172.21.15.71:0/1193363184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1b802cff0 con 0x7fd1b8103270 2024-03-04T08:56:57.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.809+0000 7fd1af7fe640 1 -- 172.21.15.71:0/1193363184 <== mon.0 v2:172.21.15.71:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11339 (secure 0 0 0) 0x7fd1b00bcfa0 con 0x7fd1b8103270 2024-03-04T08:56:57.806 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:57.806 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3975379807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3975379807},{"type":"v1","addr":"172.21.15.71:6835","nonce":3975379807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1962929347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1962929347},{"type":"v1","addr":"172.21.15.71:6837","nonce":1962929347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/1459252176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1459252176},{"type":"v1","addr":"172.21.15.71:6839","nonce":1459252176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6841/1083546600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":1083546600},{"type":"v1","addr":"172.21.15.71:6841","nonce":1083546600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/178179528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":178179528},{"type":"v1","addr":"172.21.15.158:6833","nonce":178179528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2468877675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2468877675},{"type":"v1","addr":"172.21.15.150:6833","nonce":2468877675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1652094309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1652094309},{"type":"v1","addr":"172.21.15.158:6835","nonce":1652094309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1802416462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1802416462},{"type":"v1","addr":"172.21.15.150:6835","nonce":1802416462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/617058380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":617058380},{"type":"v1","addr":"172.21.15.158:6837","nonce":617058380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/913465600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":913465600},{"type":"v1","addr":"172.21.15.150:6837","nonce":913465600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T08:56:30.343357+0000","modified":"2024-03-04T08:56:54.901214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.158:6839/2921704804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6838","nonce":2921704804},{"type":"v1","addr":"172.21.15.158:6839","nonce":2921704804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/2639278083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2639278083},{"type":"v1","addr":"172.21.15.150:6839","nonce":2639278083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491]},"id":1}]} 2024-03-04T08:56:57.806 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 20 2024-03-04T08:56:57.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.811+0000 7fd1c0781640 1 -- 172.21.15.71:0/1193363184 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd1a40a1d80 msgr2=0x7fd1a40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:57.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.811+0000 7fd1c0781640 1 --2- 172.21.15.71:0/1193363184 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd1a40a1d80 0x7fd1a40a4240 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80058c0 tx=0x7fd1a8005f90 comp rx=0 tx=0).stop 2024-03-04T08:56:57.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.811+0000 7fd1c0781640 1 -- 172.21.15.71:0/1193363184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1b8103270 msgr2=0x7fd1b819ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:57.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.811+0000 7fd1c0781640 1 --2- 172.21.15.71:0/1193363184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1b8103270 0x7fd1b819ee80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009bf0 tx=0x7fd1b0032ec0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.811+0000 7fd1c0781640 1 -- 172.21.15.71:0/1193363184 shutdown_connections 2024-03-04T08:56:57.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.811+0000 7fd1c0781640 1 --2- 172.21.15.71:0/1193363184 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd1a40a1d80 0x7fd1a40a4240 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.811+0000 7fd1c0781640 1 --2- 172.21.15.71:0/1193363184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1b8103270 0x7fd1b819ee80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.811+0000 7fd1c0781640 1 --2- 172.21.15.71:0/1193363184 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd1b81028b0 0x7fd1b8197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.811+0000 7fd1c0781640 1 --2- 172.21.15.71:0/1193363184 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1b80ff8e0 0x7fd1b81978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.811+0000 7fd1c0781640 1 -- 172.21.15.71:0/1193363184 >> 172.21.15.71:0/1193363184 conn(0x7fd1b80fb210 msgr2=0x7fd1b8105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:57.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.812+0000 7fd1c0781640 1 -- 172.21.15.71:0/1193363184 shutdown_connections 2024-03-04T08:56:57.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.812+0000 7fd1c0781640 1 -- 172.21.15.71:0/1193363184 wait complete. 2024-03-04T08:56:57.819 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T08:56:57.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.924+0000 7f1794d0f640 1 -- 172.21.15.71:0/4040757960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17901031b0 msgr2=0x7f1790109a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:57.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.924+0000 7f1794d0f640 1 --2- 172.21.15.71:0/4040757960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17901031b0 0x7f1790109a40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1788009ca0 tx=0x7f178802c0f0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.924+0000 7f1794d0f640 1 -- 172.21.15.71:0/4040757960 shutdown_connections 2024-03-04T08:56:57.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.924+0000 7f1794d0f640 1 --2- 172.21.15.71:0/4040757960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17901031b0 0x7f1790109a40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.924+0000 7f1794d0f640 1 --2- 172.21.15.71:0/4040757960 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17901027f0 0x7f1790102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.924+0000 7f1794d0f640 1 --2- 172.21.15.71:0/4040757960 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1790101600 0x7f1790101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.924+0000 7f1794d0f640 1 -- 172.21.15.71:0/4040757960 >> 172.21.15.71:0/4040757960 conn(0x7f17900fcb70 msgr2=0x7f17900fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:57.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.924+0000 7f1794d0f640 1 -- 172.21.15.71:0/4040757960 shutdown_connections 2024-03-04T08:56:57.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.924+0000 7f1794d0f640 1 -- 172.21.15.71:0/4040757960 wait complete. 2024-03-04T08:56:57.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.925+0000 7f1794d0f640 1 Processor -- start 2024-03-04T08:56:57.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.925+0000 7f1794d0f640 1 -- start start 2024-03-04T08:56:57.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.925+0000 7f1794d0f640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1790101600 0x7f1790199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:57.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.925+0000 7f1794d0f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17901027f0 0x7f1790199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:57.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.925+0000 7f1794d0f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17901031b0 0x7f17901a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:57.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.925+0000 7f1794d0f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179010d280 con 0x7f17901031b0 2024-03-04T08:56:57.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.925+0000 7f1794d0f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179010d100 con 0x7f17901027f0 2024-03-04T08:56:57.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.925+0000 7f1794d0f640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179010d400 con 0x7f1790101600 2024-03-04T08:56:57.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.925+0000 7f178ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17901031b0 0x7f17901a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:57.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.925+0000 7f178ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17901031b0 0x7f17901a1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52292/0 (socket says 172.21.15.71:52292) 2024-03-04T08:56:57.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.925+0000 7f178ed9d640 1 -- 172.21.15.71:0/2648067688 learned_addr learned my addr 172.21.15.71:0/2648067688 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:57.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.925+0000 7f1787fff640 1 --2- 172.21.15.71:0/2648067688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17901027f0 0x7f1790199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:57.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.925+0000 7f178e59c640 1 --2- 172.21.15.71:0/2648067688 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1790101600 0x7f1790199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:57.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.925+0000 7f178ed9d640 1 -- 172.21.15.71:0/2648067688 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1790101600 msgr2=0x7f1790199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:57.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.925+0000 7f178ed9d640 1 --2- 172.21.15.71:0/2648067688 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1790101600 0x7f1790199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.925+0000 7f178ed9d640 1 -- 172.21.15.71:0/2648067688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17901027f0 msgr2=0x7f1790199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:57.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.925+0000 7f178ed9d640 1 --2- 172.21.15.71:0/2648067688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17901027f0 0x7f1790199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:57.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.925+0000 7f178ed9d640 1 -- 172.21.15.71:0/2648067688 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17901a16d0 con 0x7f17901031b0 2024-03-04T08:56:57.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.926+0000 7f178ed9d640 1 --2- 172.21.15.71:0/2648067688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17901031b0 0x7f17901a1060 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1788037040 tx=0x7f1788032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:57.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.926+0000 7f17877fe640 1 -- 172.21.15.71:0/2648067688 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1788041020 con 0x7f17901031b0 2024-03-04T08:56:57.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.926+0000 7f17877fe640 1 -- 172.21.15.71:0/2648067688 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1788034d90 con 0x7f17901031b0 2024-03-04T08:56:57.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.926+0000 7f1794d0f640 1 -- 172.21.15.71:0/2648067688 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17901a1960 con 0x7f17901031b0 2024-03-04T08:56:57.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.926+0000 7f17877fe640 1 -- 172.21.15.71:0/2648067688 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1788039040 con 0x7f17901031b0 2024-03-04T08:56:57.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.926+0000 7f1794d0f640 1 -- 172.21.15.71:0/2648067688 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17901a1e50 con 0x7f17901031b0 2024-03-04T08:56:57.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.927+0000 7f1794d0f640 1 -- 172.21.15.71:0/2648067688 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17901a2080 con 0x7f17901031b0 2024-03-04T08:56:57.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.928+0000 7f17877fe640 1 -- 172.21.15.71:0/2648067688 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f1788044020 con 0x7f17901031b0 2024-03-04T08:56:57.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.928+0000 7f17877fe640 1 --2- 172.21.15.71:0/2648067688 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f177c0a1e10 0x7f177c0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:57.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.928+0000 7f178e59c640 1 --2- 172.21.15.71:0/2648067688 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f177c0a1e10 0x7f177c0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:57.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.928+0000 7f17877fe640 1 -- 172.21.15.71:0/2648067688 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f17880effc0 con 0x7f17901031b0 2024-03-04T08:56:57.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.929+0000 7f178e59c640 1 --2- 172.21.15.71:0/2648067688 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f177c0a1e10 0x7f177c0a42d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1780005fd0 tx=0x7f1780005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:57.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:57.933+0000 7f17877fe640 1 -- 172.21.15.71:0/2648067688 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f178804b020 con 0x7f17901031b0 2024-03-04T08:56:58.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:57 smithi150 ceph-mon[36807]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:56:58.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:57 smithi150 ceph-mon[36807]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:56:58.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:57 smithi150 ceph-mon[36807]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 426 B/s wr, 0 op/s 2024-03-04T08:56:58.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:57 smithi150 ceph-mon[36807]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:56:58.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:57 smithi158 ceph-mon[37241]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:56:58.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:57 smithi158 ceph-mon[37241]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:56:58.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:57 smithi158 ceph-mon[37241]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 426 B/s wr, 0 op/s 2024-03-04T08:56:58.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:57 smithi158 ceph-mon[37241]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:56:58.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.110+0000 7f1794d0f640 1 -- 172.21.15.71:0/2648067688 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17901a2770 con 0x7f17901031b0 2024-03-04T08:56:58.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.111+0000 7f17877fe640 1 -- 172.21.15.71:0/2648067688 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24153 (secure 0 0 0) 0x7f17880bd160 con 0x7f17901031b0 2024-03-04T08:56:58.107 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:58.108 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":91,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","created":"2024-03-04T08:47:28.546102+0000","modified":"2024-03-04T08:56:35.007858+0000","last_up_change":"2024-03-04T08:55:30.744961+0000","last_in_change":"2024-03-04T08:55:11.570990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T08:51:14.933487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T08:56:28.253074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T08:56:29.543622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T08:56:32.884220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7e43b47d-8888-4631-bc96-8a96fcf8cd46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6803","nonce":1173641530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6805","nonce":1173641530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6809","nonce":1173641530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6807","nonce":1173641530}]},"public_addr":"172.21.15.71:6803/1173641530","cluster_addr":"172.21.15.71:6805/1173641530","heartbeat_back_addr":"172.21.15.71:6809/1173641530","heartbeat_front_addr":"172.21.15.71:6807/1173641530","state":["exists","up"]},{"osd":1,"uuid":"1209f214-f2e2-4209-bf69-4528a43a2123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6801","nonce":3383948110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6803","nonce":3383948110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6807","nonce":3383948110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6805","nonce":3383948110}]},"public_addr":"172.21.15.150:6801/3383948110","cluster_addr":"172.21.15.150:6803/3383948110","heartbeat_back_addr":"172.21.15.150:6807/3383948110","heartbeat_front_addr":"172.21.15.150:6805/3383948110","state":["exists","up"]},{"osd":2,"uuid":"b88c3082-93a9-4c9b-9221-a7105a44c91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6801","nonce":586944144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6803","nonce":586944144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6807","nonce":586944144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6805","nonce":586944144}]},"public_addr":"172.21.15.158:6801/586944144","cluster_addr":"172.21.15.158:6803/586944144","heartbeat_back_addr":"172.21.15.158:6807/586944144","heartbeat_front_addr":"172.21.15.158:6805/586944144","state":["exists","up"]},{"osd":3,"uuid":"e4fc7723-fd73-40b2-b9d0-7f9bda282ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6811","nonce":1311903262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6813","nonce":1311903262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6817","nonce":1311903262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6815","nonce":1311903262}]},"public_addr":"172.21.15.71:6811/1311903262","cluster_addr":"172.21.15.71:6813/1311903262","heartbeat_back_addr":"172.21.15.71:6817/1311903262","heartbeat_front_addr":"172.21.15.71:6815/1311903262","state":["exists","up"]},{"osd":4,"uuid":"e7eb72ac-f066-4ccd-b2bf-22a6a26daf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6809","nonce":2300837573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6811","nonce":2300837573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6815","nonce":2300837573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6813","nonce":2300837573}]},"public_addr":"172.21.15.150:6809/2300837573","cluster_addr":"172.21.15.150:6811/2300837573","heartbeat_back_addr":"172.21.15.150:6815/2300837573","heartbeat_front_addr":"172.21.15.150:6813/2300837573","state":["exists","up"]},{"osd":5,"uuid":"9b295840-e104-4a66-9cc4-7020627c1765","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6809","nonce":1922694045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6811","nonce":1922694045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6815","nonce":1922694045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6813","nonce":1922694045}]},"public_addr":"172.21.15.158:6809/1922694045","cluster_addr":"172.21.15.158:6811/1922694045","heartbeat_back_addr":"172.21.15.158:6815/1922694045","heartbeat_front_addr":"172.21.15.158:6813/1922694045","state":["exists","up"]},{"osd":6,"uuid":"6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6819","nonce":3848842707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6821","nonce":3848842707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6825","nonce":3848842707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6823","nonce":3848842707}]},"public_addr":"172.21.15.71:6819/3848842707","cluster_addr":"172.21.15.71:6821/3848842707","heartbeat_back_addr":"172.21.15.71:6825/3848842707","heartbeat_front_addr":"172.21.15.71:6823/3848842707","state":["exists","up"]},{"osd":7,"uuid":"6160a40a-d30f-4569-949c-0afb1bb3d133","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6817","nonce":1328330305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6819","nonce":1328330305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6823","nonce":1328330305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6821","nonce":1328330305}]},"public_addr":"172.21.15.150:6817/1328330305","cluster_addr":"172.21.15.150:6819/1328330305","heartbeat_back_addr":"172.21.15.150:6823/1328330305","heartbeat_front_addr":"172.21.15.150:6821/1328330305","state":["exists","up"]},{"osd":8,"uuid":"95ed2341-84a4-465c-9948-ac667680ce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6817","nonce":2195304577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6819","nonce":2195304577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6823","nonce":2195304577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6821","nonce":2195304577}]},"public_addr":"172.21.15.158:6817/2195304577","cluster_addr":"172.21.15.158:6819/2195304577","heartbeat_back_addr":"172.21.15.158:6823/2195304577","heartbeat_front_addr":"172.21.15.158:6821/2195304577","state":["exists","up"]},{"osd":9,"uuid":"088f6419-63a0-4768-afae-dbb8a5f5a565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6827","nonce":1913779347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6829","nonce":1913779347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6833","nonce":1913779347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6831","nonce":1913779347}]},"public_addr":"172.21.15.71:6827/1913779347","cluster_addr":"172.21.15.71:6829/1913779347","heartbeat_back_addr":"172.21.15.71:6833/1913779347","heartbeat_front_addr":"172.21.15.71:6831/1913779347","state":["exists","up"]},{"osd":10,"uuid":"18c85e24-068d-4f1e-a3b8-f7915b363234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6825","nonce":3295420996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6827","nonce":3295420996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6831","nonce":3295420996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6829","nonce":3295420996}]},"public_addr":"172.21.15.150:6825/3295420996","cluster_addr":"172.21.15.150:6827/3295420996","heartbeat_back_addr":"172.21.15.150:6831/3295420996","heartbeat_front_addr":"172.21.15.150:6829/3295420996","state":["exists","up"]},{"osd":11,"uuid":"4f6b3ddf-c482-4740-b1fd-de836e047c2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6825","nonce":3332083865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6827","nonce":3332083865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6831","nonce":3332083865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6829","nonce":3332083865}]},"public_addr":"172.21.15.158:6825/3332083865","cluster_addr":"172.21.15.158:6827/3332083865","heartbeat_back_addr":"172.21.15.158:6831/3332083865","heartbeat_front_addr":"172.21.15.158:6829/3332083865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:05.997832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:29.314235+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:08.819465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:41.194275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:06.032479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:38.814498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:13.067960+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:38.568479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:03.416713+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:38.349479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:04.096945+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:29.425250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1287712358":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/3770452860":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/718743094":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/1904717532":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:6800/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6800/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/2862184313":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/3621754630":"2024-03-05T08:48:22.593931+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T08:56:58.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.113+0000 7f1794d0f640 1 -- 172.21.15.71:0/2648067688 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f177c0a1e10 msgr2=0x7f177c0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:58.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.113+0000 7f1794d0f640 1 --2- 172.21.15.71:0/2648067688 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f177c0a1e10 0x7f177c0a42d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1780005fd0 tx=0x7f1780005850 comp rx=0 tx=0).stop 2024-03-04T08:56:58.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.113+0000 7f1794d0f640 1 -- 172.21.15.71:0/2648067688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17901031b0 msgr2=0x7f17901a1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:58.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.113+0000 7f1794d0f640 1 --2- 172.21.15.71:0/2648067688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17901031b0 0x7f17901a1060 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1788037040 tx=0x7f1788032f20 comp rx=0 tx=0).stop 2024-03-04T08:56:58.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.113+0000 7f1794d0f640 1 -- 172.21.15.71:0/2648067688 shutdown_connections 2024-03-04T08:56:58.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.113+0000 7f1794d0f640 1 --2- 172.21.15.71:0/2648067688 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f177c0a1e10 0x7f177c0a42d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.113+0000 7f1794d0f640 1 --2- 172.21.15.71:0/2648067688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17901031b0 0x7f17901a1060 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.113+0000 7f1794d0f640 1 --2- 172.21.15.71:0/2648067688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17901027f0 0x7f1790199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.113+0000 7f1794d0f640 1 --2- 172.21.15.71:0/2648067688 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1790101600 0x7f1790199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.113+0000 7f1794d0f640 1 -- 172.21.15.71:0/2648067688 >> 172.21.15.71:0/2648067688 conn(0x7f17900fcb70 msgr2=0x7f1790073800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:58.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.113+0000 7f1794d0f640 1 -- 172.21.15.71:0/2648067688 shutdown_connections 2024-03-04T08:56:58.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.113+0000 7f1794d0f640 1 -- 172.21.15.71:0/2648067688 wait complete. 2024-03-04T08:56:58.119 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:57 smithi071 ceph-mon[32789]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:56:58.119 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:57 smithi071 ceph-mon[32789]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:56:58.119 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:57 smithi071 ceph-mon[32789]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 426 B/s wr, 0 op/s 2024-03-04T08:56:58.119 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:57 smithi071 ceph-mon[32789]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:56:58.121 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-04T08:56:58.121 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T08:56:58.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.227+0000 7f02f2972640 1 -- 172.21.15.71:0/1012610119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02ec1049a0 msgr2=0x7f02ec106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:58.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.227+0000 7f02f2972640 1 --2- 172.21.15.71:0/1012610119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02ec1049a0 0x7f02ec106d90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009900 tx=0x7f02d402bc30 comp rx=0 tx=0).stop 2024-03-04T08:56:58.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.227+0000 7f02f2972640 1 -- 172.21.15.71:0/1012610119 shutdown_connections 2024-03-04T08:56:58.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.227+0000 7f02f2972640 1 --2- 172.21.15.71:0/1012610119 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f02ec1072d0 0x7f02ec109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.227+0000 7f02f2972640 1 --2- 172.21.15.71:0/1012610119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02ec1049a0 0x7f02ec106d90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.227+0000 7f02f2972640 1 --2- 172.21.15.71:0/1012610119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02ec102070 0x7f02ec104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.227+0000 7f02f2972640 1 -- 172.21.15.71:0/1012610119 >> 172.21.15.71:0/1012610119 conn(0x7f02ec0fba60 msgr2=0x7f02ec0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:58.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.227+0000 7f02f2972640 1 -- 172.21.15.71:0/1012610119 shutdown_connections 2024-03-04T08:56:58.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.227+0000 7f02f2972640 1 -- 172.21.15.71:0/1012610119 wait complete. 2024-03-04T08:56:58.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.228+0000 7f02f2972640 1 Processor -- start 2024-03-04T08:56:58.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.228+0000 7f02f2972640 1 -- start start 2024-03-04T08:56:58.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.228+0000 7f02f2972640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02ec102070 0x7f02ec197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:58.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.228+0000 7f02f2972640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f02ec1049a0 0x7f02ec197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:58.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.228+0000 7f02f2972640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02ec1072d0 0x7f02ec19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:58.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.228+0000 7f02f2972640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec10d250 con 0x7f02ec102070 2024-03-04T08:56:58.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.228+0000 7f02f2972640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec10d0d0 con 0x7f02ec1072d0 2024-03-04T08:56:58.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.228+0000 7f02f2972640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec10d3d0 con 0x7f02ec1049a0 2024-03-04T08:56:58.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.228+0000 7f02ebfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02ec102070 0x7f02ec197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:58.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.228+0000 7f02ebfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02ec102070 0x7f02ec197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52320/0 (socket says 172.21.15.71:52320) 2024-03-04T08:56:58.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.228+0000 7f02ebfff640 1 -- 172.21.15.71:0/2057568458 learned_addr learned my addr 172.21.15.71:0/2057568458 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:58.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.228+0000 7f02f0f0f640 1 --2- 172.21.15.71:0/2057568458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02ec1072d0 0x7f02ec19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:58.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.228+0000 7f02eb7fe640 1 --2- 172.21.15.71:0/2057568458 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f02ec1049a0 0x7f02ec197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:58.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.229+0000 7f02ebfff640 1 -- 172.21.15.71:0/2057568458 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f02ec1049a0 msgr2=0x7f02ec197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:58.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.229+0000 7f02ebfff640 1 --2- 172.21.15.71:0/2057568458 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f02ec1049a0 0x7f02ec197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.229+0000 7f02ebfff640 1 -- 172.21.15.71:0/2057568458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02ec1072d0 msgr2=0x7f02ec19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:58.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.229+0000 7f02ebfff640 1 --2- 172.21.15.71:0/2057568458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02ec1072d0 0x7f02ec19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.229+0000 7f02ebfff640 1 -- 172.21.15.71:0/2057568458 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ec19f520 con 0x7f02ec102070 2024-03-04T08:56:58.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.229+0000 7f02eb7fe640 1 --2- 172.21.15.71:0/2057568458 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f02ec1049a0 0x7f02ec197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:56:58.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.229+0000 7f02ebfff640 1 --2- 172.21.15.71:0/2057568458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02ec102070 0x7f02ec197900 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00e9f0 tx=0x7f02dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:58.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.229+0000 7f02e97fa640 1 -- 172.21.15.71:0/2057568458 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc00cd80 con 0x7f02ec102070 2024-03-04T08:56:58.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.229+0000 7f02e97fa640 1 -- 172.21.15.71:0/2057568458 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02dc010910 con 0x7f02ec102070 2024-03-04T08:56:58.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.229+0000 7f02f2972640 1 -- 172.21.15.71:0/2057568458 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02ec19f810 con 0x7f02ec102070 2024-03-04T08:56:58.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.229+0000 7f02e97fa640 1 -- 172.21.15.71:0/2057568458 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc018990 con 0x7f02ec102070 2024-03-04T08:56:58.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.229+0000 7f02f2972640 1 -- 172.21.15.71:0/2057568458 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ec19fd60 con 0x7f02ec102070 2024-03-04T08:56:58.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.230+0000 7f02f2972640 1 -- 172.21.15.71:0/2057568458 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02ec0fd550 con 0x7f02ec102070 2024-03-04T08:56:58.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.231+0000 7f02e97fa640 1 -- 172.21.15.71:0/2057568458 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f02dc010ac0 con 0x7f02ec102070 2024-03-04T08:56:58.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.231+0000 7f02e97fa640 1 --2- 172.21.15.71:0/2057568458 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f02d80a1e10 0x7f02d80a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:58.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.231+0000 7f02eb7fe640 1 --2- 172.21.15.71:0/2057568458 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f02d80a1e10 0x7f02d80a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:58.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.232+0000 7f02e97fa640 1 -- 172.21.15.71:0/2057568458 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f02dc0ce540 con 0x7f02ec102070 2024-03-04T08:56:58.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.232+0000 7f02eb7fe640 1 --2- 172.21.15.71:0/2057568458 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f02d80a1e10 0x7f02d80a42d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f02d4006010 tx=0x7f02d4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:58.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.236+0000 7f02e97fa640 1 -- 172.21.15.71:0/2057568458 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f02dc09b5d0 con 0x7f02ec102070 2024-03-04T08:56:58.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.406+0000 7f02f2972640 1 -- 172.21.15.71:0/2057568458 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f02ec061a10 con 0x7f02d80a1e10 2024-03-04T08:56:58.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.415+0000 7f02e97fa640 1 -- 172.21.15.71:0/2057568458 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340201 (secure 0 0 0) 0x7f02ec061a10 con 0x7f02d80a1e10 2024-03-04T08:56:58.416 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:58.418 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-03-04T08:56:58.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.419+0000 7f02f2972640 1 -- 172.21.15.71:0/2057568458 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f02d80a1e10 msgr2=0x7f02d80a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:58.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.419+0000 7f02f2972640 1 --2- 172.21.15.71:0/2057568458 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f02d80a1e10 0x7f02d80a42d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f02d4006010 tx=0x7f02d4000bc0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.419+0000 7f02f2972640 1 -- 172.21.15.71:0/2057568458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02ec102070 msgr2=0x7f02ec197900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:58.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.419+0000 7f02f2972640 1 --2- 172.21.15.71:0/2057568458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02ec102070 0x7f02ec197900 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00e9f0 tx=0x7f02dc00eeb0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.419+0000 7f02f2972640 1 -- 172.21.15.71:0/2057568458 shutdown_connections 2024-03-04T08:56:58.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.419+0000 7f02f2972640 1 --2- 172.21.15.71:0/2057568458 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f02d80a1e10 0x7f02d80a42d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.419+0000 7f02f2972640 1 --2- 172.21.15.71:0/2057568458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02ec1072d0 0x7f02ec19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.419+0000 7f02f2972640 1 --2- 172.21.15.71:0/2057568458 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f02ec1049a0 0x7f02ec197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.419+0000 7f02f2972640 1 --2- 172.21.15.71:0/2057568458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02ec102070 0x7f02ec197900 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.419+0000 7f02f2972640 1 -- 172.21.15.71:0/2057568458 >> 172.21.15.71:0/2057568458 conn(0x7f02ec0fba60 msgr2=0x7f02ec1029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:58.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.419+0000 7f02f2972640 1 -- 172.21.15.71:0/2057568458 shutdown_connections 2024-03-04T08:56:58.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.419+0000 7f02f2972640 1 -- 172.21.15.71:0/2057568458 wait complete. 2024-03-04T08:56:58.426 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-03-04T08:56:56.694502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":143,"num_read_kb":127,"num_write":230,"num_write_kb":2957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":180,"ondisk_log_size":180,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358640,"kb_used_data":13596,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583184,"statfs":{"total":1151940427776,"available":1151573180416,"internally_reserved":0,"allocated":13922304,"data_stored":7617696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":8,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003416"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084515+0000","last_change":"2024-03-04T08:56:30.650344+0000","last_active":"2024-03-04T08:56:37.084515+0000","last_peered":"2024-03-04T08:56:37.084515+0000","last_clean":"2024-03-04T08:56:37.084515+0000","last_became_active":"2024-03-04T08:56:30.649842+0000","last_became_peered":"2024-03-04T08:56:30.649842+0000","last_unstale":"2024-03-04T08:56:37.084515+0000","last_undegraded":"2024-03-04T08:56:37.084515+0000","last_fullsized":"2024-03-04T08:56:37.084515+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:04:57.162321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084709+0000","last_change":"2024-03-04T08:56:30.651616+0000","last_active":"2024-03-04T08:56:37.084709+0000","last_peered":"2024-03-04T08:56:37.084709+0000","last_clean":"2024-03-04T08:56:37.084709+0000","last_became_active":"2024-03-04T08:56:30.651283+0000","last_became_peered":"2024-03-04T08:56:30.651283+0000","last_unstale":"2024-03-04T08:56:37.084709+0000","last_undegraded":"2024-03-04T08:56:37.084709+0000","last_fullsized":"2024-03-04T08:56:37.084709+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:18:48.018257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084789+0000","last_change":"2024-03-04T08:56:30.657459+0000","last_active":"2024-03-04T08:56:37.084789+0000","last_peered":"2024-03-04T08:56:37.084789+0000","last_clean":"2024-03-04T08:56:37.084789+0000","last_became_active":"2024-03-04T08:56:30.656993+0000","last_became_peered":"2024-03-04T08:56:30.656993+0000","last_unstale":"2024-03-04T08:56:37.084789+0000","last_undegraded":"2024-03-04T08:56:37.084789+0000","last_fullsized":"2024-03-04T08:56:37.084789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:07:34.898068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083746+0000","last_change":"2024-03-04T08:56:30.644543+0000","last_active":"2024-03-04T08:56:37.083746+0000","last_peered":"2024-03-04T08:56:37.083746+0000","last_clean":"2024-03-04T08:56:37.083746+0000","last_became_active":"2024-03-04T08:56:30.644251+0000","last_became_peered":"2024-03-04T08:56:30.644251+0000","last_unstale":"2024-03-04T08:56:37.083746+0000","last_undegraded":"2024-03-04T08:56:37.083746+0000","last_fullsized":"2024-03-04T08:56:37.083746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:21:40.260400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.582670+0000","last_change":"2024-03-04T08:56:30.654640+0000","last_active":"2024-03-04T08:56:36.582670+0000","last_peered":"2024-03-04T08:56:36.582670+0000","last_clean":"2024-03-04T08:56:36.582670+0000","last_became_active":"2024-03-04T08:56:30.654312+0000","last_became_peered":"2024-03-04T08:56:30.654312+0000","last_unstale":"2024-03-04T08:56:36.582670+0000","last_undegraded":"2024-03-04T08:56:36.582670+0000","last_fullsized":"2024-03-04T08:56:36.582670+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:09:43.930713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.374337+0000","last_change":"2024-03-04T08:56:30.647445+0000","last_active":"2024-03-04T08:56:36.374337+0000","last_peered":"2024-03-04T08:56:36.374337+0000","last_clean":"2024-03-04T08:56:36.374337+0000","last_became_active":"2024-03-04T08:56:30.647130+0000","last_became_peered":"2024-03-04T08:56:30.647130+0000","last_unstale":"2024-03-04T08:56:36.374337+0000","last_undegraded":"2024-03-04T08:56:36.374337+0000","last_fullsized":"2024-03-04T08:56:36.374337+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:40:18.906604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671553+0000","last_change":"2024-03-04T08:56:30.645776+0000","last_active":"2024-03-04T08:56:35.671553+0000","last_peered":"2024-03-04T08:56:35.671553+0000","last_clean":"2024-03-04T08:56:35.671553+0000","last_became_active":"2024-03-04T08:56:30.645107+0000","last_became_peered":"2024-03-04T08:56:30.645107+0000","last_unstale":"2024-03-04T08:56:35.671553+0000","last_undegraded":"2024-03-04T08:56:35.671553+0000","last_fullsized":"2024-03-04T08:56:35.671553+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:44:35.230491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.768312+0000","last_change":"2024-03-04T08:56:30.646424+0000","last_active":"2024-03-04T08:56:35.768312+0000","last_peered":"2024-03-04T08:56:35.768312+0000","last_clean":"2024-03-04T08:56:35.768312+0000","last_became_active":"2024-03-04T08:56:30.646115+0000","last_became_peered":"2024-03-04T08:56:30.646115+0000","last_unstale":"2024-03-04T08:56:35.768312+0000","last_undegraded":"2024-03-04T08:56:35.768312+0000","last_fullsized":"2024-03-04T08:56:35.768312+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:42:36.201984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671159+0000","last_change":"2024-03-04T08:56:30.655027+0000","last_active":"2024-03-04T08:56:35.671159+0000","last_peered":"2024-03-04T08:56:35.671159+0000","last_clean":"2024-03-04T08:56:35.671159+0000","last_became_active":"2024-03-04T08:56:30.654766+0000","last_became_peered":"2024-03-04T08:56:30.654766+0000","last_unstale":"2024-03-04T08:56:35.671159+0000","last_undegraded":"2024-03-04T08:56:35.671159+0000","last_fullsized":"2024-03-04T08:56:35.671159+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:00:34.168740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.374661+0000","last_change":"2024-03-04T08:56:30.648268+0000","last_active":"2024-03-04T08:56:36.374661+0000","last_peered":"2024-03-04T08:56:36.374661+0000","last_clean":"2024-03-04T08:56:36.374661+0000","last_became_active":"2024-03-04T08:56:30.647905+0000","last_became_peered":"2024-03-04T08:56:30.647905+0000","last_unstale":"2024-03-04T08:56:36.374661+0000","last_undegraded":"2024-03-04T08:56:36.374661+0000","last_fullsized":"2024-03-04T08:56:36.374661+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:07:32.387510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.672115+0000","last_change":"2024-03-04T08:56:30.648061+0000","last_active":"2024-03-04T08:56:35.672115+0000","last_peered":"2024-03-04T08:56:35.672115+0000","last_clean":"2024-03-04T08:56:35.672115+0000","last_became_active":"2024-03-04T08:56:30.647740+0000","last_became_peered":"2024-03-04T08:56:30.647740+0000","last_unstale":"2024-03-04T08:56:35.672115+0000","last_undegraded":"2024-03-04T08:56:35.672115+0000","last_fullsized":"2024-03-04T08:56:35.672115+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:35:23.065902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083837+0000","last_change":"2024-03-04T08:56:30.645802+0000","last_active":"2024-03-04T08:56:37.083837+0000","last_peered":"2024-03-04T08:56:37.083837+0000","last_clean":"2024-03-04T08:56:37.083837+0000","last_became_active":"2024-03-04T08:56:30.645474+0000","last_became_peered":"2024-03-04T08:56:30.645474+0000","last_unstale":"2024-03-04T08:56:37.083837+0000","last_undegraded":"2024-03-04T08:56:37.083837+0000","last_fullsized":"2024-03-04T08:56:37.083837+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:35:40.207215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083876+0000","last_change":"2024-03-04T08:56:30.656149+0000","last_active":"2024-03-04T08:56:37.083876+0000","last_peered":"2024-03-04T08:56:37.083876+0000","last_clean":"2024-03-04T08:56:37.083876+0000","last_became_active":"2024-03-04T08:56:30.655493+0000","last_became_peered":"2024-03-04T08:56:30.655493+0000","last_unstale":"2024-03-04T08:56:37.083876+0000","last_undegraded":"2024-03-04T08:56:37.083876+0000","last_fullsized":"2024-03-04T08:56:37.083876+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:23:28.320328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084063+0000","last_change":"2024-03-04T08:56:30.658309+0000","last_active":"2024-03-04T08:56:37.084063+0000","last_peered":"2024-03-04T08:56:37.084063+0000","last_clean":"2024-03-04T08:56:37.084063+0000","last_became_active":"2024-03-04T08:56:30.656773+0000","last_became_peered":"2024-03-04T08:56:30.656773+0000","last_unstale":"2024-03-04T08:56:37.084063+0000","last_undegraded":"2024-03-04T08:56:37.084063+0000","last_fullsized":"2024-03-04T08:56:37.084063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:26:57.021442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084019+0000","last_change":"2024-03-04T08:56:30.650336+0000","last_active":"2024-03-04T08:56:37.084019+0000","last_peered":"2024-03-04T08:56:37.084019+0000","last_clean":"2024-03-04T08:56:37.084019+0000","last_became_active":"2024-03-04T08:56:30.648377+0000","last_became_peered":"2024-03-04T08:56:30.648377+0000","last_unstale":"2024-03-04T08:56:37.084019+0000","last_undegraded":"2024-03-04T08:56:37.084019+0000","last_fullsized":"2024-03-04T08:56:37.084019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:56:41.358895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.670486+0000","last_change":"2024-03-04T08:56:30.653327+0000","last_active":"2024-03-04T08:56:35.670486+0000","last_peered":"2024-03-04T08:56:35.670486+0000","last_clean":"2024-03-04T08:56:35.670486+0000","last_became_active":"2024-03-04T08:56:30.652523+0000","last_became_peered":"2024-03-04T08:56:30.652523+0000","last_unstale":"2024-03-04T08:56:35.670486+0000","last_undegraded":"2024-03-04T08:56:35.670486+0000","last_fullsized":"2024-03-04T08:56:35.670486+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:18:41.726921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084247+0000","last_change":"2024-03-04T08:56:30.647210+0000","last_active":"2024-03-04T08:56:37.084247+0000","last_peered":"2024-03-04T08:56:37.084247+0000","last_clean":"2024-03-04T08:56:37.084247+0000","last_became_active":"2024-03-04T08:56:30.646897+0000","last_became_peered":"2024-03-04T08:56:30.646897+0000","last_unstale":"2024-03-04T08:56:37.084247+0000","last_undegraded":"2024-03-04T08:56:37.084247+0000","last_fullsized":"2024-03-04T08:56:37.084247+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:40:35.021061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.668081+0000","last_change":"2024-03-04T08:56:30.648751+0000","last_active":"2024-03-04T08:56:35.668081+0000","last_peered":"2024-03-04T08:56:35.668081+0000","last_clean":"2024-03-04T08:56:35.668081+0000","last_became_active":"2024-03-04T08:56:30.648457+0000","last_became_peered":"2024-03-04T08:56:30.648457+0000","last_unstale":"2024-03-04T08:56:35.668081+0000","last_undegraded":"2024-03-04T08:56:35.668081+0000","last_fullsized":"2024-03-04T08:56:35.668081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:20:20.435769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.844328+0000","last_change":"2024-03-04T08:56:30.644523+0000","last_active":"2024-03-04T08:56:35.844328+0000","last_peered":"2024-03-04T08:56:35.844328+0000","last_clean":"2024-03-04T08:56:35.844328+0000","last_became_active":"2024-03-04T08:56:30.644183+0000","last_became_peered":"2024-03-04T08:56:30.644183+0000","last_unstale":"2024-03-04T08:56:35.844328+0000","last_undegraded":"2024-03-04T08:56:35.844328+0000","last_fullsized":"2024-03-04T08:56:35.844328+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:58:16.065571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.843802+0000","last_change":"2024-03-04T08:56:34.681350+0000","last_active":"2024-03-04T08:56:35.843802+0000","last_peered":"2024-03-04T08:56:35.843802+0000","last_clean":"2024-03-04T08:56:35.843802+0000","last_became_active":"2024-03-04T08:56:34.680940+0000","last_became_peered":"2024-03-04T08:56:34.680940+0000","last_unstale":"2024-03-04T08:56:35.843802+0000","last_undegraded":"2024-03-04T08:56:35.843802+0000","last_fullsized":"2024-03-04T08:56:35.843802+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:41:00.690541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.668309+0000","last_change":"2024-03-04T08:56:29.989827+0000","last_active":"2024-03-04T08:56:35.668309+0000","last_peered":"2024-03-04T08:56:35.668309+0000","last_clean":"2024-03-04T08:56:35.668309+0000","last_became_active":"2024-03-04T08:56:29.989509+0000","last_became_peered":"2024-03-04T08:56:29.989509+0000","last_unstale":"2024-03-04T08:56:35.668309+0000","last_undegraded":"2024-03-04T08:56:35.668309+0000","last_fullsized":"2024-03-04T08:56:35.668309+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:27:37.771477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.768541+0000","last_change":"2024-03-04T08:56:30.638354+0000","last_active":"2024-03-04T08:56:35.768541+0000","last_peered":"2024-03-04T08:56:35.768541+0000","last_clean":"2024-03-04T08:56:35.768541+0000","last_became_active":"2024-03-04T08:56:30.638058+0000","last_became_peered":"2024-03-04T08:56:30.638058+0000","last_unstale":"2024-03-04T08:56:35.768541+0000","last_undegraded":"2024-03-04T08:56:35.768541+0000","last_fullsized":"2024-03-04T08:56:35.768541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:43:51.920411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083731+0000","last_change":"2024-03-04T08:56:34.680206+0000","last_active":"2024-03-04T08:56:37.083731+0000","last_peered":"2024-03-04T08:56:37.083731+0000","last_clean":"2024-03-04T08:56:37.083731+0000","last_became_active":"2024-03-04T08:56:34.679821+0000","last_became_peered":"2024-03-04T08:56:34.679821+0000","last_unstale":"2024-03-04T08:56:37.083731+0000","last_undegraded":"2024-03-04T08:56:37.083731+0000","last_fullsized":"2024-03-04T08:56:37.083731+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:18:39.945962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.769029+0000","last_change":"2024-03-04T08:56:29.994017+0000","last_active":"2024-03-04T08:56:35.769029+0000","last_peered":"2024-03-04T08:56:35.769029+0000","last_clean":"2024-03-04T08:56:35.769029+0000","last_became_active":"2024-03-04T08:56:29.993687+0000","last_became_peered":"2024-03-04T08:56:29.993687+0000","last_unstale":"2024-03-04T08:56:35.769029+0000","last_undegraded":"2024-03-04T08:56:35.769029+0000","last_fullsized":"2024-03-04T08:56:35.769029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:32:51.542130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084890+0000","last_change":"2024-03-04T08:56:30.654446+0000","last_active":"2024-03-04T08:56:37.084890+0000","last_peered":"2024-03-04T08:56:37.084890+0000","last_clean":"2024-03-04T08:56:37.084890+0000","last_became_active":"2024-03-04T08:56:30.653920+0000","last_became_peered":"2024-03-04T08:56:30.653920+0000","last_unstale":"2024-03-04T08:56:37.084890+0000","last_undegraded":"2024-03-04T08:56:37.084890+0000","last_fullsized":"2024-03-04T08:56:37.084890+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:48:43.177893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083601+0000","last_change":"2024-03-04T08:56:34.680888+0000","last_active":"2024-03-04T08:56:37.083601+0000","last_peered":"2024-03-04T08:56:37.083601+0000","last_clean":"2024-03-04T08:56:37.083601+0000","last_became_active":"2024-03-04T08:56:34.679950+0000","last_became_peered":"2024-03-04T08:56:34.679950+0000","last_unstale":"2024-03-04T08:56:37.083601+0000","last_undegraded":"2024-03-04T08:56:37.083601+0000","last_fullsized":"2024-03-04T08:56:37.083601+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:08:26.660124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.583182+0000","last_change":"2024-03-04T08:56:29.986476+0000","last_active":"2024-03-04T08:56:36.583182+0000","last_peered":"2024-03-04T08:56:36.583182+0000","last_clean":"2024-03-04T08:56:36.583182+0000","last_became_active":"2024-03-04T08:56:29.986106+0000","last_became_peered":"2024-03-04T08:56:29.986106+0000","last_unstale":"2024-03-04T08:56:36.583182+0000","last_undegraded":"2024-03-04T08:56:36.583182+0000","last_fullsized":"2024-03-04T08:56:36.583182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:45:15.775106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.670715+0000","last_change":"2024-03-04T08:56:30.645841+0000","last_active":"2024-03-04T08:56:35.670715+0000","last_peered":"2024-03-04T08:56:35.670715+0000","last_clean":"2024-03-04T08:56:35.670715+0000","last_became_active":"2024-03-04T08:56:30.645459+0000","last_became_peered":"2024-03-04T08:56:30.645459+0000","last_unstale":"2024-03-04T08:56:35.670715+0000","last_undegraded":"2024-03-04T08:56:35.670715+0000","last_fullsized":"2024-03-04T08:56:35.670715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:37:51.204517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083635+0000","last_change":"2024-03-04T08:56:34.674562+0000","last_active":"2024-03-04T08:56:37.083635+0000","last_peered":"2024-03-04T08:56:37.083635+0000","last_clean":"2024-03-04T08:56:37.083635+0000","last_became_active":"2024-03-04T08:56:34.674239+0000","last_became_peered":"2024-03-04T08:56:34.674239+0000","last_unstale":"2024-03-04T08:56:37.083635+0000","last_undegraded":"2024-03-04T08:56:37.083635+0000","last_fullsized":"2024-03-04T08:56:37.083635+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:47:34.819487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.583010+0000","last_change":"2024-03-04T08:56:29.984025+0000","last_active":"2024-03-04T08:56:36.583010+0000","last_peered":"2024-03-04T08:56:36.583010+0000","last_clean":"2024-03-04T08:56:36.583010+0000","last_became_active":"2024-03-04T08:56:29.983676+0000","last_became_peered":"2024-03-04T08:56:29.983676+0000","last_unstale":"2024-03-04T08:56:36.583010+0000","last_undegraded":"2024-03-04T08:56:36.583010+0000","last_fullsized":"2024-03-04T08:56:36.583010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:48:24.920513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551593+0000","last_change":"2024-03-04T08:56:30.649301+0000","last_active":"2024-03-04T08:56:36.551593+0000","last_peered":"2024-03-04T08:56:36.551593+0000","last_clean":"2024-03-04T08:56:36.551593+0000","last_became_active":"2024-03-04T08:56:30.643987+0000","last_became_peered":"2024-03-04T08:56:30.643987+0000","last_unstale":"2024-03-04T08:56:36.551593+0000","last_undegraded":"2024-03-04T08:56:36.551593+0000","last_fullsized":"2024-03-04T08:56:36.551593+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:05:21.171953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.844201+0000","last_change":"2024-03-04T08:56:34.672893+0000","last_active":"2024-03-04T08:56:35.844201+0000","last_peered":"2024-03-04T08:56:35.844201+0000","last_clean":"2024-03-04T08:56:35.844201+0000","last_became_active":"2024-03-04T08:56:34.672364+0000","last_became_peered":"2024-03-04T08:56:34.672364+0000","last_unstale":"2024-03-04T08:56:35.844201+0000","last_undegraded":"2024-03-04T08:56:35.844201+0000","last_fullsized":"2024-03-04T08:56:35.844201+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:33:29.287772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'1","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551306+0000","last_change":"2024-03-04T08:56:29.986822+0000","last_active":"2024-03-04T08:56:36.551306+0000","last_peered":"2024-03-04T08:56:36.551306+0000","last_clean":"2024-03-04T08:56:36.551306+0000","last_became_active":"2024-03-04T08:56:29.986498+0000","last_became_peered":"2024-03-04T08:56:29.986498+0000","last_unstale":"2024-03-04T08:56:36.551306+0000","last_undegraded":"2024-03-04T08:56:36.551306+0000","last_fullsized":"2024-03-04T08:56:36.551306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:15:01.585997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.582694+0000","last_change":"2024-03-04T08:56:30.646164+0000","last_active":"2024-03-04T08:56:36.582694+0000","last_peered":"2024-03-04T08:56:36.582694+0000","last_clean":"2024-03-04T08:56:36.582694+0000","last_became_active":"2024-03-04T08:56:30.645843+0000","last_became_peered":"2024-03-04T08:56:30.645843+0000","last_unstale":"2024-03-04T08:56:36.582694+0000","last_undegraded":"2024-03-04T08:56:36.582694+0000","last_fullsized":"2024-03-04T08:56:36.582694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:11:48.082226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.843983+0000","last_change":"2024-03-04T08:56:34.680387+0000","last_active":"2024-03-04T08:56:35.843983+0000","last_peered":"2024-03-04T08:56:35.843983+0000","last_clean":"2024-03-04T08:56:35.843983+0000","last_became_active":"2024-03-04T08:56:34.680005+0000","last_became_peered":"2024-03-04T08:56:34.680005+0000","last_unstale":"2024-03-04T08:56:35.843983+0000","last_undegraded":"2024-03-04T08:56:35.843983+0000","last_fullsized":"2024-03-04T08:56:35.843983+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:04:40.403187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.374276+0000","last_change":"2024-03-04T08:56:29.984047+0000","last_active":"2024-03-04T08:56:36.374276+0000","last_peered":"2024-03-04T08:56:36.374276+0000","last_clean":"2024-03-04T08:56:36.374276+0000","last_became_active":"2024-03-04T08:56:29.983709+0000","last_became_peered":"2024-03-04T08:56:29.983709+0000","last_unstale":"2024-03-04T08:56:36.374276+0000","last_undegraded":"2024-03-04T08:56:36.374276+0000","last_fullsized":"2024-03-04T08:56:36.374276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:44:25.015970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551450+0000","last_change":"2024-03-04T08:56:30.649321+0000","last_active":"2024-03-04T08:56:36.551450+0000","last_peered":"2024-03-04T08:56:36.551450+0000","last_clean":"2024-03-04T08:56:36.551450+0000","last_became_active":"2024-03-04T08:56:30.648705+0000","last_became_peered":"2024-03-04T08:56:30.648705+0000","last_unstale":"2024-03-04T08:56:36.551450+0000","last_undegraded":"2024-03-04T08:56:36.551450+0000","last_fullsized":"2024-03-04T08:56:36.551450+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:26:12.549471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084375+0000","last_change":"2024-03-04T08:56:34.682560+0000","last_active":"2024-03-04T08:56:37.084375+0000","last_peered":"2024-03-04T08:56:37.084375+0000","last_clean":"2024-03-04T08:56:37.084375+0000","last_became_active":"2024-03-04T08:56:34.682156+0000","last_became_peered":"2024-03-04T08:56:34.682156+0000","last_unstale":"2024-03-04T08:56:37.084375+0000","last_undegraded":"2024-03-04T08:56:37.084375+0000","last_fullsized":"2024-03-04T08:56:37.084375+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:57:49.132436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083988+0000","last_change":"2024-03-04T08:56:29.984947+0000","last_active":"2024-03-04T08:56:37.083988+0000","last_peered":"2024-03-04T08:56:37.083988+0000","last_clean":"2024-03-04T08:56:37.083988+0000","last_became_active":"2024-03-04T08:56:29.984575+0000","last_became_peered":"2024-03-04T08:56:29.984575+0000","last_unstale":"2024-03-04T08:56:37.083988+0000","last_undegraded":"2024-03-04T08:56:37.083988+0000","last_fullsized":"2024-03-04T08:56:37.083988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:01:16.698417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.768447+0000","last_change":"2024-03-04T08:56:30.649000+0000","last_active":"2024-03-04T08:56:35.768447+0000","last_peered":"2024-03-04T08:56:35.768447+0000","last_clean":"2024-03-04T08:56:35.768447+0000","last_became_active":"2024-03-04T08:56:30.648700+0000","last_became_peered":"2024-03-04T08:56:30.648700+0000","last_unstale":"2024-03-04T08:56:35.768447+0000","last_undegraded":"2024-03-04T08:56:35.768447+0000","last_fullsized":"2024-03-04T08:56:35.768447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:46:09.384040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084654+0000","last_change":"2024-03-04T08:56:34.671042+0000","last_active":"2024-03-04T08:56:37.084654+0000","last_peered":"2024-03-04T08:56:37.084654+0000","last_clean":"2024-03-04T08:56:37.084654+0000","last_became_active":"2024-03-04T08:56:34.670683+0000","last_became_peered":"2024-03-04T08:56:34.670683+0000","last_unstale":"2024-03-04T08:56:37.084654+0000","last_undegraded":"2024-03-04T08:56:37.084654+0000","last_fullsized":"2024-03-04T08:56:37.084654+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:13:09.166986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551002+0000","last_change":"2024-03-04T08:56:29.983359+0000","last_active":"2024-03-04T08:56:36.551002+0000","last_peered":"2024-03-04T08:56:36.551002+0000","last_clean":"2024-03-04T08:56:36.551002+0000","last_became_active":"2024-03-04T08:56:29.983009+0000","last_became_peered":"2024-03-04T08:56:29.983009+0000","last_unstale":"2024-03-04T08:56:36.551002+0000","last_undegraded":"2024-03-04T08:56:36.551002+0000","last_fullsized":"2024-03-04T08:56:36.551002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:03:16.549109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.667758+0000","last_change":"2024-03-04T08:56:30.649612+0000","last_active":"2024-03-04T08:56:35.667758+0000","last_peered":"2024-03-04T08:56:35.667758+0000","last_clean":"2024-03-04T08:56:35.667758+0000","last_became_active":"2024-03-04T08:56:30.649299+0000","last_became_peered":"2024-03-04T08:56:30.649299+0000","last_unstale":"2024-03-04T08:56:35.667758+0000","last_undegraded":"2024-03-04T08:56:35.667758+0000","last_fullsized":"2024-03-04T08:56:35.667758+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:57:50.571696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.768590+0000","last_change":"2024-03-04T08:56:34.677974+0000","last_active":"2024-03-04T08:56:35.768590+0000","last_peered":"2024-03-04T08:56:35.768590+0000","last_clean":"2024-03-04T08:56:35.768590+0000","last_became_active":"2024-03-04T08:56:34.676628+0000","last_became_peered":"2024-03-04T08:56:34.676628+0000","last_unstale":"2024-03-04T08:56:35.768590+0000","last_undegraded":"2024-03-04T08:56:35.768590+0000","last_fullsized":"2024-03-04T08:56:35.768590+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:18:40.301336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.374580+0000","last_change":"2024-03-04T08:56:29.976760+0000","last_active":"2024-03-04T08:56:36.374580+0000","last_peered":"2024-03-04T08:56:36.374580+0000","last_clean":"2024-03-04T08:56:36.374580+0000","last_became_active":"2024-03-04T08:56:29.976446+0000","last_became_peered":"2024-03-04T08:56:29.976446+0000","last_unstale":"2024-03-04T08:56:36.374580+0000","last_undegraded":"2024-03-04T08:56:36.374580+0000","last_fullsized":"2024-03-04T08:56:36.374580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:46:35.038622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671108+0000","last_change":"2024-03-04T08:56:30.644326+0000","last_active":"2024-03-04T08:56:35.671108+0000","last_peered":"2024-03-04T08:56:35.671108+0000","last_clean":"2024-03-04T08:56:35.671108+0000","last_became_active":"2024-03-04T08:56:30.644017+0000","last_became_peered":"2024-03-04T08:56:30.644017+0000","last_unstale":"2024-03-04T08:56:35.671108+0000","last_undegraded":"2024-03-04T08:56:35.671108+0000","last_fullsized":"2024-03-04T08:56:35.671108+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:01:40.843915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.667772+0000","last_change":"2024-03-04T08:56:34.675757+0000","last_active":"2024-03-04T08:56:35.667772+0000","last_peered":"2024-03-04T08:56:35.667772+0000","last_clean":"2024-03-04T08:56:35.667772+0000","last_became_active":"2024-03-04T08:56:34.675375+0000","last_became_peered":"2024-03-04T08:56:34.675375+0000","last_unstale":"2024-03-04T08:56:35.667772+0000","last_undegraded":"2024-03-04T08:56:35.667772+0000","last_fullsized":"2024-03-04T08:56:35.667772+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:16:12.762024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671919+0000","last_change":"2024-03-04T08:56:29.987614+0000","last_active":"2024-03-04T08:56:35.671919+0000","last_peered":"2024-03-04T08:56:35.671919+0000","last_clean":"2024-03-04T08:56:35.671919+0000","last_became_active":"2024-03-04T08:56:29.986639+0000","last_became_peered":"2024-03-04T08:56:29.986639+0000","last_unstale":"2024-03-04T08:56:35.671919+0000","last_undegraded":"2024-03-04T08:56:35.671919+0000","last_fullsized":"2024-03-04T08:56:35.671919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:18:50.320069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084369+0000","last_change":"2024-03-04T08:56:30.655145+0000","last_active":"2024-03-04T08:56:37.084369+0000","last_peered":"2024-03-04T08:56:37.084369+0000","last_clean":"2024-03-04T08:56:37.084369+0000","last_became_active":"2024-03-04T08:56:30.654834+0000","last_became_peered":"2024-03-04T08:56:30.654834+0000","last_unstale":"2024-03-04T08:56:37.084369+0000","last_undegraded":"2024-03-04T08:56:37.084369+0000","last_fullsized":"2024-03-04T08:56:37.084369+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:45:19.567325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083773+0000","last_change":"2024-03-04T08:56:34.680530+0000","last_active":"2024-03-04T08:56:37.083773+0000","last_peered":"2024-03-04T08:56:37.083773+0000","last_clean":"2024-03-04T08:56:37.083773+0000","last_became_active":"2024-03-04T08:56:34.679860+0000","last_became_peered":"2024-03-04T08:56:34.679860+0000","last_unstale":"2024-03-04T08:56:37.083773+0000","last_undegraded":"2024-03-04T08:56:37.083773+0000","last_fullsized":"2024-03-04T08:56:37.083773+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:25:09.036145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'3","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551470+0000","last_change":"2024-03-04T08:56:29.982516+0000","last_active":"2024-03-04T08:56:36.551470+0000","last_peered":"2024-03-04T08:56:36.551470+0000","last_clean":"2024-03-04T08:56:36.551470+0000","last_became_active":"2024-03-04T08:56:29.982166+0000","last_became_peered":"2024-03-04T08:56:29.982166+0000","last_unstale":"2024-03-04T08:56:36.551470+0000","last_undegraded":"2024-03-04T08:56:36.551470+0000","last_fullsized":"2024-03-04T08:56:36.551470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:48:57.463584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.670904+0000","last_change":"2024-03-04T08:56:30.650030+0000","last_active":"2024-03-04T08:56:35.670904+0000","last_peered":"2024-03-04T08:56:35.670904+0000","last_clean":"2024-03-04T08:56:35.670904+0000","last_became_active":"2024-03-04T08:56:30.647826+0000","last_became_peered":"2024-03-04T08:56:30.647826+0000","last_unstale":"2024-03-04T08:56:35.670904+0000","last_undegraded":"2024-03-04T08:56:35.670904+0000","last_fullsized":"2024-03-04T08:56:35.670904+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:07:45.189935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.667654+0000","last_change":"2024-03-04T08:56:34.679189+0000","last_active":"2024-03-04T08:56:35.667654+0000","last_peered":"2024-03-04T08:56:35.667654+0000","last_clean":"2024-03-04T08:56:35.667654+0000","last_became_active":"2024-03-04T08:56:34.678744+0000","last_became_peered":"2024-03-04T08:56:34.678744+0000","last_unstale":"2024-03-04T08:56:35.667654+0000","last_undegraded":"2024-03-04T08:56:35.667654+0000","last_fullsized":"2024-03-04T08:56:35.667654+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:05:15.128703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'10","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:55.561704+0000","last_change":"2024-03-04T08:56:29.978908+0000","last_active":"2024-03-04T08:56:55.561704+0000","last_peered":"2024-03-04T08:56:55.561704+0000","last_clean":"2024-03-04T08:56:55.561704+0000","last_became_active":"2024-03-04T08:56:29.978604+0000","last_became_peered":"2024-03-04T08:56:29.978604+0000","last_unstale":"2024-03-04T08:56:55.561704+0000","last_undegraded":"2024-03-04T08:56:55.561704+0000","last_fullsized":"2024-03-04T08:56:55.561704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:03:45.683462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":7,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.582953+0000","last_change":"2024-03-04T08:56:30.647764+0000","last_active":"2024-03-04T08:56:36.582953+0000","last_peered":"2024-03-04T08:56:36.582953+0000","last_clean":"2024-03-04T08:56:36.582953+0000","last_became_active":"2024-03-04T08:56:30.647445+0000","last_became_peered":"2024-03-04T08:56:30.647445+0000","last_unstale":"2024-03-04T08:56:36.582953+0000","last_undegraded":"2024-03-04T08:56:36.582953+0000","last_fullsized":"2024-03-04T08:56:36.582953+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:13:25.135993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551915+0000","last_change":"2024-03-04T08:56:34.674674+0000","last_active":"2024-03-04T08:56:36.551915+0000","last_peered":"2024-03-04T08:56:36.551915+0000","last_clean":"2024-03-04T08:56:36.551915+0000","last_became_active":"2024-03-04T08:56:34.674138+0000","last_became_peered":"2024-03-04T08:56:34.674138+0000","last_unstale":"2024-03-04T08:56:36.551915+0000","last_undegraded":"2024-03-04T08:56:36.551915+0000","last_fullsized":"2024-03-04T08:56:36.551915+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:20:36.123111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.843998+0000","last_change":"2024-03-04T08:56:29.980997+0000","last_active":"2024-03-04T08:56:35.843998+0000","last_peered":"2024-03-04T08:56:35.843998+0000","last_clean":"2024-03-04T08:56:35.843998+0000","last_became_active":"2024-03-04T08:56:29.980651+0000","last_became_peered":"2024-03-04T08:56:29.980651+0000","last_unstale":"2024-03-04T08:56:35.843998+0000","last_undegraded":"2024-03-04T08:56:35.843998+0000","last_fullsized":"2024-03-04T08:56:35.843998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:23:33.434703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084161+0000","last_change":"2024-03-04T08:56:30.657229+0000","last_active":"2024-03-04T08:56:37.084161+0000","last_peered":"2024-03-04T08:56:37.084161+0000","last_clean":"2024-03-04T08:56:37.084161+0000","last_became_active":"2024-03-04T08:56:30.656877+0000","last_became_peered":"2024-03-04T08:56:30.656877+0000","last_unstale":"2024-03-04T08:56:37.084161+0000","last_undegraded":"2024-03-04T08:56:37.084161+0000","last_fullsized":"2024-03-04T08:56:37.084161+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:17:55.063418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084120+0000","last_change":"2024-03-04T08:56:34.680619+0000","last_active":"2024-03-04T08:56:37.084120+0000","last_peered":"2024-03-04T08:56:37.084120+0000","last_clean":"2024-03-04T08:56:37.084120+0000","last_became_active":"2024-03-04T08:56:34.680205+0000","last_became_peered":"2024-03-04T08:56:34.680205+0000","last_unstale":"2024-03-04T08:56:37.084120+0000","last_undegraded":"2024-03-04T08:56:37.084120+0000","last_fullsized":"2024-03-04T08:56:37.084120+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:42:14.650581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551374+0000","last_change":"2024-03-04T08:56:29.987759+0000","last_active":"2024-03-04T08:56:36.551374+0000","last_peered":"2024-03-04T08:56:36.551374+0000","last_clean":"2024-03-04T08:56:36.551374+0000","last_became_active":"2024-03-04T08:56:29.987458+0000","last_became_peered":"2024-03-04T08:56:29.987458+0000","last_unstale":"2024-03-04T08:56:36.551374+0000","last_undegraded":"2024-03-04T08:56:36.551374+0000","last_fullsized":"2024-03-04T08:56:36.551374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:36:18.878556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671741+0000","last_change":"2024-03-04T08:56:30.645935+0000","last_active":"2024-03-04T08:56:35.671741+0000","last_peered":"2024-03-04T08:56:35.671741+0000","last_clean":"2024-03-04T08:56:35.671741+0000","last_became_active":"2024-03-04T08:56:30.644370+0000","last_became_peered":"2024-03-04T08:56:30.644370+0000","last_unstale":"2024-03-04T08:56:35.671741+0000","last_undegraded":"2024-03-04T08:56:35.671741+0000","last_fullsized":"2024-03-04T08:56:35.671741+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:28:46.131719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.668127+0000","last_change":"2024-03-04T08:56:34.683727+0000","last_active":"2024-03-04T08:56:35.668127+0000","last_peered":"2024-03-04T08:56:35.668127+0000","last_clean":"2024-03-04T08:56:35.668127+0000","last_became_active":"2024-03-04T08:56:34.681865+0000","last_became_peered":"2024-03-04T08:56:34.681865+0000","last_unstale":"2024-03-04T08:56:35.668127+0000","last_undegraded":"2024-03-04T08:56:35.668127+0000","last_fullsized":"2024-03-04T08:56:35.668127+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:44:46.586791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083581+0000","last_change":"2024-03-04T08:56:29.983385+0000","last_active":"2024-03-04T08:56:37.083581+0000","last_peered":"2024-03-04T08:56:37.083581+0000","last_clean":"2024-03-04T08:56:37.083581+0000","last_became_active":"2024-03-04T08:56:29.983079+0000","last_became_peered":"2024-03-04T08:56:29.983079+0000","last_unstale":"2024-03-04T08:56:37.083581+0000","last_undegraded":"2024-03-04T08:56:37.083581+0000","last_fullsized":"2024-03-04T08:56:37.083581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:23:15.417389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671847+0000","last_change":"2024-03-04T08:56:30.641187+0000","last_active":"2024-03-04T08:56:35.671847+0000","last_peered":"2024-03-04T08:56:35.671847+0000","last_clean":"2024-03-04T08:56:35.671847+0000","last_became_active":"2024-03-04T08:56:30.640863+0000","last_became_peered":"2024-03-04T08:56:30.640863+0000","last_unstale":"2024-03-04T08:56:35.671847+0000","last_undegraded":"2024-03-04T08:56:35.671847+0000","last_fullsized":"2024-03-04T08:56:35.671847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:08:30.257806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084649+0000","last_change":"2024-03-04T08:56:34.684634+0000","last_active":"2024-03-04T08:56:37.084649+0000","last_peered":"2024-03-04T08:56:37.084649+0000","last_clean":"2024-03-04T08:56:37.084649+0000","last_became_active":"2024-03-04T08:56:34.684255+0000","last_became_peered":"2024-03-04T08:56:34.684255+0000","last_unstale":"2024-03-04T08:56:37.084649+0000","last_undegraded":"2024-03-04T08:56:37.084649+0000","last_fullsized":"2024-03-04T08:56:37.084649+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:30:59.381000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.582567+0000","last_change":"2024-03-04T08:56:29.994016+0000","last_active":"2024-03-04T08:56:36.582567+0000","last_peered":"2024-03-04T08:56:36.582567+0000","last_clean":"2024-03-04T08:56:36.582567+0000","last_became_active":"2024-03-04T08:56:29.993683+0000","last_became_peered":"2024-03-04T08:56:29.993683+0000","last_unstale":"2024-03-04T08:56:36.582567+0000","last_undegraded":"2024-03-04T08:56:36.582567+0000","last_fullsized":"2024-03-04T08:56:36.582567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:25:37.931248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551003+0000","last_change":"2024-03-04T08:56:30.647390+0000","last_active":"2024-03-04T08:56:36.551003+0000","last_peered":"2024-03-04T08:56:36.551003+0000","last_clean":"2024-03-04T08:56:36.551003+0000","last_became_active":"2024-03-04T08:56:30.643879+0000","last_became_peered":"2024-03-04T08:56:30.643879+0000","last_unstale":"2024-03-04T08:56:36.551003+0000","last_undegraded":"2024-03-04T08:56:36.551003+0000","last_fullsized":"2024-03-04T08:56:36.551003+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:41:13.030295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551192+0000","last_change":"2024-03-04T08:56:34.674836+0000","last_active":"2024-03-04T08:56:36.551192+0000","last_peered":"2024-03-04T08:56:36.551192+0000","last_clean":"2024-03-04T08:56:36.551192+0000","last_became_active":"2024-03-04T08:56:34.674418+0000","last_became_peered":"2024-03-04T08:56:34.674418+0000","last_unstale":"2024-03-04T08:56:36.551192+0000","last_undegraded":"2024-03-04T08:56:36.551192+0000","last_fullsized":"2024-03-04T08:56:36.551192+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:42:23.423166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.583144+0000","last_change":"2024-03-04T08:56:29.983195+0000","last_active":"2024-03-04T08:56:36.583144+0000","last_peered":"2024-03-04T08:56:36.583144+0000","last_clean":"2024-03-04T08:56:36.583144+0000","last_became_active":"2024-03-04T08:56:29.982881+0000","last_became_peered":"2024-03-04T08:56:29.982881+0000","last_unstale":"2024-03-04T08:56:36.583144+0000","last_undegraded":"2024-03-04T08:56:36.583144+0000","last_fullsized":"2024-03-04T08:56:36.583144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:59:12.802430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084168+0000","last_change":"2024-03-04T08:56:30.650452+0000","last_active":"2024-03-04T08:56:37.084168+0000","last_peered":"2024-03-04T08:56:37.084168+0000","last_clean":"2024-03-04T08:56:37.084168+0000","last_became_active":"2024-03-04T08:56:30.650148+0000","last_became_peered":"2024-03-04T08:56:30.650148+0000","last_unstale":"2024-03-04T08:56:37.084168+0000","last_undegraded":"2024-03-04T08:56:37.084168+0000","last_fullsized":"2024-03-04T08:56:37.084168+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:22:55.741505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.582668+0000","last_change":"2024-03-04T08:56:34.860608+0000","last_active":"2024-03-04T08:56:36.582668+0000","last_peered":"2024-03-04T08:56:36.582668+0000","last_clean":"2024-03-04T08:56:36.582668+0000","last_became_active":"2024-03-04T08:56:34.860087+0000","last_became_peered":"2024-03-04T08:56:34.860087+0000","last_unstale":"2024-03-04T08:56:36.582668+0000","last_undegraded":"2024-03-04T08:56:36.582668+0000","last_fullsized":"2024-03-04T08:56:36.582668+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:33:35.675886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.667783+0000","last_change":"2024-03-04T08:56:29.980947+0000","last_active":"2024-03-04T08:56:35.667783+0000","last_peered":"2024-03-04T08:56:35.667783+0000","last_clean":"2024-03-04T08:56:35.667783+0000","last_became_active":"2024-03-04T08:56:29.980625+0000","last_became_peered":"2024-03-04T08:56:29.980625+0000","last_unstale":"2024-03-04T08:56:35.667783+0000","last_undegraded":"2024-03-04T08:56:35.667783+0000","last_fullsized":"2024-03-04T08:56:35.667783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:53:53.944112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.374452+0000","last_change":"2024-03-04T08:56:30.642047+0000","last_active":"2024-03-04T08:56:36.374452+0000","last_peered":"2024-03-04T08:56:36.374452+0000","last_clean":"2024-03-04T08:56:36.374452+0000","last_became_active":"2024-03-04T08:56:30.641700+0000","last_became_peered":"2024-03-04T08:56:30.641700+0000","last_unstale":"2024-03-04T08:56:36.374452+0000","last_undegraded":"2024-03-04T08:56:36.374452+0000","last_fullsized":"2024-03-04T08:56:36.374452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:43:53.641512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084308+0000","last_change":"2024-03-04T08:56:34.674944+0000","last_active":"2024-03-04T08:56:37.084308+0000","last_peered":"2024-03-04T08:56:37.084308+0000","last_clean":"2024-03-04T08:56:37.084308+0000","last_became_active":"2024-03-04T08:56:34.674000+0000","last_became_peered":"2024-03-04T08:56:34.674000+0000","last_unstale":"2024-03-04T08:56:37.084308+0000","last_undegraded":"2024-03-04T08:56:37.084308+0000","last_fullsized":"2024-03-04T08:56:37.084308+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:26:19.386329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671613+0000","last_change":"2024-03-04T08:56:29.989020+0000","last_active":"2024-03-04T08:56:35.671613+0000","last_peered":"2024-03-04T08:56:35.671613+0000","last_clean":"2024-03-04T08:56:35.671613+0000","last_became_active":"2024-03-04T08:56:29.988751+0000","last_became_peered":"2024-03-04T08:56:29.988751+0000","last_unstale":"2024-03-04T08:56:35.671613+0000","last_undegraded":"2024-03-04T08:56:35.671613+0000","last_fullsized":"2024-03-04T08:56:35.671613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:38:45.003221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.374333+0000","last_change":"2024-03-04T08:56:30.649976+0000","last_active":"2024-03-04T08:56:36.374333+0000","last_peered":"2024-03-04T08:56:36.374333+0000","last_clean":"2024-03-04T08:56:36.374333+0000","last_became_active":"2024-03-04T08:56:30.649680+0000","last_became_peered":"2024-03-04T08:56:30.649680+0000","last_unstale":"2024-03-04T08:56:36.374333+0000","last_undegraded":"2024-03-04T08:56:36.374333+0000","last_fullsized":"2024-03-04T08:56:36.374333+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:50:02.197087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084391+0000","last_change":"2024-03-04T08:56:34.860654+0000","last_active":"2024-03-04T08:56:37.084391+0000","last_peered":"2024-03-04T08:56:37.084391+0000","last_clean":"2024-03-04T08:56:37.084391+0000","last_became_active":"2024-03-04T08:56:34.860170+0000","last_became_peered":"2024-03-04T08:56:34.860170+0000","last_unstale":"2024-03-04T08:56:37.084391+0000","last_undegraded":"2024-03-04T08:56:37.084391+0000","last_fullsized":"2024-03-04T08:56:37.084391+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:08:49.703317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551196+0000","last_change":"2024-03-04T08:56:30.643510+0000","last_active":"2024-03-04T08:56:36.551196+0000","last_peered":"2024-03-04T08:56:36.551196+0000","last_clean":"2024-03-04T08:56:36.551196+0000","last_became_active":"2024-03-04T08:56:30.643141+0000","last_became_peered":"2024-03-04T08:56:30.643141+0000","last_unstale":"2024-03-04T08:56:36.551196+0000","last_undegraded":"2024-03-04T08:56:36.551196+0000","last_fullsized":"2024-03-04T08:56:36.551196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:36:03.619735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083885+0000","last_change":"2024-03-04T08:56:29.994030+0000","last_active":"2024-03-04T08:56:37.083885+0000","last_peered":"2024-03-04T08:56:37.083885+0000","last_clean":"2024-03-04T08:56:37.083885+0000","last_became_active":"2024-03-04T08:56:29.993698+0000","last_became_peered":"2024-03-04T08:56:29.993698+0000","last_unstale":"2024-03-04T08:56:37.083885+0000","last_undegraded":"2024-03-04T08:56:37.083885+0000","last_fullsized":"2024-03-04T08:56:37.083885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:28:38.609330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.672003+0000","last_change":"2024-03-04T08:56:34.678337+0000","last_active":"2024-03-04T08:56:35.672003+0000","last_peered":"2024-03-04T08:56:35.672003+0000","last_clean":"2024-03-04T08:56:35.672003+0000","last_became_active":"2024-03-04T08:56:34.678091+0000","last_became_peered":"2024-03-04T08:56:34.678091+0000","last_unstale":"2024-03-04T08:56:35.672003+0000","last_undegraded":"2024-03-04T08:56:35.672003+0000","last_fullsized":"2024-03-04T08:56:35.672003+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:05:54.075257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.768434+0000","last_change":"2024-03-04T08:56:30.641609+0000","last_active":"2024-03-04T08:56:35.768434+0000","last_peered":"2024-03-04T08:56:35.768434+0000","last_clean":"2024-03-04T08:56:35.768434+0000","last_became_active":"2024-03-04T08:56:30.641323+0000","last_became_peered":"2024-03-04T08:56:30.641323+0000","last_unstale":"2024-03-04T08:56:35.768434+0000","last_undegraded":"2024-03-04T08:56:35.768434+0000","last_fullsized":"2024-03-04T08:56:35.768434+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:39:03.935424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083567+0000","last_change":"2024-03-04T08:56:29.976581+0000","last_active":"2024-03-04T08:56:37.083567+0000","last_peered":"2024-03-04T08:56:37.083567+0000","last_clean":"2024-03-04T08:56:37.083567+0000","last_became_active":"2024-03-04T08:56:29.976215+0000","last_became_peered":"2024-03-04T08:56:29.976215+0000","last_unstale":"2024-03-04T08:56:37.083567+0000","last_undegraded":"2024-03-04T08:56:37.083567+0000","last_fullsized":"2024-03-04T08:56:37.083567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:53:29.040404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":356,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671484+0000","last_change":"2024-03-04T08:55:10.344909+0000","last_active":"2024-03-04T08:56:35.671484+0000","last_peered":"2024-03-04T08:56:35.671484+0000","last_clean":"2024-03-04T08:56:35.671484+0000","last_became_active":"2024-03-04T08:55:10.344424+0000","last_became_peered":"2024-03-04T08:55:10.344424+0000","last_unstale":"2024-03-04T08:56:35.671484+0000","last_undegraded":"2024-03-04T08:56:35.671484+0000","last_fullsized":"2024-03-04T08:56:35.671484+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:51:15.334055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:51:15.334055+0000","last_clean_scrub_stamp":"2024-03-04T08:51:15.334055+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:36:29.766101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551809+0000","last_change":"2024-03-04T08:56:34.861141+0000","last_active":"2024-03-04T08:56:36.551809+0000","last_peered":"2024-03-04T08:56:36.551809+0000","last_clean":"2024-03-04T08:56:36.551809+0000","last_became_active":"2024-03-04T08:56:34.860618+0000","last_became_peered":"2024-03-04T08:56:34.860618+0000","last_unstale":"2024-03-04T08:56:36.551809+0000","last_undegraded":"2024-03-04T08:56:36.551809+0000","last_fullsized":"2024-03-04T08:56:36.551809+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:53:20.918385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083670+0000","last_change":"2024-03-04T08:56:30.644248+0000","last_active":"2024-03-04T08:56:37.083670+0000","last_peered":"2024-03-04T08:56:37.083670+0000","last_clean":"2024-03-04T08:56:37.083670+0000","last_became_active":"2024-03-04T08:56:30.643956+0000","last_became_peered":"2024-03-04T08:56:30.643956+0000","last_unstale":"2024-03-04T08:56:37.083670+0000","last_undegraded":"2024-03-04T08:56:37.083670+0000","last_fullsized":"2024-03-04T08:56:37.083670+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:19:24.298145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671596+0000","last_change":"2024-03-04T08:56:29.987584+0000","last_active":"2024-03-04T08:56:35.671596+0000","last_peered":"2024-03-04T08:56:35.671596+0000","last_clean":"2024-03-04T08:56:35.671596+0000","last_became_active":"2024-03-04T08:56:29.986741+0000","last_became_peered":"2024-03-04T08:56:29.986741+0000","last_unstale":"2024-03-04T08:56:35.671596+0000","last_undegraded":"2024-03-04T08:56:35.671596+0000","last_fullsized":"2024-03-04T08:56:35.671596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:46:19.672027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084402+0000","last_change":"2024-03-04T08:56:34.675485+0000","last_active":"2024-03-04T08:56:37.084402+0000","last_peered":"2024-03-04T08:56:37.084402+0000","last_clean":"2024-03-04T08:56:37.084402+0000","last_became_active":"2024-03-04T08:56:34.674961+0000","last_became_peered":"2024-03-04T08:56:34.674961+0000","last_unstale":"2024-03-04T08:56:37.084402+0000","last_undegraded":"2024-03-04T08:56:37.084402+0000","last_fullsized":"2024-03-04T08:56:37.084402+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:39:43.387849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.583044+0000","last_change":"2024-03-04T08:56:29.992287+0000","last_active":"2024-03-04T08:56:36.583044+0000","last_peered":"2024-03-04T08:56:36.583044+0000","last_clean":"2024-03-04T08:56:36.583044+0000","last_became_active":"2024-03-04T08:56:29.991998+0000","last_became_peered":"2024-03-04T08:56:29.991998+0000","last_unstale":"2024-03-04T08:56:36.583044+0000","last_undegraded":"2024-03-04T08:56:36.583044+0000","last_fullsized":"2024-03-04T08:56:36.583044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:33:10.557684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084308+0000","last_change":"2024-03-04T08:56:30.644190+0000","last_active":"2024-03-04T08:56:37.084308+0000","last_peered":"2024-03-04T08:56:37.084308+0000","last_clean":"2024-03-04T08:56:37.084308+0000","last_became_active":"2024-03-04T08:56:30.643783+0000","last_became_peered":"2024-03-04T08:56:30.643783+0000","last_unstale":"2024-03-04T08:56:37.084308+0000","last_undegraded":"2024-03-04T08:56:37.084308+0000","last_fullsized":"2024-03-04T08:56:37.084308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:25:12.886305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.768587+0000","last_change":"2024-03-04T08:56:34.860415+0000","last_active":"2024-03-04T08:56:35.768587+0000","last_peered":"2024-03-04T08:56:35.768587+0000","last_clean":"2024-03-04T08:56:35.768587+0000","last_became_active":"2024-03-04T08:56:34.860008+0000","last_became_peered":"2024-03-04T08:56:34.860008+0000","last_unstale":"2024-03-04T08:56:35.768587+0000","last_undegraded":"2024-03-04T08:56:35.768587+0000","last_fullsized":"2024-03-04T08:56:35.768587+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:55:45.673567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'1","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.668116+0000","last_change":"2024-03-04T08:56:29.989599+0000","last_active":"2024-03-04T08:56:35.668116+0000","last_peered":"2024-03-04T08:56:35.668116+0000","last_clean":"2024-03-04T08:56:35.668116+0000","last_became_active":"2024-03-04T08:56:29.989281+0000","last_became_peered":"2024-03-04T08:56:29.989281+0000","last_unstale":"2024-03-04T08:56:35.668116+0000","last_undegraded":"2024-03-04T08:56:35.668116+0000","last_fullsized":"2024-03-04T08:56:35.668116+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:59:04.006117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.768979+0000","last_change":"2024-03-04T08:56:30.640315+0000","last_active":"2024-03-04T08:56:35.768979+0000","last_peered":"2024-03-04T08:56:35.768979+0000","last_clean":"2024-03-04T08:56:35.768979+0000","last_became_active":"2024-03-04T08:56:30.640001+0000","last_became_peered":"2024-03-04T08:56:30.640001+0000","last_unstale":"2024-03-04T08:56:35.768979+0000","last_undegraded":"2024-03-04T08:56:35.768979+0000","last_fullsized":"2024-03-04T08:56:35.768979+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:43:57.049948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.843778+0000","last_change":"2024-03-04T08:56:34.667973+0000","last_active":"2024-03-04T08:56:35.843778+0000","last_peered":"2024-03-04T08:56:35.843778+0000","last_clean":"2024-03-04T08:56:35.843778+0000","last_became_active":"2024-03-04T08:56:34.667527+0000","last_became_peered":"2024-03-04T08:56:34.667527+0000","last_unstale":"2024-03-04T08:56:35.843778+0000","last_undegraded":"2024-03-04T08:56:35.843778+0000","last_fullsized":"2024-03-04T08:56:35.843778+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:07:23.991933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083772+0000","last_change":"2024-03-04T08:56:29.982263+0000","last_active":"2024-03-04T08:56:37.083772+0000","last_peered":"2024-03-04T08:56:37.083772+0000","last_clean":"2024-03-04T08:56:37.083772+0000","last_became_active":"2024-03-04T08:56:29.981802+0000","last_became_peered":"2024-03-04T08:56:29.981802+0000","last_unstale":"2024-03-04T08:56:37.083772+0000","last_undegraded":"2024-03-04T08:56:37.083772+0000","last_fullsized":"2024-03-04T08:56:37.083772+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:24:33.351428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084784+0000","last_change":"2024-03-04T08:56:30.657847+0000","last_active":"2024-03-04T08:56:37.084784+0000","last_peered":"2024-03-04T08:56:37.084784+0000","last_clean":"2024-03-04T08:56:37.084784+0000","last_became_active":"2024-03-04T08:56:30.657533+0000","last_became_peered":"2024-03-04T08:56:30.657533+0000","last_unstale":"2024-03-04T08:56:37.084784+0000","last_undegraded":"2024-03-04T08:56:37.084784+0000","last_fullsized":"2024-03-04T08:56:37.084784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:11:01.914455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083824+0000","last_change":"2024-03-04T08:56:34.861093+0000","last_active":"2024-03-04T08:56:37.083824+0000","last_peered":"2024-03-04T08:56:37.083824+0000","last_clean":"2024-03-04T08:56:37.083824+0000","last_became_active":"2024-03-04T08:56:34.860660+0000","last_became_peered":"2024-03-04T08:56:34.860660+0000","last_unstale":"2024-03-04T08:56:37.083824+0000","last_undegraded":"2024-03-04T08:56:37.083824+0000","last_fullsized":"2024-03-04T08:56:37.083824+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:37:38.424914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.670571+0000","last_change":"2024-03-04T08:56:29.990546+0000","last_active":"2024-03-04T08:56:35.670571+0000","last_peered":"2024-03-04T08:56:35.670571+0000","last_clean":"2024-03-04T08:56:35.670571+0000","last_became_active":"2024-03-04T08:56:29.990342+0000","last_became_peered":"2024-03-04T08:56:29.990342+0000","last_unstale":"2024-03-04T08:56:35.670571+0000","last_undegraded":"2024-03-04T08:56:35.670571+0000","last_fullsized":"2024-03-04T08:56:35.670571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:01:06.929112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.582559+0000","last_change":"2024-03-04T08:56:30.653228+0000","last_active":"2024-03-04T08:56:36.582559+0000","last_peered":"2024-03-04T08:56:36.582559+0000","last_clean":"2024-03-04T08:56:36.582559+0000","last_became_active":"2024-03-04T08:56:30.652926+0000","last_became_peered":"2024-03-04T08:56:30.652926+0000","last_unstale":"2024-03-04T08:56:36.582559+0000","last_undegraded":"2024-03-04T08:56:36.582559+0000","last_fullsized":"2024-03-04T08:56:36.582559+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:06:27.728135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.668292+0000","last_change":"2024-03-04T08:56:34.681096+0000","last_active":"2024-03-04T08:56:35.668292+0000","last_peered":"2024-03-04T08:56:35.668292+0000","last_clean":"2024-03-04T08:56:35.668292+0000","last_became_active":"2024-03-04T08:56:34.680718+0000","last_became_peered":"2024-03-04T08:56:34.680718+0000","last_unstale":"2024-03-04T08:56:35.668292+0000","last_undegraded":"2024-03-04T08:56:35.668292+0000","last_fullsized":"2024-03-04T08:56:35.668292+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:04:58.908904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.768441+0000","last_change":"2024-03-04T08:56:29.988039+0000","last_active":"2024-03-04T08:56:35.768441+0000","last_peered":"2024-03-04T08:56:35.768441+0000","last_clean":"2024-03-04T08:56:35.768441+0000","last_became_active":"2024-03-04T08:56:29.987310+0000","last_became_peered":"2024-03-04T08:56:29.987310+0000","last_unstale":"2024-03-04T08:56:35.768441+0000","last_undegraded":"2024-03-04T08:56:35.768441+0000","last_fullsized":"2024-03-04T08:56:35.768441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:19:39.319965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083909+0000","last_change":"2024-03-04T08:56:30.649094+0000","last_active":"2024-03-04T08:56:37.083909+0000","last_peered":"2024-03-04T08:56:37.083909+0000","last_clean":"2024-03-04T08:56:37.083909+0000","last_became_active":"2024-03-04T08:56:30.648776+0000","last_became_peered":"2024-03-04T08:56:30.648776+0000","last_unstale":"2024-03-04T08:56:37.083909+0000","last_undegraded":"2024-03-04T08:56:37.083909+0000","last_fullsized":"2024-03-04T08:56:37.083909+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:23:17.738792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083859+0000","last_change":"2024-03-04T08:56:34.680923+0000","last_active":"2024-03-04T08:56:37.083859+0000","last_peered":"2024-03-04T08:56:37.083859+0000","last_clean":"2024-03-04T08:56:37.083859+0000","last_became_active":"2024-03-04T08:56:34.680114+0000","last_became_peered":"2024-03-04T08:56:34.680114+0000","last_unstale":"2024-03-04T08:56:37.083859+0000","last_undegraded":"2024-03-04T08:56:37.083859+0000","last_fullsized":"2024-03-04T08:56:37.083859+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:05:53.434735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.843884+0000","last_change":"2024-03-04T08:56:29.985725+0000","last_active":"2024-03-04T08:56:35.843884+0000","last_peered":"2024-03-04T08:56:35.843884+0000","last_clean":"2024-03-04T08:56:35.843884+0000","last_became_active":"2024-03-04T08:56:29.985363+0000","last_became_peered":"2024-03-04T08:56:29.985363+0000","last_unstale":"2024-03-04T08:56:35.843884+0000","last_undegraded":"2024-03-04T08:56:35.843884+0000","last_fullsized":"2024-03-04T08:56:35.843884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:15:07.432128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.768545+0000","last_change":"2024-03-04T08:56:30.654617+0000","last_active":"2024-03-04T08:56:35.768545+0000","last_peered":"2024-03-04T08:56:35.768545+0000","last_clean":"2024-03-04T08:56:35.768545+0000","last_became_active":"2024-03-04T08:56:30.653341+0000","last_became_peered":"2024-03-04T08:56:30.653341+0000","last_unstale":"2024-03-04T08:56:35.768545+0000","last_undegraded":"2024-03-04T08:56:35.768545+0000","last_fullsized":"2024-03-04T08:56:35.768545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:10:26.234514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083933+0000","last_change":"2024-03-04T08:56:34.860680+0000","last_active":"2024-03-04T08:56:37.083933+0000","last_peered":"2024-03-04T08:56:37.083933+0000","last_clean":"2024-03-04T08:56:37.083933+0000","last_became_active":"2024-03-04T08:56:34.860290+0000","last_became_peered":"2024-03-04T08:56:34.860290+0000","last_unstale":"2024-03-04T08:56:37.083933+0000","last_undegraded":"2024-03-04T08:56:37.083933+0000","last_fullsized":"2024-03-04T08:56:37.083933+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:14:32.126839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.582698+0000","last_change":"2024-03-04T08:56:29.989843+0000","last_active":"2024-03-04T08:56:36.582698+0000","last_peered":"2024-03-04T08:56:36.582698+0000","last_clean":"2024-03-04T08:56:36.582698+0000","last_became_active":"2024-03-04T08:56:29.989508+0000","last_became_peered":"2024-03-04T08:56:29.989508+0000","last_unstale":"2024-03-04T08:56:36.582698+0000","last_undegraded":"2024-03-04T08:56:36.582698+0000","last_fullsized":"2024-03-04T08:56:36.582698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:46:01.738580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671715+0000","last_change":"2024-03-04T08:56:30.644598+0000","last_active":"2024-03-04T08:56:35.671715+0000","last_peered":"2024-03-04T08:56:35.671715+0000","last_clean":"2024-03-04T08:56:35.671715+0000","last_became_active":"2024-03-04T08:56:30.644263+0000","last_became_peered":"2024-03-04T08:56:30.644263+0000","last_unstale":"2024-03-04T08:56:35.671715+0000","last_undegraded":"2024-03-04T08:56:35.671715+0000","last_fullsized":"2024-03-04T08:56:35.671715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:47:07.013271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671176+0000","last_change":"2024-03-04T08:56:34.675116+0000","last_active":"2024-03-04T08:56:35.671176+0000","last_peered":"2024-03-04T08:56:35.671176+0000","last_clean":"2024-03-04T08:56:35.671176+0000","last_became_active":"2024-03-04T08:56:34.674727+0000","last_became_peered":"2024-03-04T08:56:34.674727+0000","last_unstale":"2024-03-04T08:56:35.671176+0000","last_undegraded":"2024-03-04T08:56:35.671176+0000","last_fullsized":"2024-03-04T08:56:35.671176+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:56:23.461299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.844227+0000","last_change":"2024-03-04T08:56:29.983545+0000","last_active":"2024-03-04T08:56:35.844227+0000","last_peered":"2024-03-04T08:56:35.844227+0000","last_clean":"2024-03-04T08:56:35.844227+0000","last_became_active":"2024-03-04T08:56:29.983244+0000","last_became_peered":"2024-03-04T08:56:29.983244+0000","last_unstale":"2024-03-04T08:56:35.844227+0000","last_undegraded":"2024-03-04T08:56:35.844227+0000","last_fullsized":"2024-03-04T08:56:35.844227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:59:43.475203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.672182+0000","last_change":"2024-03-04T08:56:30.641244+0000","last_active":"2024-03-04T08:56:35.672182+0000","last_peered":"2024-03-04T08:56:35.672182+0000","last_clean":"2024-03-04T08:56:35.672182+0000","last_became_active":"2024-03-04T08:56:30.640952+0000","last_became_peered":"2024-03-04T08:56:30.640952+0000","last_unstale":"2024-03-04T08:56:35.672182+0000","last_undegraded":"2024-03-04T08:56:35.672182+0000","last_fullsized":"2024-03-04T08:56:35.672182+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:38:51.134697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083960+0000","last_change":"2024-03-04T08:56:34.670075+0000","last_active":"2024-03-04T08:56:37.083960+0000","last_peered":"2024-03-04T08:56:37.083960+0000","last_clean":"2024-03-04T08:56:37.083960+0000","last_became_active":"2024-03-04T08:56:34.669715+0000","last_became_peered":"2024-03-04T08:56:34.669715+0000","last_unstale":"2024-03-04T08:56:37.083960+0000","last_undegraded":"2024-03-04T08:56:37.083960+0000","last_fullsized":"2024-03-04T08:56:37.083960+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:32:01.768610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551603+0000","last_change":"2024-03-04T08:56:29.980781+0000","last_active":"2024-03-04T08:56:36.551603+0000","last_peered":"2024-03-04T08:56:36.551603+0000","last_clean":"2024-03-04T08:56:36.551603+0000","last_became_active":"2024-03-04T08:56:29.980446+0000","last_became_peered":"2024-03-04T08:56:29.980446+0000","last_unstale":"2024-03-04T08:56:36.551603+0000","last_undegraded":"2024-03-04T08:56:36.551603+0000","last_fullsized":"2024-03-04T08:56:36.551603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:05:37.325902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671770+0000","last_change":"2024-03-04T08:56:30.648350+0000","last_active":"2024-03-04T08:56:35.671770+0000","last_peered":"2024-03-04T08:56:35.671770+0000","last_clean":"2024-03-04T08:56:35.671770+0000","last_became_active":"2024-03-04T08:56:30.648059+0000","last_became_peered":"2024-03-04T08:56:30.648059+0000","last_unstale":"2024-03-04T08:56:35.671770+0000","last_undegraded":"2024-03-04T08:56:35.671770+0000","last_fullsized":"2024-03-04T08:56:35.671770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:59:07.571168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.550909+0000","last_change":"2024-03-04T08:56:34.673401+0000","last_active":"2024-03-04T08:56:36.550909+0000","last_peered":"2024-03-04T08:56:36.550909+0000","last_clean":"2024-03-04T08:56:36.550909+0000","last_became_active":"2024-03-04T08:56:34.672596+0000","last_became_peered":"2024-03-04T08:56:34.672596+0000","last_unstale":"2024-03-04T08:56:36.550909+0000","last_undegraded":"2024-03-04T08:56:36.550909+0000","last_fullsized":"2024-03-04T08:56:36.550909+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:39:56.613366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084498+0000","last_change":"2024-03-04T08:56:29.988331+0000","last_active":"2024-03-04T08:56:37.084498+0000","last_peered":"2024-03-04T08:56:37.084498+0000","last_clean":"2024-03-04T08:56:37.084498+0000","last_became_active":"2024-03-04T08:56:29.988009+0000","last_became_peered":"2024-03-04T08:56:29.988009+0000","last_unstale":"2024-03-04T08:56:37.084498+0000","last_undegraded":"2024-03-04T08:56:37.084498+0000","last_fullsized":"2024-03-04T08:56:37.084498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:50:43.693924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671958+0000","last_change":"2024-03-04T08:56:30.647443+0000","last_active":"2024-03-04T08:56:35.671958+0000","last_peered":"2024-03-04T08:56:35.671958+0000","last_clean":"2024-03-04T08:56:35.671958+0000","last_became_active":"2024-03-04T08:56:30.647106+0000","last_became_peered":"2024-03-04T08:56:30.647106+0000","last_unstale":"2024-03-04T08:56:35.671958+0000","last_undegraded":"2024-03-04T08:56:35.671958+0000","last_fullsized":"2024-03-04T08:56:35.671958+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:09:56.320806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083968+0000","last_change":"2024-03-04T08:56:30.651243+0000","last_active":"2024-03-04T08:56:37.083968+0000","last_peered":"2024-03-04T08:56:37.083968+0000","last_clean":"2024-03-04T08:56:37.083968+0000","last_became_active":"2024-03-04T08:56:30.650781+0000","last_became_peered":"2024-03-04T08:56:30.650781+0000","last_unstale":"2024-03-04T08:56:37.083968+0000","last_undegraded":"2024-03-04T08:56:37.083968+0000","last_fullsized":"2024-03-04T08:56:37.083968+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:58:16.723542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.844206+0000","last_change":"2024-03-04T08:56:30.645737+0000","last_active":"2024-03-04T08:56:35.844206+0000","last_peered":"2024-03-04T08:56:35.844206+0000","last_clean":"2024-03-04T08:56:35.844206+0000","last_became_active":"2024-03-04T08:56:30.645293+0000","last_became_peered":"2024-03-04T08:56:30.645293+0000","last_unstale":"2024-03-04T08:56:35.844206+0000","last_undegraded":"2024-03-04T08:56:35.844206+0000","last_fullsized":"2024-03-04T08:56:35.844206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:56:04.302166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.843732+0000","last_change":"2024-03-04T08:56:30.649851+0000","last_active":"2024-03-04T08:56:35.843732+0000","last_peered":"2024-03-04T08:56:35.843732+0000","last_clean":"2024-03-04T08:56:35.843732+0000","last_became_active":"2024-03-04T08:56:30.649528+0000","last_became_peered":"2024-03-04T08:56:30.649528+0000","last_unstale":"2024-03-04T08:56:35.843732+0000","last_undegraded":"2024-03-04T08:56:35.843732+0000","last_fullsized":"2024-03-04T08:56:35.843732+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:36:50.431866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084432+0000","last_change":"2024-03-04T08:56:30.646893+0000","last_active":"2024-03-04T08:56:37.084432+0000","last_peered":"2024-03-04T08:56:37.084432+0000","last_clean":"2024-03-04T08:56:37.084432+0000","last_became_active":"2024-03-04T08:56:30.646615+0000","last_became_peered":"2024-03-04T08:56:30.646615+0000","last_unstale":"2024-03-04T08:56:37.084432+0000","last_undegraded":"2024-03-04T08:56:37.084432+0000","last_fullsized":"2024-03-04T08:56:37.084432+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:07:12.584686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084415+0000","last_change":"2024-03-04T08:56:30.649772+0000","last_active":"2024-03-04T08:56:37.084415+0000","last_peered":"2024-03-04T08:56:37.084415+0000","last_clean":"2024-03-04T08:56:37.084415+0000","last_became_active":"2024-03-04T08:56:30.649480+0000","last_became_peered":"2024-03-04T08:56:30.649480+0000","last_unstale":"2024-03-04T08:56:37.084415+0000","last_undegraded":"2024-03-04T08:56:37.084415+0000","last_fullsized":"2024-03-04T08:56:37.084415+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:14:10.553290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084181+0000","last_change":"2024-03-04T08:56:30.650819+0000","last_active":"2024-03-04T08:56:37.084181+0000","last_peered":"2024-03-04T08:56:37.084181+0000","last_clean":"2024-03-04T08:56:37.084181+0000","last_became_active":"2024-03-04T08:56:30.650505+0000","last_became_peered":"2024-03-04T08:56:30.650505+0000","last_unstale":"2024-03-04T08:56:37.084181+0000","last_undegraded":"2024-03-04T08:56:37.084181+0000","last_fullsized":"2024-03-04T08:56:37.084181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:53:36.877201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.844452+0000","last_change":"2024-03-04T08:56:30.646870+0000","last_active":"2024-03-04T08:56:35.844452+0000","last_peered":"2024-03-04T08:56:35.844452+0000","last_clean":"2024-03-04T08:56:35.844452+0000","last_became_active":"2024-03-04T08:56:30.646369+0000","last_became_peered":"2024-03-04T08:56:30.646369+0000","last_unstale":"2024-03-04T08:56:35.844452+0000","last_undegraded":"2024-03-04T08:56:35.844452+0000","last_fullsized":"2024-03-04T08:56:35.844452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:25:34.969089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.769070+0000","last_change":"2024-03-04T08:56:30.640854+0000","last_active":"2024-03-04T08:56:35.769070+0000","last_peered":"2024-03-04T08:56:35.769070+0000","last_clean":"2024-03-04T08:56:35.769070+0000","last_became_active":"2024-03-04T08:56:30.640484+0000","last_became_peered":"2024-03-04T08:56:30.640484+0000","last_unstale":"2024-03-04T08:56:35.769070+0000","last_undegraded":"2024-03-04T08:56:35.769070+0000","last_fullsized":"2024-03-04T08:56:35.769070+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:16:01.298720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551079+0000","last_change":"2024-03-04T08:56:30.648593+0000","last_active":"2024-03-04T08:56:36.551079+0000","last_peered":"2024-03-04T08:56:36.551079+0000","last_clean":"2024-03-04T08:56:36.551079+0000","last_became_active":"2024-03-04T08:56:30.648287+0000","last_became_peered":"2024-03-04T08:56:30.648287+0000","last_unstale":"2024-03-04T08:56:36.551079+0000","last_undegraded":"2024-03-04T08:56:36.551079+0000","last_fullsized":"2024-03-04T08:56:36.551079+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:04:52.525357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.768566+0000","last_change":"2024-03-04T08:56:30.640284+0000","last_active":"2024-03-04T08:56:35.768566+0000","last_peered":"2024-03-04T08:56:35.768566+0000","last_clean":"2024-03-04T08:56:35.768566+0000","last_became_active":"2024-03-04T08:56:30.639968+0000","last_became_peered":"2024-03-04T08:56:30.639968+0000","last_unstale":"2024-03-04T08:56:35.768566+0000","last_undegraded":"2024-03-04T08:56:35.768566+0000","last_fullsized":"2024-03-04T08:56:35.768566+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:37:26.063318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.670931+0000","last_change":"2024-03-04T08:56:30.649547+0000","last_active":"2024-03-04T08:56:35.670931+0000","last_peered":"2024-03-04T08:56:35.670931+0000","last_clean":"2024-03-04T08:56:35.670931+0000","last_became_active":"2024-03-04T08:56:30.649170+0000","last_became_peered":"2024-03-04T08:56:30.649170+0000","last_unstale":"2024-03-04T08:56:35.670931+0000","last_undegraded":"2024-03-04T08:56:35.670931+0000","last_fullsized":"2024-03-04T08:56:35.670931+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:02:22.805496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083961+0000","last_change":"2024-03-04T08:56:30.650892+0000","last_active":"2024-03-04T08:56:37.083961+0000","last_peered":"2024-03-04T08:56:37.083961+0000","last_clean":"2024-03-04T08:56:37.083961+0000","last_became_active":"2024-03-04T08:56:30.650442+0000","last_became_peered":"2024-03-04T08:56:30.650442+0000","last_unstale":"2024-03-04T08:56:37.083961+0000","last_undegraded":"2024-03-04T08:56:37.083961+0000","last_fullsized":"2024-03-04T08:56:37.083961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:40:38.217900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084330+0000","last_change":"2024-03-04T08:56:30.647282+0000","last_active":"2024-03-04T08:56:37.084330+0000","last_peered":"2024-03-04T08:56:37.084330+0000","last_clean":"2024-03-04T08:56:37.084330+0000","last_became_active":"2024-03-04T08:56:30.646916+0000","last_became_peered":"2024-03-04T08:56:30.646916+0000","last_unstale":"2024-03-04T08:56:37.084330+0000","last_undegraded":"2024-03-04T08:56:37.084330+0000","last_fullsized":"2024-03-04T08:56:37.084330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:36:11.714223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":7,"num_read_kb":8,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":13920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318291,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":486056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579928,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1080460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743454,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1007616,"data_stored":486638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005095,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":486660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299433,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1601536,"data_stored":1077006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462959,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":486056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":489898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":486146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":489932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149760,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":486160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888123,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":486078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724598,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28592,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716560,"statfs":{"total":95995035648,"available":95965757440,"internally_reserved":0,"allocated":1609728,"data_stored":1076606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T08:56:58.434 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T08:56:58.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.573+0000 7fb8a1f93640 1 -- 172.21.15.71:0/839669482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb89c1020a0 msgr2=0x7fb89c108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:58.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.573+0000 7fb8a1f93640 1 --2- 172.21.15.71:0/839669482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb89c1020a0 0x7fb89c108930 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb894009ca0 tx=0x7fb89402c100 comp rx=0 tx=0).stop 2024-03-04T08:56:58.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.573+0000 7fb8a1f93640 1 -- 172.21.15.71:0/839669482 shutdown_connections 2024-03-04T08:56:58.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.573+0000 7fb8a1f93640 1 --2- 172.21.15.71:0/839669482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb89c1020a0 0x7fb89c108930 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.573+0000 7fb8a1f93640 1 --2- 172.21.15.71:0/839669482 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb89c1016e0 0x7fb89c101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.573+0000 7fb8a1f93640 1 --2- 172.21.15.71:0/839669482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb89c1004f0 0x7fb89c100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.573+0000 7fb8a1f93640 1 -- 172.21.15.71:0/839669482 >> 172.21.15.71:0/839669482 conn(0x7fb89c0fba60 msgr2=0x7fb89c0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:58.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.573+0000 7fb8a1f93640 1 -- 172.21.15.71:0/839669482 shutdown_connections 2024-03-04T08:56:58.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.573+0000 7fb8a1f93640 1 -- 172.21.15.71:0/839669482 wait complete. 2024-03-04T08:56:58.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.573+0000 7fb8a1f93640 1 Processor -- start 2024-03-04T08:56:58.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.573+0000 7fb8a1f93640 1 -- start start 2024-03-04T08:56:58.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.574+0000 7fb8a1f93640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb89c1004f0 0x7fb89c0692a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:58.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.574+0000 7fb8a1f93640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb89c1016e0 0x7fb89c19c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:58.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.574+0000 7fb8a1f93640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb89c1020a0 0x7fb89c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:58.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.574+0000 7fb8a1f93640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c10c180 con 0x7fb89c1016e0 2024-03-04T08:56:58.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.574+0000 7fb8a1f93640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c10c000 con 0x7fb89c1020a0 2024-03-04T08:56:58.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.574+0000 7fb8a1f93640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c10c300 con 0x7fb89c1004f0 2024-03-04T08:56:58.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.574+0000 7fb892ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb89c1016e0 0x7fb89c19c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:58.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.574+0000 7fb892ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb89c1016e0 0x7fb89c19c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52338/0 (socket says 172.21.15.71:52338) 2024-03-04T08:56:58.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.574+0000 7fb892ffd640 1 -- 172.21.15.71:0/1419653050 learned_addr learned my addr 172.21.15.71:0/1419653050 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:56:58.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.574+0000 7fb89bfff640 1 --2- 172.21.15.71:0/1419653050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb89c1020a0 0x7fb89c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:58.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.574+0000 7fb89b7fe640 1 --2- 172.21.15.71:0/1419653050 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb89c1004f0 0x7fb89c0692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:58.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.574+0000 7fb892ffd640 1 -- 172.21.15.71:0/1419653050 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb89c1004f0 msgr2=0x7fb89c0692a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:58.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.574+0000 7fb892ffd640 1 --2- 172.21.15.71:0/1419653050 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb89c1004f0 0x7fb89c0692a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.574+0000 7fb892ffd640 1 -- 172.21.15.71:0/1419653050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb89c1020a0 msgr2=0x7fb89c19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:58.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.574+0000 7fb892ffd640 1 --2- 172.21.15.71:0/1419653050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb89c1020a0 0x7fb89c19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.574+0000 7fb892ffd640 1 -- 172.21.15.71:0/1419653050 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89c19f480 con 0x7fb89c1016e0 2024-03-04T08:56:58.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.574+0000 7fb892ffd640 1 --2- 172.21.15.71:0/1419653050 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb89c1016e0 0x7fb89c19c550 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb88400c920 tx=0x7fb88400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:58.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.574+0000 7fb8997fa640 1 -- 172.21.15.71:0/1419653050 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb884007c70 con 0x7fb89c1016e0 2024-03-04T08:56:58.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.574+0000 7fb8a1f93640 1 -- 172.21.15.71:0/1419653050 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb89c19f710 con 0x7fb89c1016e0 2024-03-04T08:56:58.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.574+0000 7fb8997fa640 1 -- 172.21.15.71:0/1419653050 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb884007e10 con 0x7fb89c1016e0 2024-03-04T08:56:58.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.574+0000 7fb8997fa640 1 -- 172.21.15.71:0/1419653050 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb884018890 con 0x7fb89c1016e0 2024-03-04T08:56:58.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.574+0000 7fb8a1f93640 1 -- 172.21.15.71:0/1419653050 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb89c19fcb0 con 0x7fb89c1016e0 2024-03-04T08:56:58.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.575+0000 7fb8a1f93640 1 -- 172.21.15.71:0/1419653050 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb89c1077b0 con 0x7fb89c1016e0 2024-03-04T08:56:58.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.576+0000 7fb8997fa640 1 -- 172.21.15.71:0/1419653050 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fb884018a70 con 0x7fb89c1016e0 2024-03-04T08:56:58.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.577+0000 7fb8997fa640 1 --2- 172.21.15.71:0/1419653050 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb8880a1e10 0x7fb8880a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:56:58.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.577+0000 7fb89b7fe640 1 --2- 172.21.15.71:0/1419653050 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb8880a1e10 0x7fb8880a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:56:58.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.577+0000 7fb8997fa640 1 -- 172.21.15.71:0/1419653050 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb8840ce220 con 0x7fb89c1016e0 2024-03-04T08:56:58.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.577+0000 7fb89b7fe640 1 --2- 172.21.15.71:0/1419653050 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb8880a1e10 0x7fb8880a42d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0095f0 tx=0x7fb88c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:56:58.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.581+0000 7fb8997fa640 1 -- 172.21.15.71:0/1419653050 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb884014030 con 0x7fb89c1016e0 2024-03-04T08:56:58.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.755+0000 7fb8a1f93640 1 -- 172.21.15.71:0/1419653050 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb89c061a10 con 0x7fb8880a1e10 2024-03-04T08:56:58.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.765+0000 7fb8997fa640 1 -- 172.21.15.71:0/1419653050 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340234 (secure 0 0 0) 0x7fb89c061a10 con 0x7fb8880a1e10 2024-03-04T08:56:58.765 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:56:58.767 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-03-04T08:56:58.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.768+0000 7fb8a1f93640 1 -- 172.21.15.71:0/1419653050 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb8880a1e10 msgr2=0x7fb8880a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:58.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.768+0000 7fb8a1f93640 1 --2- 172.21.15.71:0/1419653050 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb8880a1e10 0x7fb8880a42d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0095f0 tx=0x7fb88c005f90 comp rx=0 tx=0).stop 2024-03-04T08:56:58.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.768+0000 7fb8a1f93640 1 -- 172.21.15.71:0/1419653050 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb89c1016e0 msgr2=0x7fb89c19c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:56:58.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.768+0000 7fb8a1f93640 1 --2- 172.21.15.71:0/1419653050 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb89c1016e0 0x7fb89c19c550 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb88400c920 tx=0x7fb88400cde0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.768+0000 7fb8a1f93640 1 -- 172.21.15.71:0/1419653050 shutdown_connections 2024-03-04T08:56:58.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.768+0000 7fb8a1f93640 1 --2- 172.21.15.71:0/1419653050 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb8880a1e10 0x7fb8880a42d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.768+0000 7fb8a1f93640 1 --2- 172.21.15.71:0/1419653050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb89c1020a0 0x7fb89c19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.768+0000 7fb8a1f93640 1 --2- 172.21.15.71:0/1419653050 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb89c1016e0 0x7fb89c19c550 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.768+0000 7fb8a1f93640 1 --2- 172.21.15.71:0/1419653050 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb89c1004f0 0x7fb89c0692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:56:58.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.768+0000 7fb8a1f93640 1 -- 172.21.15.71:0/1419653050 >> 172.21.15.71:0/1419653050 conn(0x7fb89c0fba60 msgr2=0x7fb89c104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:56:58.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.768+0000 7fb8a1f93640 1 -- 172.21.15.71:0/1419653050 shutdown_connections 2024-03-04T08:56:58.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:56:58.768+0000 7fb8a1f93640 1 -- 172.21.15.71:0/1419653050 wait complete. 2024-03-04T08:56:58.775 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-03-04T08:56:58.694945+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595125,"num_objects":25,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":144,"num_read_kb":128,"num_write":231,"num_write_kb":2958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":181,"ondisk_log_size":181,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358660,"kb_used_data":13620,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583164,"statfs":{"total":1151940427776,"available":1151573159936,"internally_reserved":0,"allocated":13946880,"data_stored":7641091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":258,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":9,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003316"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084515+0000","last_change":"2024-03-04T08:56:30.650344+0000","last_active":"2024-03-04T08:56:37.084515+0000","last_peered":"2024-03-04T08:56:37.084515+0000","last_clean":"2024-03-04T08:56:37.084515+0000","last_became_active":"2024-03-04T08:56:30.649842+0000","last_became_peered":"2024-03-04T08:56:30.649842+0000","last_unstale":"2024-03-04T08:56:37.084515+0000","last_undegraded":"2024-03-04T08:56:37.084515+0000","last_fullsized":"2024-03-04T08:56:37.084515+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:04:57.162321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084709+0000","last_change":"2024-03-04T08:56:30.651616+0000","last_active":"2024-03-04T08:56:37.084709+0000","last_peered":"2024-03-04T08:56:37.084709+0000","last_clean":"2024-03-04T08:56:37.084709+0000","last_became_active":"2024-03-04T08:56:30.651283+0000","last_became_peered":"2024-03-04T08:56:30.651283+0000","last_unstale":"2024-03-04T08:56:37.084709+0000","last_undegraded":"2024-03-04T08:56:37.084709+0000","last_fullsized":"2024-03-04T08:56:37.084709+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:18:48.018257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084789+0000","last_change":"2024-03-04T08:56:30.657459+0000","last_active":"2024-03-04T08:56:37.084789+0000","last_peered":"2024-03-04T08:56:37.084789+0000","last_clean":"2024-03-04T08:56:37.084789+0000","last_became_active":"2024-03-04T08:56:30.656993+0000","last_became_peered":"2024-03-04T08:56:30.656993+0000","last_unstale":"2024-03-04T08:56:37.084789+0000","last_undegraded":"2024-03-04T08:56:37.084789+0000","last_fullsized":"2024-03-04T08:56:37.084789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:07:34.898068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083746+0000","last_change":"2024-03-04T08:56:30.644543+0000","last_active":"2024-03-04T08:56:37.083746+0000","last_peered":"2024-03-04T08:56:37.083746+0000","last_clean":"2024-03-04T08:56:37.083746+0000","last_became_active":"2024-03-04T08:56:30.644251+0000","last_became_peered":"2024-03-04T08:56:30.644251+0000","last_unstale":"2024-03-04T08:56:37.083746+0000","last_undegraded":"2024-03-04T08:56:37.083746+0000","last_fullsized":"2024-03-04T08:56:37.083746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:21:40.260400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.582670+0000","last_change":"2024-03-04T08:56:30.654640+0000","last_active":"2024-03-04T08:56:36.582670+0000","last_peered":"2024-03-04T08:56:36.582670+0000","last_clean":"2024-03-04T08:56:36.582670+0000","last_became_active":"2024-03-04T08:56:30.654312+0000","last_became_peered":"2024-03-04T08:56:30.654312+0000","last_unstale":"2024-03-04T08:56:36.582670+0000","last_undegraded":"2024-03-04T08:56:36.582670+0000","last_fullsized":"2024-03-04T08:56:36.582670+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:09:43.930713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.374337+0000","last_change":"2024-03-04T08:56:30.647445+0000","last_active":"2024-03-04T08:56:36.374337+0000","last_peered":"2024-03-04T08:56:36.374337+0000","last_clean":"2024-03-04T08:56:36.374337+0000","last_became_active":"2024-03-04T08:56:30.647130+0000","last_became_peered":"2024-03-04T08:56:30.647130+0000","last_unstale":"2024-03-04T08:56:36.374337+0000","last_undegraded":"2024-03-04T08:56:36.374337+0000","last_fullsized":"2024-03-04T08:56:36.374337+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:40:18.906604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671553+0000","last_change":"2024-03-04T08:56:30.645776+0000","last_active":"2024-03-04T08:56:35.671553+0000","last_peered":"2024-03-04T08:56:35.671553+0000","last_clean":"2024-03-04T08:56:35.671553+0000","last_became_active":"2024-03-04T08:56:30.645107+0000","last_became_peered":"2024-03-04T08:56:30.645107+0000","last_unstale":"2024-03-04T08:56:35.671553+0000","last_undegraded":"2024-03-04T08:56:35.671553+0000","last_fullsized":"2024-03-04T08:56:35.671553+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:44:35.230491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.768312+0000","last_change":"2024-03-04T08:56:30.646424+0000","last_active":"2024-03-04T08:56:35.768312+0000","last_peered":"2024-03-04T08:56:35.768312+0000","last_clean":"2024-03-04T08:56:35.768312+0000","last_became_active":"2024-03-04T08:56:30.646115+0000","last_became_peered":"2024-03-04T08:56:30.646115+0000","last_unstale":"2024-03-04T08:56:35.768312+0000","last_undegraded":"2024-03-04T08:56:35.768312+0000","last_fullsized":"2024-03-04T08:56:35.768312+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:42:36.201984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671159+0000","last_change":"2024-03-04T08:56:30.655027+0000","last_active":"2024-03-04T08:56:35.671159+0000","last_peered":"2024-03-04T08:56:35.671159+0000","last_clean":"2024-03-04T08:56:35.671159+0000","last_became_active":"2024-03-04T08:56:30.654766+0000","last_became_peered":"2024-03-04T08:56:30.654766+0000","last_unstale":"2024-03-04T08:56:35.671159+0000","last_undegraded":"2024-03-04T08:56:35.671159+0000","last_fullsized":"2024-03-04T08:56:35.671159+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:00:34.168740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.374661+0000","last_change":"2024-03-04T08:56:30.648268+0000","last_active":"2024-03-04T08:56:36.374661+0000","last_peered":"2024-03-04T08:56:36.374661+0000","last_clean":"2024-03-04T08:56:36.374661+0000","last_became_active":"2024-03-04T08:56:30.647905+0000","last_became_peered":"2024-03-04T08:56:30.647905+0000","last_unstale":"2024-03-04T08:56:36.374661+0000","last_undegraded":"2024-03-04T08:56:36.374661+0000","last_fullsized":"2024-03-04T08:56:36.374661+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:07:32.387510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.672115+0000","last_change":"2024-03-04T08:56:30.648061+0000","last_active":"2024-03-04T08:56:35.672115+0000","last_peered":"2024-03-04T08:56:35.672115+0000","last_clean":"2024-03-04T08:56:35.672115+0000","last_became_active":"2024-03-04T08:56:30.647740+0000","last_became_peered":"2024-03-04T08:56:30.647740+0000","last_unstale":"2024-03-04T08:56:35.672115+0000","last_undegraded":"2024-03-04T08:56:35.672115+0000","last_fullsized":"2024-03-04T08:56:35.672115+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:35:23.065902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083837+0000","last_change":"2024-03-04T08:56:30.645802+0000","last_active":"2024-03-04T08:56:37.083837+0000","last_peered":"2024-03-04T08:56:37.083837+0000","last_clean":"2024-03-04T08:56:37.083837+0000","last_became_active":"2024-03-04T08:56:30.645474+0000","last_became_peered":"2024-03-04T08:56:30.645474+0000","last_unstale":"2024-03-04T08:56:37.083837+0000","last_undegraded":"2024-03-04T08:56:37.083837+0000","last_fullsized":"2024-03-04T08:56:37.083837+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:35:40.207215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083876+0000","last_change":"2024-03-04T08:56:30.656149+0000","last_active":"2024-03-04T08:56:37.083876+0000","last_peered":"2024-03-04T08:56:37.083876+0000","last_clean":"2024-03-04T08:56:37.083876+0000","last_became_active":"2024-03-04T08:56:30.655493+0000","last_became_peered":"2024-03-04T08:56:30.655493+0000","last_unstale":"2024-03-04T08:56:37.083876+0000","last_undegraded":"2024-03-04T08:56:37.083876+0000","last_fullsized":"2024-03-04T08:56:37.083876+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:23:28.320328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084063+0000","last_change":"2024-03-04T08:56:30.658309+0000","last_active":"2024-03-04T08:56:37.084063+0000","last_peered":"2024-03-04T08:56:37.084063+0000","last_clean":"2024-03-04T08:56:37.084063+0000","last_became_active":"2024-03-04T08:56:30.656773+0000","last_became_peered":"2024-03-04T08:56:30.656773+0000","last_unstale":"2024-03-04T08:56:37.084063+0000","last_undegraded":"2024-03-04T08:56:37.084063+0000","last_fullsized":"2024-03-04T08:56:37.084063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:26:57.021442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084019+0000","last_change":"2024-03-04T08:56:30.650336+0000","last_active":"2024-03-04T08:56:37.084019+0000","last_peered":"2024-03-04T08:56:37.084019+0000","last_clean":"2024-03-04T08:56:37.084019+0000","last_became_active":"2024-03-04T08:56:30.648377+0000","last_became_peered":"2024-03-04T08:56:30.648377+0000","last_unstale":"2024-03-04T08:56:37.084019+0000","last_undegraded":"2024-03-04T08:56:37.084019+0000","last_fullsized":"2024-03-04T08:56:37.084019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:56:41.358895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.670486+0000","last_change":"2024-03-04T08:56:30.653327+0000","last_active":"2024-03-04T08:56:35.670486+0000","last_peered":"2024-03-04T08:56:35.670486+0000","last_clean":"2024-03-04T08:56:35.670486+0000","last_became_active":"2024-03-04T08:56:30.652523+0000","last_became_peered":"2024-03-04T08:56:30.652523+0000","last_unstale":"2024-03-04T08:56:35.670486+0000","last_undegraded":"2024-03-04T08:56:35.670486+0000","last_fullsized":"2024-03-04T08:56:35.670486+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:18:41.726921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084247+0000","last_change":"2024-03-04T08:56:30.647210+0000","last_active":"2024-03-04T08:56:37.084247+0000","last_peered":"2024-03-04T08:56:37.084247+0000","last_clean":"2024-03-04T08:56:37.084247+0000","last_became_active":"2024-03-04T08:56:30.646897+0000","last_became_peered":"2024-03-04T08:56:30.646897+0000","last_unstale":"2024-03-04T08:56:37.084247+0000","last_undegraded":"2024-03-04T08:56:37.084247+0000","last_fullsized":"2024-03-04T08:56:37.084247+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:40:35.021061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.668081+0000","last_change":"2024-03-04T08:56:30.648751+0000","last_active":"2024-03-04T08:56:35.668081+0000","last_peered":"2024-03-04T08:56:35.668081+0000","last_clean":"2024-03-04T08:56:35.668081+0000","last_became_active":"2024-03-04T08:56:30.648457+0000","last_became_peered":"2024-03-04T08:56:30.648457+0000","last_unstale":"2024-03-04T08:56:35.668081+0000","last_undegraded":"2024-03-04T08:56:35.668081+0000","last_fullsized":"2024-03-04T08:56:35.668081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:20:20.435769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.844328+0000","last_change":"2024-03-04T08:56:30.644523+0000","last_active":"2024-03-04T08:56:35.844328+0000","last_peered":"2024-03-04T08:56:35.844328+0000","last_clean":"2024-03-04T08:56:35.844328+0000","last_became_active":"2024-03-04T08:56:30.644183+0000","last_became_peered":"2024-03-04T08:56:30.644183+0000","last_unstale":"2024-03-04T08:56:35.844328+0000","last_undegraded":"2024-03-04T08:56:35.844328+0000","last_fullsized":"2024-03-04T08:56:35.844328+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:58:16.065571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.843802+0000","last_change":"2024-03-04T08:56:34.681350+0000","last_active":"2024-03-04T08:56:35.843802+0000","last_peered":"2024-03-04T08:56:35.843802+0000","last_clean":"2024-03-04T08:56:35.843802+0000","last_became_active":"2024-03-04T08:56:34.680940+0000","last_became_peered":"2024-03-04T08:56:34.680940+0000","last_unstale":"2024-03-04T08:56:35.843802+0000","last_undegraded":"2024-03-04T08:56:35.843802+0000","last_fullsized":"2024-03-04T08:56:35.843802+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:41:00.690541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.668309+0000","last_change":"2024-03-04T08:56:29.989827+0000","last_active":"2024-03-04T08:56:35.668309+0000","last_peered":"2024-03-04T08:56:35.668309+0000","last_clean":"2024-03-04T08:56:35.668309+0000","last_became_active":"2024-03-04T08:56:29.989509+0000","last_became_peered":"2024-03-04T08:56:29.989509+0000","last_unstale":"2024-03-04T08:56:35.668309+0000","last_undegraded":"2024-03-04T08:56:35.668309+0000","last_fullsized":"2024-03-04T08:56:35.668309+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:27:37.771477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.768541+0000","last_change":"2024-03-04T08:56:30.638354+0000","last_active":"2024-03-04T08:56:35.768541+0000","last_peered":"2024-03-04T08:56:35.768541+0000","last_clean":"2024-03-04T08:56:35.768541+0000","last_became_active":"2024-03-04T08:56:30.638058+0000","last_became_peered":"2024-03-04T08:56:30.638058+0000","last_unstale":"2024-03-04T08:56:35.768541+0000","last_undegraded":"2024-03-04T08:56:35.768541+0000","last_fullsized":"2024-03-04T08:56:35.768541+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:43:51.920411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083731+0000","last_change":"2024-03-04T08:56:34.680206+0000","last_active":"2024-03-04T08:56:37.083731+0000","last_peered":"2024-03-04T08:56:37.083731+0000","last_clean":"2024-03-04T08:56:37.083731+0000","last_became_active":"2024-03-04T08:56:34.679821+0000","last_became_peered":"2024-03-04T08:56:34.679821+0000","last_unstale":"2024-03-04T08:56:37.083731+0000","last_undegraded":"2024-03-04T08:56:37.083731+0000","last_fullsized":"2024-03-04T08:56:37.083731+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:18:39.945962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.769029+0000","last_change":"2024-03-04T08:56:29.994017+0000","last_active":"2024-03-04T08:56:35.769029+0000","last_peered":"2024-03-04T08:56:35.769029+0000","last_clean":"2024-03-04T08:56:35.769029+0000","last_became_active":"2024-03-04T08:56:29.993687+0000","last_became_peered":"2024-03-04T08:56:29.993687+0000","last_unstale":"2024-03-04T08:56:35.769029+0000","last_undegraded":"2024-03-04T08:56:35.769029+0000","last_fullsized":"2024-03-04T08:56:35.769029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:32:51.542130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084890+0000","last_change":"2024-03-04T08:56:30.654446+0000","last_active":"2024-03-04T08:56:37.084890+0000","last_peered":"2024-03-04T08:56:37.084890+0000","last_clean":"2024-03-04T08:56:37.084890+0000","last_became_active":"2024-03-04T08:56:30.653920+0000","last_became_peered":"2024-03-04T08:56:30.653920+0000","last_unstale":"2024-03-04T08:56:37.084890+0000","last_undegraded":"2024-03-04T08:56:37.084890+0000","last_fullsized":"2024-03-04T08:56:37.084890+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:48:43.177893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083601+0000","last_change":"2024-03-04T08:56:34.680888+0000","last_active":"2024-03-04T08:56:37.083601+0000","last_peered":"2024-03-04T08:56:37.083601+0000","last_clean":"2024-03-04T08:56:37.083601+0000","last_became_active":"2024-03-04T08:56:34.679950+0000","last_became_peered":"2024-03-04T08:56:34.679950+0000","last_unstale":"2024-03-04T08:56:37.083601+0000","last_undegraded":"2024-03-04T08:56:37.083601+0000","last_fullsized":"2024-03-04T08:56:37.083601+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:08:26.660124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.583182+0000","last_change":"2024-03-04T08:56:29.986476+0000","last_active":"2024-03-04T08:56:36.583182+0000","last_peered":"2024-03-04T08:56:36.583182+0000","last_clean":"2024-03-04T08:56:36.583182+0000","last_became_active":"2024-03-04T08:56:29.986106+0000","last_became_peered":"2024-03-04T08:56:29.986106+0000","last_unstale":"2024-03-04T08:56:36.583182+0000","last_undegraded":"2024-03-04T08:56:36.583182+0000","last_fullsized":"2024-03-04T08:56:36.583182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:45:15.775106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.670715+0000","last_change":"2024-03-04T08:56:30.645841+0000","last_active":"2024-03-04T08:56:35.670715+0000","last_peered":"2024-03-04T08:56:35.670715+0000","last_clean":"2024-03-04T08:56:35.670715+0000","last_became_active":"2024-03-04T08:56:30.645459+0000","last_became_peered":"2024-03-04T08:56:30.645459+0000","last_unstale":"2024-03-04T08:56:35.670715+0000","last_undegraded":"2024-03-04T08:56:35.670715+0000","last_fullsized":"2024-03-04T08:56:35.670715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:37:51.204517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083635+0000","last_change":"2024-03-04T08:56:34.674562+0000","last_active":"2024-03-04T08:56:37.083635+0000","last_peered":"2024-03-04T08:56:37.083635+0000","last_clean":"2024-03-04T08:56:37.083635+0000","last_became_active":"2024-03-04T08:56:34.674239+0000","last_became_peered":"2024-03-04T08:56:34.674239+0000","last_unstale":"2024-03-04T08:56:37.083635+0000","last_undegraded":"2024-03-04T08:56:37.083635+0000","last_fullsized":"2024-03-04T08:56:37.083635+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:47:34.819487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.583010+0000","last_change":"2024-03-04T08:56:29.984025+0000","last_active":"2024-03-04T08:56:36.583010+0000","last_peered":"2024-03-04T08:56:36.583010+0000","last_clean":"2024-03-04T08:56:36.583010+0000","last_became_active":"2024-03-04T08:56:29.983676+0000","last_became_peered":"2024-03-04T08:56:29.983676+0000","last_unstale":"2024-03-04T08:56:36.583010+0000","last_undegraded":"2024-03-04T08:56:36.583010+0000","last_fullsized":"2024-03-04T08:56:36.583010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:48:24.920513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551593+0000","last_change":"2024-03-04T08:56:30.649301+0000","last_active":"2024-03-04T08:56:36.551593+0000","last_peered":"2024-03-04T08:56:36.551593+0000","last_clean":"2024-03-04T08:56:36.551593+0000","last_became_active":"2024-03-04T08:56:30.643987+0000","last_became_peered":"2024-03-04T08:56:30.643987+0000","last_unstale":"2024-03-04T08:56:36.551593+0000","last_undegraded":"2024-03-04T08:56:36.551593+0000","last_fullsized":"2024-03-04T08:56:36.551593+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:05:21.171953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.844201+0000","last_change":"2024-03-04T08:56:34.672893+0000","last_active":"2024-03-04T08:56:35.844201+0000","last_peered":"2024-03-04T08:56:35.844201+0000","last_clean":"2024-03-04T08:56:35.844201+0000","last_became_active":"2024-03-04T08:56:34.672364+0000","last_became_peered":"2024-03-04T08:56:34.672364+0000","last_unstale":"2024-03-04T08:56:35.844201+0000","last_undegraded":"2024-03-04T08:56:35.844201+0000","last_fullsized":"2024-03-04T08:56:35.844201+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:33:29.287772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'1","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551306+0000","last_change":"2024-03-04T08:56:29.986822+0000","last_active":"2024-03-04T08:56:36.551306+0000","last_peered":"2024-03-04T08:56:36.551306+0000","last_clean":"2024-03-04T08:56:36.551306+0000","last_became_active":"2024-03-04T08:56:29.986498+0000","last_became_peered":"2024-03-04T08:56:29.986498+0000","last_unstale":"2024-03-04T08:56:36.551306+0000","last_undegraded":"2024-03-04T08:56:36.551306+0000","last_fullsized":"2024-03-04T08:56:36.551306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:15:01.585997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.582694+0000","last_change":"2024-03-04T08:56:30.646164+0000","last_active":"2024-03-04T08:56:36.582694+0000","last_peered":"2024-03-04T08:56:36.582694+0000","last_clean":"2024-03-04T08:56:36.582694+0000","last_became_active":"2024-03-04T08:56:30.645843+0000","last_became_peered":"2024-03-04T08:56:30.645843+0000","last_unstale":"2024-03-04T08:56:36.582694+0000","last_undegraded":"2024-03-04T08:56:36.582694+0000","last_fullsized":"2024-03-04T08:56:36.582694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:11:48.082226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.843983+0000","last_change":"2024-03-04T08:56:34.680387+0000","last_active":"2024-03-04T08:56:35.843983+0000","last_peered":"2024-03-04T08:56:35.843983+0000","last_clean":"2024-03-04T08:56:35.843983+0000","last_became_active":"2024-03-04T08:56:34.680005+0000","last_became_peered":"2024-03-04T08:56:34.680005+0000","last_unstale":"2024-03-04T08:56:35.843983+0000","last_undegraded":"2024-03-04T08:56:35.843983+0000","last_fullsized":"2024-03-04T08:56:35.843983+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:04:40.403187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.374276+0000","last_change":"2024-03-04T08:56:29.984047+0000","last_active":"2024-03-04T08:56:36.374276+0000","last_peered":"2024-03-04T08:56:36.374276+0000","last_clean":"2024-03-04T08:56:36.374276+0000","last_became_active":"2024-03-04T08:56:29.983709+0000","last_became_peered":"2024-03-04T08:56:29.983709+0000","last_unstale":"2024-03-04T08:56:36.374276+0000","last_undegraded":"2024-03-04T08:56:36.374276+0000","last_fullsized":"2024-03-04T08:56:36.374276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:44:25.015970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551450+0000","last_change":"2024-03-04T08:56:30.649321+0000","last_active":"2024-03-04T08:56:36.551450+0000","last_peered":"2024-03-04T08:56:36.551450+0000","last_clean":"2024-03-04T08:56:36.551450+0000","last_became_active":"2024-03-04T08:56:30.648705+0000","last_became_peered":"2024-03-04T08:56:30.648705+0000","last_unstale":"2024-03-04T08:56:36.551450+0000","last_undegraded":"2024-03-04T08:56:36.551450+0000","last_fullsized":"2024-03-04T08:56:36.551450+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:26:12.549471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084375+0000","last_change":"2024-03-04T08:56:34.682560+0000","last_active":"2024-03-04T08:56:37.084375+0000","last_peered":"2024-03-04T08:56:37.084375+0000","last_clean":"2024-03-04T08:56:37.084375+0000","last_became_active":"2024-03-04T08:56:34.682156+0000","last_became_peered":"2024-03-04T08:56:34.682156+0000","last_unstale":"2024-03-04T08:56:37.084375+0000","last_undegraded":"2024-03-04T08:56:37.084375+0000","last_fullsized":"2024-03-04T08:56:37.084375+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:57:49.132436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083988+0000","last_change":"2024-03-04T08:56:29.984947+0000","last_active":"2024-03-04T08:56:37.083988+0000","last_peered":"2024-03-04T08:56:37.083988+0000","last_clean":"2024-03-04T08:56:37.083988+0000","last_became_active":"2024-03-04T08:56:29.984575+0000","last_became_peered":"2024-03-04T08:56:29.984575+0000","last_unstale":"2024-03-04T08:56:37.083988+0000","last_undegraded":"2024-03-04T08:56:37.083988+0000","last_fullsized":"2024-03-04T08:56:37.083988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:01:16.698417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.768447+0000","last_change":"2024-03-04T08:56:30.649000+0000","last_active":"2024-03-04T08:56:35.768447+0000","last_peered":"2024-03-04T08:56:35.768447+0000","last_clean":"2024-03-04T08:56:35.768447+0000","last_became_active":"2024-03-04T08:56:30.648700+0000","last_became_peered":"2024-03-04T08:56:30.648700+0000","last_unstale":"2024-03-04T08:56:35.768447+0000","last_undegraded":"2024-03-04T08:56:35.768447+0000","last_fullsized":"2024-03-04T08:56:35.768447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:46:09.384040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084654+0000","last_change":"2024-03-04T08:56:34.671042+0000","last_active":"2024-03-04T08:56:37.084654+0000","last_peered":"2024-03-04T08:56:37.084654+0000","last_clean":"2024-03-04T08:56:37.084654+0000","last_became_active":"2024-03-04T08:56:34.670683+0000","last_became_peered":"2024-03-04T08:56:34.670683+0000","last_unstale":"2024-03-04T08:56:37.084654+0000","last_undegraded":"2024-03-04T08:56:37.084654+0000","last_fullsized":"2024-03-04T08:56:37.084654+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:13:09.166986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551002+0000","last_change":"2024-03-04T08:56:29.983359+0000","last_active":"2024-03-04T08:56:36.551002+0000","last_peered":"2024-03-04T08:56:36.551002+0000","last_clean":"2024-03-04T08:56:36.551002+0000","last_became_active":"2024-03-04T08:56:29.983009+0000","last_became_peered":"2024-03-04T08:56:29.983009+0000","last_unstale":"2024-03-04T08:56:36.551002+0000","last_undegraded":"2024-03-04T08:56:36.551002+0000","last_fullsized":"2024-03-04T08:56:36.551002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:03:16.549109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.667758+0000","last_change":"2024-03-04T08:56:30.649612+0000","last_active":"2024-03-04T08:56:35.667758+0000","last_peered":"2024-03-04T08:56:35.667758+0000","last_clean":"2024-03-04T08:56:35.667758+0000","last_became_active":"2024-03-04T08:56:30.649299+0000","last_became_peered":"2024-03-04T08:56:30.649299+0000","last_unstale":"2024-03-04T08:56:35.667758+0000","last_undegraded":"2024-03-04T08:56:35.667758+0000","last_fullsized":"2024-03-04T08:56:35.667758+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:57:50.571696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.768590+0000","last_change":"2024-03-04T08:56:34.677974+0000","last_active":"2024-03-04T08:56:35.768590+0000","last_peered":"2024-03-04T08:56:35.768590+0000","last_clean":"2024-03-04T08:56:35.768590+0000","last_became_active":"2024-03-04T08:56:34.676628+0000","last_became_peered":"2024-03-04T08:56:34.676628+0000","last_unstale":"2024-03-04T08:56:35.768590+0000","last_undegraded":"2024-03-04T08:56:35.768590+0000","last_fullsized":"2024-03-04T08:56:35.768590+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:18:40.301336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.374580+0000","last_change":"2024-03-04T08:56:29.976760+0000","last_active":"2024-03-04T08:56:36.374580+0000","last_peered":"2024-03-04T08:56:36.374580+0000","last_clean":"2024-03-04T08:56:36.374580+0000","last_became_active":"2024-03-04T08:56:29.976446+0000","last_became_peered":"2024-03-04T08:56:29.976446+0000","last_unstale":"2024-03-04T08:56:36.374580+0000","last_undegraded":"2024-03-04T08:56:36.374580+0000","last_fullsized":"2024-03-04T08:56:36.374580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:46:35.038622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671108+0000","last_change":"2024-03-04T08:56:30.644326+0000","last_active":"2024-03-04T08:56:35.671108+0000","last_peered":"2024-03-04T08:56:35.671108+0000","last_clean":"2024-03-04T08:56:35.671108+0000","last_became_active":"2024-03-04T08:56:30.644017+0000","last_became_peered":"2024-03-04T08:56:30.644017+0000","last_unstale":"2024-03-04T08:56:35.671108+0000","last_undegraded":"2024-03-04T08:56:35.671108+0000","last_fullsized":"2024-03-04T08:56:35.671108+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:01:40.843915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.667772+0000","last_change":"2024-03-04T08:56:34.675757+0000","last_active":"2024-03-04T08:56:35.667772+0000","last_peered":"2024-03-04T08:56:35.667772+0000","last_clean":"2024-03-04T08:56:35.667772+0000","last_became_active":"2024-03-04T08:56:34.675375+0000","last_became_peered":"2024-03-04T08:56:34.675375+0000","last_unstale":"2024-03-04T08:56:35.667772+0000","last_undegraded":"2024-03-04T08:56:35.667772+0000","last_fullsized":"2024-03-04T08:56:35.667772+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:16:12.762024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671919+0000","last_change":"2024-03-04T08:56:29.987614+0000","last_active":"2024-03-04T08:56:35.671919+0000","last_peered":"2024-03-04T08:56:35.671919+0000","last_clean":"2024-03-04T08:56:35.671919+0000","last_became_active":"2024-03-04T08:56:29.986639+0000","last_became_peered":"2024-03-04T08:56:29.986639+0000","last_unstale":"2024-03-04T08:56:35.671919+0000","last_undegraded":"2024-03-04T08:56:35.671919+0000","last_fullsized":"2024-03-04T08:56:35.671919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:18:50.320069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084369+0000","last_change":"2024-03-04T08:56:30.655145+0000","last_active":"2024-03-04T08:56:37.084369+0000","last_peered":"2024-03-04T08:56:37.084369+0000","last_clean":"2024-03-04T08:56:37.084369+0000","last_became_active":"2024-03-04T08:56:30.654834+0000","last_became_peered":"2024-03-04T08:56:30.654834+0000","last_unstale":"2024-03-04T08:56:37.084369+0000","last_undegraded":"2024-03-04T08:56:37.084369+0000","last_fullsized":"2024-03-04T08:56:37.084369+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:45:19.567325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083773+0000","last_change":"2024-03-04T08:56:34.680530+0000","last_active":"2024-03-04T08:56:37.083773+0000","last_peered":"2024-03-04T08:56:37.083773+0000","last_clean":"2024-03-04T08:56:37.083773+0000","last_became_active":"2024-03-04T08:56:34.679860+0000","last_became_peered":"2024-03-04T08:56:34.679860+0000","last_unstale":"2024-03-04T08:56:37.083773+0000","last_undegraded":"2024-03-04T08:56:37.083773+0000","last_fullsized":"2024-03-04T08:56:37.083773+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:25:09.036145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'3","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551470+0000","last_change":"2024-03-04T08:56:29.982516+0000","last_active":"2024-03-04T08:56:36.551470+0000","last_peered":"2024-03-04T08:56:36.551470+0000","last_clean":"2024-03-04T08:56:36.551470+0000","last_became_active":"2024-03-04T08:56:29.982166+0000","last_became_peered":"2024-03-04T08:56:29.982166+0000","last_unstale":"2024-03-04T08:56:36.551470+0000","last_undegraded":"2024-03-04T08:56:36.551470+0000","last_fullsized":"2024-03-04T08:56:36.551470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:48:57.463584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.670904+0000","last_change":"2024-03-04T08:56:30.650030+0000","last_active":"2024-03-04T08:56:35.670904+0000","last_peered":"2024-03-04T08:56:35.670904+0000","last_clean":"2024-03-04T08:56:35.670904+0000","last_became_active":"2024-03-04T08:56:30.647826+0000","last_became_peered":"2024-03-04T08:56:30.647826+0000","last_unstale":"2024-03-04T08:56:35.670904+0000","last_undegraded":"2024-03-04T08:56:35.670904+0000","last_fullsized":"2024-03-04T08:56:35.670904+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:07:45.189935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.667654+0000","last_change":"2024-03-04T08:56:34.679189+0000","last_active":"2024-03-04T08:56:35.667654+0000","last_peered":"2024-03-04T08:56:35.667654+0000","last_clean":"2024-03-04T08:56:35.667654+0000","last_became_active":"2024-03-04T08:56:34.678744+0000","last_became_peered":"2024-03-04T08:56:34.678744+0000","last_unstale":"2024-03-04T08:56:35.667654+0000","last_undegraded":"2024-03-04T08:56:35.667654+0000","last_fullsized":"2024-03-04T08:56:35.667654+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:05:15.128703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'10","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:55.561704+0000","last_change":"2024-03-04T08:56:29.978908+0000","last_active":"2024-03-04T08:56:55.561704+0000","last_peered":"2024-03-04T08:56:55.561704+0000","last_clean":"2024-03-04T08:56:55.561704+0000","last_became_active":"2024-03-04T08:56:29.978604+0000","last_became_peered":"2024-03-04T08:56:29.978604+0000","last_unstale":"2024-03-04T08:56:55.561704+0000","last_undegraded":"2024-03-04T08:56:55.561704+0000","last_fullsized":"2024-03-04T08:56:55.561704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:03:45.683462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":7,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.582953+0000","last_change":"2024-03-04T08:56:30.647764+0000","last_active":"2024-03-04T08:56:36.582953+0000","last_peered":"2024-03-04T08:56:36.582953+0000","last_clean":"2024-03-04T08:56:36.582953+0000","last_became_active":"2024-03-04T08:56:30.647445+0000","last_became_peered":"2024-03-04T08:56:30.647445+0000","last_unstale":"2024-03-04T08:56:36.582953+0000","last_undegraded":"2024-03-04T08:56:36.582953+0000","last_fullsized":"2024-03-04T08:56:36.582953+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:13:25.135993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551915+0000","last_change":"2024-03-04T08:56:34.674674+0000","last_active":"2024-03-04T08:56:36.551915+0000","last_peered":"2024-03-04T08:56:36.551915+0000","last_clean":"2024-03-04T08:56:36.551915+0000","last_became_active":"2024-03-04T08:56:34.674138+0000","last_became_peered":"2024-03-04T08:56:34.674138+0000","last_unstale":"2024-03-04T08:56:36.551915+0000","last_undegraded":"2024-03-04T08:56:36.551915+0000","last_fullsized":"2024-03-04T08:56:36.551915+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:20:36.123111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.843998+0000","last_change":"2024-03-04T08:56:29.980997+0000","last_active":"2024-03-04T08:56:35.843998+0000","last_peered":"2024-03-04T08:56:35.843998+0000","last_clean":"2024-03-04T08:56:35.843998+0000","last_became_active":"2024-03-04T08:56:29.980651+0000","last_became_peered":"2024-03-04T08:56:29.980651+0000","last_unstale":"2024-03-04T08:56:35.843998+0000","last_undegraded":"2024-03-04T08:56:35.843998+0000","last_fullsized":"2024-03-04T08:56:35.843998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:23:33.434703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084161+0000","last_change":"2024-03-04T08:56:30.657229+0000","last_active":"2024-03-04T08:56:37.084161+0000","last_peered":"2024-03-04T08:56:37.084161+0000","last_clean":"2024-03-04T08:56:37.084161+0000","last_became_active":"2024-03-04T08:56:30.656877+0000","last_became_peered":"2024-03-04T08:56:30.656877+0000","last_unstale":"2024-03-04T08:56:37.084161+0000","last_undegraded":"2024-03-04T08:56:37.084161+0000","last_fullsized":"2024-03-04T08:56:37.084161+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:17:55.063418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084120+0000","last_change":"2024-03-04T08:56:34.680619+0000","last_active":"2024-03-04T08:56:37.084120+0000","last_peered":"2024-03-04T08:56:37.084120+0000","last_clean":"2024-03-04T08:56:37.084120+0000","last_became_active":"2024-03-04T08:56:34.680205+0000","last_became_peered":"2024-03-04T08:56:34.680205+0000","last_unstale":"2024-03-04T08:56:37.084120+0000","last_undegraded":"2024-03-04T08:56:37.084120+0000","last_fullsized":"2024-03-04T08:56:37.084120+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:42:14.650581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551374+0000","last_change":"2024-03-04T08:56:29.987759+0000","last_active":"2024-03-04T08:56:36.551374+0000","last_peered":"2024-03-04T08:56:36.551374+0000","last_clean":"2024-03-04T08:56:36.551374+0000","last_became_active":"2024-03-04T08:56:29.987458+0000","last_became_peered":"2024-03-04T08:56:29.987458+0000","last_unstale":"2024-03-04T08:56:36.551374+0000","last_undegraded":"2024-03-04T08:56:36.551374+0000","last_fullsized":"2024-03-04T08:56:36.551374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:36:18.878556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671741+0000","last_change":"2024-03-04T08:56:30.645935+0000","last_active":"2024-03-04T08:56:35.671741+0000","last_peered":"2024-03-04T08:56:35.671741+0000","last_clean":"2024-03-04T08:56:35.671741+0000","last_became_active":"2024-03-04T08:56:30.644370+0000","last_became_peered":"2024-03-04T08:56:30.644370+0000","last_unstale":"2024-03-04T08:56:35.671741+0000","last_undegraded":"2024-03-04T08:56:35.671741+0000","last_fullsized":"2024-03-04T08:56:35.671741+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:28:46.131719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.668127+0000","last_change":"2024-03-04T08:56:34.683727+0000","last_active":"2024-03-04T08:56:35.668127+0000","last_peered":"2024-03-04T08:56:35.668127+0000","last_clean":"2024-03-04T08:56:35.668127+0000","last_became_active":"2024-03-04T08:56:34.681865+0000","last_became_peered":"2024-03-04T08:56:34.681865+0000","last_unstale":"2024-03-04T08:56:35.668127+0000","last_undegraded":"2024-03-04T08:56:35.668127+0000","last_fullsized":"2024-03-04T08:56:35.668127+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:44:46.586791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083581+0000","last_change":"2024-03-04T08:56:29.983385+0000","last_active":"2024-03-04T08:56:37.083581+0000","last_peered":"2024-03-04T08:56:37.083581+0000","last_clean":"2024-03-04T08:56:37.083581+0000","last_became_active":"2024-03-04T08:56:29.983079+0000","last_became_peered":"2024-03-04T08:56:29.983079+0000","last_unstale":"2024-03-04T08:56:37.083581+0000","last_undegraded":"2024-03-04T08:56:37.083581+0000","last_fullsized":"2024-03-04T08:56:37.083581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:23:15.417389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671847+0000","last_change":"2024-03-04T08:56:30.641187+0000","last_active":"2024-03-04T08:56:35.671847+0000","last_peered":"2024-03-04T08:56:35.671847+0000","last_clean":"2024-03-04T08:56:35.671847+0000","last_became_active":"2024-03-04T08:56:30.640863+0000","last_became_peered":"2024-03-04T08:56:30.640863+0000","last_unstale":"2024-03-04T08:56:35.671847+0000","last_undegraded":"2024-03-04T08:56:35.671847+0000","last_fullsized":"2024-03-04T08:56:35.671847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:08:30.257806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084649+0000","last_change":"2024-03-04T08:56:34.684634+0000","last_active":"2024-03-04T08:56:37.084649+0000","last_peered":"2024-03-04T08:56:37.084649+0000","last_clean":"2024-03-04T08:56:37.084649+0000","last_became_active":"2024-03-04T08:56:34.684255+0000","last_became_peered":"2024-03-04T08:56:34.684255+0000","last_unstale":"2024-03-04T08:56:37.084649+0000","last_undegraded":"2024-03-04T08:56:37.084649+0000","last_fullsized":"2024-03-04T08:56:37.084649+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:30:59.381000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'8","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.582567+0000","last_change":"2024-03-04T08:56:29.994016+0000","last_active":"2024-03-04T08:56:36.582567+0000","last_peered":"2024-03-04T08:56:36.582567+0000","last_clean":"2024-03-04T08:56:36.582567+0000","last_became_active":"2024-03-04T08:56:29.993683+0000","last_became_peered":"2024-03-04T08:56:29.993683+0000","last_unstale":"2024-03-04T08:56:36.582567+0000","last_undegraded":"2024-03-04T08:56:36.582567+0000","last_fullsized":"2024-03-04T08:56:36.582567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:25:37.931248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551003+0000","last_change":"2024-03-04T08:56:30.647390+0000","last_active":"2024-03-04T08:56:36.551003+0000","last_peered":"2024-03-04T08:56:36.551003+0000","last_clean":"2024-03-04T08:56:36.551003+0000","last_became_active":"2024-03-04T08:56:30.643879+0000","last_became_peered":"2024-03-04T08:56:30.643879+0000","last_unstale":"2024-03-04T08:56:36.551003+0000","last_undegraded":"2024-03-04T08:56:36.551003+0000","last_fullsized":"2024-03-04T08:56:36.551003+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:41:13.030295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551192+0000","last_change":"2024-03-04T08:56:34.674836+0000","last_active":"2024-03-04T08:56:36.551192+0000","last_peered":"2024-03-04T08:56:36.551192+0000","last_clean":"2024-03-04T08:56:36.551192+0000","last_became_active":"2024-03-04T08:56:34.674418+0000","last_became_peered":"2024-03-04T08:56:34.674418+0000","last_unstale":"2024-03-04T08:56:36.551192+0000","last_undegraded":"2024-03-04T08:56:36.551192+0000","last_fullsized":"2024-03-04T08:56:36.551192+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:42:23.423166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.583144+0000","last_change":"2024-03-04T08:56:29.983195+0000","last_active":"2024-03-04T08:56:36.583144+0000","last_peered":"2024-03-04T08:56:36.583144+0000","last_clean":"2024-03-04T08:56:36.583144+0000","last_became_active":"2024-03-04T08:56:29.982881+0000","last_became_peered":"2024-03-04T08:56:29.982881+0000","last_unstale":"2024-03-04T08:56:36.583144+0000","last_undegraded":"2024-03-04T08:56:36.583144+0000","last_fullsized":"2024-03-04T08:56:36.583144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:59:12.802430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084168+0000","last_change":"2024-03-04T08:56:30.650452+0000","last_active":"2024-03-04T08:56:37.084168+0000","last_peered":"2024-03-04T08:56:37.084168+0000","last_clean":"2024-03-04T08:56:37.084168+0000","last_became_active":"2024-03-04T08:56:30.650148+0000","last_became_peered":"2024-03-04T08:56:30.650148+0000","last_unstale":"2024-03-04T08:56:37.084168+0000","last_undegraded":"2024-03-04T08:56:37.084168+0000","last_fullsized":"2024-03-04T08:56:37.084168+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:22:55.741505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.582668+0000","last_change":"2024-03-04T08:56:34.860608+0000","last_active":"2024-03-04T08:56:36.582668+0000","last_peered":"2024-03-04T08:56:36.582668+0000","last_clean":"2024-03-04T08:56:36.582668+0000","last_became_active":"2024-03-04T08:56:34.860087+0000","last_became_peered":"2024-03-04T08:56:34.860087+0000","last_unstale":"2024-03-04T08:56:36.582668+0000","last_undegraded":"2024-03-04T08:56:36.582668+0000","last_fullsized":"2024-03-04T08:56:36.582668+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:33:35.675886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.667783+0000","last_change":"2024-03-04T08:56:29.980947+0000","last_active":"2024-03-04T08:56:35.667783+0000","last_peered":"2024-03-04T08:56:35.667783+0000","last_clean":"2024-03-04T08:56:35.667783+0000","last_became_active":"2024-03-04T08:56:29.980625+0000","last_became_peered":"2024-03-04T08:56:29.980625+0000","last_unstale":"2024-03-04T08:56:35.667783+0000","last_undegraded":"2024-03-04T08:56:35.667783+0000","last_fullsized":"2024-03-04T08:56:35.667783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:53:53.944112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.374452+0000","last_change":"2024-03-04T08:56:30.642047+0000","last_active":"2024-03-04T08:56:36.374452+0000","last_peered":"2024-03-04T08:56:36.374452+0000","last_clean":"2024-03-04T08:56:36.374452+0000","last_became_active":"2024-03-04T08:56:30.641700+0000","last_became_peered":"2024-03-04T08:56:30.641700+0000","last_unstale":"2024-03-04T08:56:36.374452+0000","last_undegraded":"2024-03-04T08:56:36.374452+0000","last_fullsized":"2024-03-04T08:56:36.374452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:43:53.641512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084308+0000","last_change":"2024-03-04T08:56:34.674944+0000","last_active":"2024-03-04T08:56:37.084308+0000","last_peered":"2024-03-04T08:56:37.084308+0000","last_clean":"2024-03-04T08:56:37.084308+0000","last_became_active":"2024-03-04T08:56:34.674000+0000","last_became_peered":"2024-03-04T08:56:34.674000+0000","last_unstale":"2024-03-04T08:56:37.084308+0000","last_undegraded":"2024-03-04T08:56:37.084308+0000","last_fullsized":"2024-03-04T08:56:37.084308+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:26:19.386329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671613+0000","last_change":"2024-03-04T08:56:29.989020+0000","last_active":"2024-03-04T08:56:35.671613+0000","last_peered":"2024-03-04T08:56:35.671613+0000","last_clean":"2024-03-04T08:56:35.671613+0000","last_became_active":"2024-03-04T08:56:29.988751+0000","last_became_peered":"2024-03-04T08:56:29.988751+0000","last_unstale":"2024-03-04T08:56:35.671613+0000","last_undegraded":"2024-03-04T08:56:35.671613+0000","last_fullsized":"2024-03-04T08:56:35.671613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:38:45.003221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.374333+0000","last_change":"2024-03-04T08:56:30.649976+0000","last_active":"2024-03-04T08:56:36.374333+0000","last_peered":"2024-03-04T08:56:36.374333+0000","last_clean":"2024-03-04T08:56:36.374333+0000","last_became_active":"2024-03-04T08:56:30.649680+0000","last_became_peered":"2024-03-04T08:56:30.649680+0000","last_unstale":"2024-03-04T08:56:36.374333+0000","last_undegraded":"2024-03-04T08:56:36.374333+0000","last_fullsized":"2024-03-04T08:56:36.374333+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:50:02.197087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084391+0000","last_change":"2024-03-04T08:56:34.860654+0000","last_active":"2024-03-04T08:56:37.084391+0000","last_peered":"2024-03-04T08:56:37.084391+0000","last_clean":"2024-03-04T08:56:37.084391+0000","last_became_active":"2024-03-04T08:56:34.860170+0000","last_became_peered":"2024-03-04T08:56:34.860170+0000","last_unstale":"2024-03-04T08:56:37.084391+0000","last_undegraded":"2024-03-04T08:56:37.084391+0000","last_fullsized":"2024-03-04T08:56:37.084391+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:08:49.703317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551196+0000","last_change":"2024-03-04T08:56:30.643510+0000","last_active":"2024-03-04T08:56:36.551196+0000","last_peered":"2024-03-04T08:56:36.551196+0000","last_clean":"2024-03-04T08:56:36.551196+0000","last_became_active":"2024-03-04T08:56:30.643141+0000","last_became_peered":"2024-03-04T08:56:30.643141+0000","last_unstale":"2024-03-04T08:56:36.551196+0000","last_undegraded":"2024-03-04T08:56:36.551196+0000","last_fullsized":"2024-03-04T08:56:36.551196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:36:03.619735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083885+0000","last_change":"2024-03-04T08:56:29.994030+0000","last_active":"2024-03-04T08:56:37.083885+0000","last_peered":"2024-03-04T08:56:37.083885+0000","last_clean":"2024-03-04T08:56:37.083885+0000","last_became_active":"2024-03-04T08:56:29.993698+0000","last_became_peered":"2024-03-04T08:56:29.993698+0000","last_unstale":"2024-03-04T08:56:37.083885+0000","last_undegraded":"2024-03-04T08:56:37.083885+0000","last_fullsized":"2024-03-04T08:56:37.083885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:28:38.609330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.672003+0000","last_change":"2024-03-04T08:56:34.678337+0000","last_active":"2024-03-04T08:56:35.672003+0000","last_peered":"2024-03-04T08:56:35.672003+0000","last_clean":"2024-03-04T08:56:35.672003+0000","last_became_active":"2024-03-04T08:56:34.678091+0000","last_became_peered":"2024-03-04T08:56:34.678091+0000","last_unstale":"2024-03-04T08:56:35.672003+0000","last_undegraded":"2024-03-04T08:56:35.672003+0000","last_fullsized":"2024-03-04T08:56:35.672003+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:05:54.075257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.768434+0000","last_change":"2024-03-04T08:56:30.641609+0000","last_active":"2024-03-04T08:56:35.768434+0000","last_peered":"2024-03-04T08:56:35.768434+0000","last_clean":"2024-03-04T08:56:35.768434+0000","last_became_active":"2024-03-04T08:56:30.641323+0000","last_became_peered":"2024-03-04T08:56:30.641323+0000","last_unstale":"2024-03-04T08:56:35.768434+0000","last_undegraded":"2024-03-04T08:56:35.768434+0000","last_fullsized":"2024-03-04T08:56:35.768434+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:39:03.935424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083567+0000","last_change":"2024-03-04T08:56:29.976581+0000","last_active":"2024-03-04T08:56:37.083567+0000","last_peered":"2024-03-04T08:56:37.083567+0000","last_clean":"2024-03-04T08:56:37.083567+0000","last_became_active":"2024-03-04T08:56:29.976215+0000","last_became_peered":"2024-03-04T08:56:29.976215+0000","last_unstale":"2024-03-04T08:56:37.083567+0000","last_undegraded":"2024-03-04T08:56:37.083567+0000","last_fullsized":"2024-03-04T08:56:37.083567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:53:29.040404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":356,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671484+0000","last_change":"2024-03-04T08:55:10.344909+0000","last_active":"2024-03-04T08:56:35.671484+0000","last_peered":"2024-03-04T08:56:35.671484+0000","last_clean":"2024-03-04T08:56:35.671484+0000","last_became_active":"2024-03-04T08:55:10.344424+0000","last_became_peered":"2024-03-04T08:55:10.344424+0000","last_unstale":"2024-03-04T08:56:35.671484+0000","last_undegraded":"2024-03-04T08:56:35.671484+0000","last_fullsized":"2024-03-04T08:56:35.671484+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:51:15.334055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:51:15.334055+0000","last_clean_scrub_stamp":"2024-03-04T08:51:15.334055+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:36:29.766101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551809+0000","last_change":"2024-03-04T08:56:34.861141+0000","last_active":"2024-03-04T08:56:36.551809+0000","last_peered":"2024-03-04T08:56:36.551809+0000","last_clean":"2024-03-04T08:56:36.551809+0000","last_became_active":"2024-03-04T08:56:34.860618+0000","last_became_peered":"2024-03-04T08:56:34.860618+0000","last_unstale":"2024-03-04T08:56:36.551809+0000","last_undegraded":"2024-03-04T08:56:36.551809+0000","last_fullsized":"2024-03-04T08:56:36.551809+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:53:20.918385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083670+0000","last_change":"2024-03-04T08:56:30.644248+0000","last_active":"2024-03-04T08:56:37.083670+0000","last_peered":"2024-03-04T08:56:37.083670+0000","last_clean":"2024-03-04T08:56:37.083670+0000","last_became_active":"2024-03-04T08:56:30.643956+0000","last_became_peered":"2024-03-04T08:56:30.643956+0000","last_unstale":"2024-03-04T08:56:37.083670+0000","last_undegraded":"2024-03-04T08:56:37.083670+0000","last_fullsized":"2024-03-04T08:56:37.083670+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:19:24.298145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671596+0000","last_change":"2024-03-04T08:56:29.987584+0000","last_active":"2024-03-04T08:56:35.671596+0000","last_peered":"2024-03-04T08:56:35.671596+0000","last_clean":"2024-03-04T08:56:35.671596+0000","last_became_active":"2024-03-04T08:56:29.986741+0000","last_became_peered":"2024-03-04T08:56:29.986741+0000","last_unstale":"2024-03-04T08:56:35.671596+0000","last_undegraded":"2024-03-04T08:56:35.671596+0000","last_fullsized":"2024-03-04T08:56:35.671596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:46:19.672027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084402+0000","last_change":"2024-03-04T08:56:34.675485+0000","last_active":"2024-03-04T08:56:37.084402+0000","last_peered":"2024-03-04T08:56:37.084402+0000","last_clean":"2024-03-04T08:56:37.084402+0000","last_became_active":"2024-03-04T08:56:34.674961+0000","last_became_peered":"2024-03-04T08:56:34.674961+0000","last_unstale":"2024-03-04T08:56:37.084402+0000","last_undegraded":"2024-03-04T08:56:37.084402+0000","last_fullsized":"2024-03-04T08:56:37.084402+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:39:43.387849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.583044+0000","last_change":"2024-03-04T08:56:29.992287+0000","last_active":"2024-03-04T08:56:36.583044+0000","last_peered":"2024-03-04T08:56:36.583044+0000","last_clean":"2024-03-04T08:56:36.583044+0000","last_became_active":"2024-03-04T08:56:29.991998+0000","last_became_peered":"2024-03-04T08:56:29.991998+0000","last_unstale":"2024-03-04T08:56:36.583044+0000","last_undegraded":"2024-03-04T08:56:36.583044+0000","last_fullsized":"2024-03-04T08:56:36.583044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:33:10.557684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084308+0000","last_change":"2024-03-04T08:56:30.644190+0000","last_active":"2024-03-04T08:56:37.084308+0000","last_peered":"2024-03-04T08:56:37.084308+0000","last_clean":"2024-03-04T08:56:37.084308+0000","last_became_active":"2024-03-04T08:56:30.643783+0000","last_became_peered":"2024-03-04T08:56:30.643783+0000","last_unstale":"2024-03-04T08:56:37.084308+0000","last_undegraded":"2024-03-04T08:56:37.084308+0000","last_fullsized":"2024-03-04T08:56:37.084308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:25:12.886305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.768587+0000","last_change":"2024-03-04T08:56:34.860415+0000","last_active":"2024-03-04T08:56:35.768587+0000","last_peered":"2024-03-04T08:56:35.768587+0000","last_clean":"2024-03-04T08:56:35.768587+0000","last_became_active":"2024-03-04T08:56:34.860008+0000","last_became_peered":"2024-03-04T08:56:34.860008+0000","last_unstale":"2024-03-04T08:56:35.768587+0000","last_undegraded":"2024-03-04T08:56:35.768587+0000","last_fullsized":"2024-03-04T08:56:35.768587+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:55:45.673567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'1","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.668116+0000","last_change":"2024-03-04T08:56:29.989599+0000","last_active":"2024-03-04T08:56:35.668116+0000","last_peered":"2024-03-04T08:56:35.668116+0000","last_clean":"2024-03-04T08:56:35.668116+0000","last_became_active":"2024-03-04T08:56:29.989281+0000","last_became_peered":"2024-03-04T08:56:29.989281+0000","last_unstale":"2024-03-04T08:56:35.668116+0000","last_undegraded":"2024-03-04T08:56:35.668116+0000","last_fullsized":"2024-03-04T08:56:35.668116+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:59:04.006117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.768979+0000","last_change":"2024-03-04T08:56:30.640315+0000","last_active":"2024-03-04T08:56:35.768979+0000","last_peered":"2024-03-04T08:56:35.768979+0000","last_clean":"2024-03-04T08:56:35.768979+0000","last_became_active":"2024-03-04T08:56:30.640001+0000","last_became_peered":"2024-03-04T08:56:30.640001+0000","last_unstale":"2024-03-04T08:56:35.768979+0000","last_undegraded":"2024-03-04T08:56:35.768979+0000","last_fullsized":"2024-03-04T08:56:35.768979+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:43:57.049948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.843778+0000","last_change":"2024-03-04T08:56:34.667973+0000","last_active":"2024-03-04T08:56:35.843778+0000","last_peered":"2024-03-04T08:56:35.843778+0000","last_clean":"2024-03-04T08:56:35.843778+0000","last_became_active":"2024-03-04T08:56:34.667527+0000","last_became_peered":"2024-03-04T08:56:34.667527+0000","last_unstale":"2024-03-04T08:56:35.843778+0000","last_undegraded":"2024-03-04T08:56:35.843778+0000","last_fullsized":"2024-03-04T08:56:35.843778+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:07:23.991933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083772+0000","last_change":"2024-03-04T08:56:29.982263+0000","last_active":"2024-03-04T08:56:37.083772+0000","last_peered":"2024-03-04T08:56:37.083772+0000","last_clean":"2024-03-04T08:56:37.083772+0000","last_became_active":"2024-03-04T08:56:29.981802+0000","last_became_peered":"2024-03-04T08:56:29.981802+0000","last_unstale":"2024-03-04T08:56:37.083772+0000","last_undegraded":"2024-03-04T08:56:37.083772+0000","last_fullsized":"2024-03-04T08:56:37.083772+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:24:33.351428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084784+0000","last_change":"2024-03-04T08:56:30.657847+0000","last_active":"2024-03-04T08:56:37.084784+0000","last_peered":"2024-03-04T08:56:37.084784+0000","last_clean":"2024-03-04T08:56:37.084784+0000","last_became_active":"2024-03-04T08:56:30.657533+0000","last_became_peered":"2024-03-04T08:56:30.657533+0000","last_unstale":"2024-03-04T08:56:37.084784+0000","last_undegraded":"2024-03-04T08:56:37.084784+0000","last_fullsized":"2024-03-04T08:56:37.084784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:11:01.914455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083824+0000","last_change":"2024-03-04T08:56:34.861093+0000","last_active":"2024-03-04T08:56:37.083824+0000","last_peered":"2024-03-04T08:56:37.083824+0000","last_clean":"2024-03-04T08:56:37.083824+0000","last_became_active":"2024-03-04T08:56:34.860660+0000","last_became_peered":"2024-03-04T08:56:34.860660+0000","last_unstale":"2024-03-04T08:56:37.083824+0000","last_undegraded":"2024-03-04T08:56:37.083824+0000","last_fullsized":"2024-03-04T08:56:37.083824+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:37:38.424914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.670571+0000","last_change":"2024-03-04T08:56:29.990546+0000","last_active":"2024-03-04T08:56:35.670571+0000","last_peered":"2024-03-04T08:56:35.670571+0000","last_clean":"2024-03-04T08:56:35.670571+0000","last_became_active":"2024-03-04T08:56:29.990342+0000","last_became_peered":"2024-03-04T08:56:29.990342+0000","last_unstale":"2024-03-04T08:56:35.670571+0000","last_undegraded":"2024-03-04T08:56:35.670571+0000","last_fullsized":"2024-03-04T08:56:35.670571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:01:06.929112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.582559+0000","last_change":"2024-03-04T08:56:30.653228+0000","last_active":"2024-03-04T08:56:36.582559+0000","last_peered":"2024-03-04T08:56:36.582559+0000","last_clean":"2024-03-04T08:56:36.582559+0000","last_became_active":"2024-03-04T08:56:30.652926+0000","last_became_peered":"2024-03-04T08:56:30.652926+0000","last_unstale":"2024-03-04T08:56:36.582559+0000","last_undegraded":"2024-03-04T08:56:36.582559+0000","last_fullsized":"2024-03-04T08:56:36.582559+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:06:27.728135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.668292+0000","last_change":"2024-03-04T08:56:34.681096+0000","last_active":"2024-03-04T08:56:35.668292+0000","last_peered":"2024-03-04T08:56:35.668292+0000","last_clean":"2024-03-04T08:56:35.668292+0000","last_became_active":"2024-03-04T08:56:34.680718+0000","last_became_peered":"2024-03-04T08:56:34.680718+0000","last_unstale":"2024-03-04T08:56:35.668292+0000","last_undegraded":"2024-03-04T08:56:35.668292+0000","last_fullsized":"2024-03-04T08:56:35.668292+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:04:58.908904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.768441+0000","last_change":"2024-03-04T08:56:29.988039+0000","last_active":"2024-03-04T08:56:35.768441+0000","last_peered":"2024-03-04T08:56:35.768441+0000","last_clean":"2024-03-04T08:56:35.768441+0000","last_became_active":"2024-03-04T08:56:29.987310+0000","last_became_peered":"2024-03-04T08:56:29.987310+0000","last_unstale":"2024-03-04T08:56:35.768441+0000","last_undegraded":"2024-03-04T08:56:35.768441+0000","last_fullsized":"2024-03-04T08:56:35.768441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:19:39.319965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083909+0000","last_change":"2024-03-04T08:56:30.649094+0000","last_active":"2024-03-04T08:56:37.083909+0000","last_peered":"2024-03-04T08:56:37.083909+0000","last_clean":"2024-03-04T08:56:37.083909+0000","last_became_active":"2024-03-04T08:56:30.648776+0000","last_became_peered":"2024-03-04T08:56:30.648776+0000","last_unstale":"2024-03-04T08:56:37.083909+0000","last_undegraded":"2024-03-04T08:56:37.083909+0000","last_fullsized":"2024-03-04T08:56:37.083909+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:23:17.738792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:38.033048+0000","last_change":"2024-03-04T08:56:34.680923+0000","last_active":"2024-03-04T08:56:38.033048+0000","last_peered":"2024-03-04T08:56:38.033048+0000","last_clean":"2024-03-04T08:56:38.033048+0000","last_became_active":"2024-03-04T08:56:34.680114+0000","last_became_peered":"2024-03-04T08:56:34.680114+0000","last_unstale":"2024-03-04T08:56:38.033048+0000","last_undegraded":"2024-03-04T08:56:38.033048+0000","last_fullsized":"2024-03-04T08:56:38.033048+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:05:53.434735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.843884+0000","last_change":"2024-03-04T08:56:29.985725+0000","last_active":"2024-03-04T08:56:35.843884+0000","last_peered":"2024-03-04T08:56:35.843884+0000","last_clean":"2024-03-04T08:56:35.843884+0000","last_became_active":"2024-03-04T08:56:29.985363+0000","last_became_peered":"2024-03-04T08:56:29.985363+0000","last_unstale":"2024-03-04T08:56:35.843884+0000","last_undegraded":"2024-03-04T08:56:35.843884+0000","last_fullsized":"2024-03-04T08:56:35.843884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:15:07.432128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.768545+0000","last_change":"2024-03-04T08:56:30.654617+0000","last_active":"2024-03-04T08:56:35.768545+0000","last_peered":"2024-03-04T08:56:35.768545+0000","last_clean":"2024-03-04T08:56:35.768545+0000","last_became_active":"2024-03-04T08:56:30.653341+0000","last_became_peered":"2024-03-04T08:56:30.653341+0000","last_unstale":"2024-03-04T08:56:35.768545+0000","last_undegraded":"2024-03-04T08:56:35.768545+0000","last_fullsized":"2024-03-04T08:56:35.768545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:10:26.234514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083933+0000","last_change":"2024-03-04T08:56:34.860680+0000","last_active":"2024-03-04T08:56:37.083933+0000","last_peered":"2024-03-04T08:56:37.083933+0000","last_clean":"2024-03-04T08:56:37.083933+0000","last_became_active":"2024-03-04T08:56:34.860290+0000","last_became_peered":"2024-03-04T08:56:34.860290+0000","last_unstale":"2024-03-04T08:56:37.083933+0000","last_undegraded":"2024-03-04T08:56:37.083933+0000","last_fullsized":"2024-03-04T08:56:37.083933+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:14:32.126839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.582698+0000","last_change":"2024-03-04T08:56:29.989843+0000","last_active":"2024-03-04T08:56:36.582698+0000","last_peered":"2024-03-04T08:56:36.582698+0000","last_clean":"2024-03-04T08:56:36.582698+0000","last_became_active":"2024-03-04T08:56:29.989508+0000","last_became_peered":"2024-03-04T08:56:29.989508+0000","last_unstale":"2024-03-04T08:56:36.582698+0000","last_undegraded":"2024-03-04T08:56:36.582698+0000","last_fullsized":"2024-03-04T08:56:36.582698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:46:01.738580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671715+0000","last_change":"2024-03-04T08:56:30.644598+0000","last_active":"2024-03-04T08:56:35.671715+0000","last_peered":"2024-03-04T08:56:35.671715+0000","last_clean":"2024-03-04T08:56:35.671715+0000","last_became_active":"2024-03-04T08:56:30.644263+0000","last_became_peered":"2024-03-04T08:56:30.644263+0000","last_unstale":"2024-03-04T08:56:35.671715+0000","last_undegraded":"2024-03-04T08:56:35.671715+0000","last_fullsized":"2024-03-04T08:56:35.671715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:47:07.013271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671176+0000","last_change":"2024-03-04T08:56:34.675116+0000","last_active":"2024-03-04T08:56:35.671176+0000","last_peered":"2024-03-04T08:56:35.671176+0000","last_clean":"2024-03-04T08:56:35.671176+0000","last_became_active":"2024-03-04T08:56:34.674727+0000","last_became_peered":"2024-03-04T08:56:34.674727+0000","last_unstale":"2024-03-04T08:56:35.671176+0000","last_undegraded":"2024-03-04T08:56:35.671176+0000","last_fullsized":"2024-03-04T08:56:35.671176+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:56:23.461299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.844227+0000","last_change":"2024-03-04T08:56:29.983545+0000","last_active":"2024-03-04T08:56:35.844227+0000","last_peered":"2024-03-04T08:56:35.844227+0000","last_clean":"2024-03-04T08:56:35.844227+0000","last_became_active":"2024-03-04T08:56:29.983244+0000","last_became_peered":"2024-03-04T08:56:29.983244+0000","last_unstale":"2024-03-04T08:56:35.844227+0000","last_undegraded":"2024-03-04T08:56:35.844227+0000","last_fullsized":"2024-03-04T08:56:35.844227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:59:43.475203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.672182+0000","last_change":"2024-03-04T08:56:30.641244+0000","last_active":"2024-03-04T08:56:35.672182+0000","last_peered":"2024-03-04T08:56:35.672182+0000","last_clean":"2024-03-04T08:56:35.672182+0000","last_became_active":"2024-03-04T08:56:30.640952+0000","last_became_peered":"2024-03-04T08:56:30.640952+0000","last_unstale":"2024-03-04T08:56:35.672182+0000","last_undegraded":"2024-03-04T08:56:35.672182+0000","last_fullsized":"2024-03-04T08:56:35.672182+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:38:51.134697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083960+0000","last_change":"2024-03-04T08:56:34.670075+0000","last_active":"2024-03-04T08:56:37.083960+0000","last_peered":"2024-03-04T08:56:37.083960+0000","last_clean":"2024-03-04T08:56:37.083960+0000","last_became_active":"2024-03-04T08:56:34.669715+0000","last_became_peered":"2024-03-04T08:56:34.669715+0000","last_unstale":"2024-03-04T08:56:37.083960+0000","last_undegraded":"2024-03-04T08:56:37.083960+0000","last_fullsized":"2024-03-04T08:56:37.083960+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:32:01.768610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551673+0000","last_change":"2024-03-04T08:56:29.980781+0000","last_active":"2024-03-04T08:56:36.551673+0000","last_peered":"2024-03-04T08:56:36.551673+0000","last_clean":"2024-03-04T08:56:36.551673+0000","last_became_active":"2024-03-04T08:56:29.980446+0000","last_became_peered":"2024-03-04T08:56:29.980446+0000","last_unstale":"2024-03-04T08:56:36.551673+0000","last_undegraded":"2024-03-04T08:56:36.551673+0000","last_fullsized":"2024-03-04T08:56:36.551673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:05:37.325902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671770+0000","last_change":"2024-03-04T08:56:30.648350+0000","last_active":"2024-03-04T08:56:35.671770+0000","last_peered":"2024-03-04T08:56:35.671770+0000","last_clean":"2024-03-04T08:56:35.671770+0000","last_became_active":"2024-03-04T08:56:30.648059+0000","last_became_peered":"2024-03-04T08:56:30.648059+0000","last_unstale":"2024-03-04T08:56:35.671770+0000","last_undegraded":"2024-03-04T08:56:35.671770+0000","last_fullsized":"2024-03-04T08:56:35.671770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:59:07.571168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.550909+0000","last_change":"2024-03-04T08:56:34.673401+0000","last_active":"2024-03-04T08:56:36.550909+0000","last_peered":"2024-03-04T08:56:36.550909+0000","last_clean":"2024-03-04T08:56:36.550909+0000","last_became_active":"2024-03-04T08:56:34.672596+0000","last_became_peered":"2024-03-04T08:56:34.672596+0000","last_unstale":"2024-03-04T08:56:36.550909+0000","last_undegraded":"2024-03-04T08:56:36.550909+0000","last_fullsized":"2024-03-04T08:56:36.550909+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:39:56.613366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084498+0000","last_change":"2024-03-04T08:56:29.988331+0000","last_active":"2024-03-04T08:56:37.084498+0000","last_peered":"2024-03-04T08:56:37.084498+0000","last_clean":"2024-03-04T08:56:37.084498+0000","last_became_active":"2024-03-04T08:56:29.988009+0000","last_became_peered":"2024-03-04T08:56:29.988009+0000","last_unstale":"2024-03-04T08:56:37.084498+0000","last_undegraded":"2024-03-04T08:56:37.084498+0000","last_fullsized":"2024-03-04T08:56:37.084498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:50:43.693924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.671958+0000","last_change":"2024-03-04T08:56:30.647443+0000","last_active":"2024-03-04T08:56:35.671958+0000","last_peered":"2024-03-04T08:56:35.671958+0000","last_clean":"2024-03-04T08:56:35.671958+0000","last_became_active":"2024-03-04T08:56:30.647106+0000","last_became_peered":"2024-03-04T08:56:30.647106+0000","last_unstale":"2024-03-04T08:56:35.671958+0000","last_undegraded":"2024-03-04T08:56:35.671958+0000","last_fullsized":"2024-03-04T08:56:35.671958+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:09:56.320806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083968+0000","last_change":"2024-03-04T08:56:30.651243+0000","last_active":"2024-03-04T08:56:37.083968+0000","last_peered":"2024-03-04T08:56:37.083968+0000","last_clean":"2024-03-04T08:56:37.083968+0000","last_became_active":"2024-03-04T08:56:30.650781+0000","last_became_peered":"2024-03-04T08:56:30.650781+0000","last_unstale":"2024-03-04T08:56:37.083968+0000","last_undegraded":"2024-03-04T08:56:37.083968+0000","last_fullsized":"2024-03-04T08:56:37.083968+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:58:16.723542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.844206+0000","last_change":"2024-03-04T08:56:30.645737+0000","last_active":"2024-03-04T08:56:35.844206+0000","last_peered":"2024-03-04T08:56:35.844206+0000","last_clean":"2024-03-04T08:56:35.844206+0000","last_became_active":"2024-03-04T08:56:30.645293+0000","last_became_peered":"2024-03-04T08:56:30.645293+0000","last_unstale":"2024-03-04T08:56:35.844206+0000","last_undegraded":"2024-03-04T08:56:35.844206+0000","last_fullsized":"2024-03-04T08:56:35.844206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:56:04.302166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.843732+0000","last_change":"2024-03-04T08:56:30.649851+0000","last_active":"2024-03-04T08:56:35.843732+0000","last_peered":"2024-03-04T08:56:35.843732+0000","last_clean":"2024-03-04T08:56:35.843732+0000","last_became_active":"2024-03-04T08:56:30.649528+0000","last_became_peered":"2024-03-04T08:56:30.649528+0000","last_unstale":"2024-03-04T08:56:35.843732+0000","last_undegraded":"2024-03-04T08:56:35.843732+0000","last_fullsized":"2024-03-04T08:56:35.843732+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:36:50.431866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084432+0000","last_change":"2024-03-04T08:56:30.646893+0000","last_active":"2024-03-04T08:56:37.084432+0000","last_peered":"2024-03-04T08:56:37.084432+0000","last_clean":"2024-03-04T08:56:37.084432+0000","last_became_active":"2024-03-04T08:56:30.646615+0000","last_became_peered":"2024-03-04T08:56:30.646615+0000","last_unstale":"2024-03-04T08:56:37.084432+0000","last_undegraded":"2024-03-04T08:56:37.084432+0000","last_fullsized":"2024-03-04T08:56:37.084432+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:07:12.584686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084415+0000","last_change":"2024-03-04T08:56:30.649772+0000","last_active":"2024-03-04T08:56:37.084415+0000","last_peered":"2024-03-04T08:56:37.084415+0000","last_clean":"2024-03-04T08:56:37.084415+0000","last_became_active":"2024-03-04T08:56:30.649480+0000","last_became_peered":"2024-03-04T08:56:30.649480+0000","last_unstale":"2024-03-04T08:56:37.084415+0000","last_undegraded":"2024-03-04T08:56:37.084415+0000","last_fullsized":"2024-03-04T08:56:37.084415+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:14:10.553290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084181+0000","last_change":"2024-03-04T08:56:30.650819+0000","last_active":"2024-03-04T08:56:37.084181+0000","last_peered":"2024-03-04T08:56:37.084181+0000","last_clean":"2024-03-04T08:56:37.084181+0000","last_became_active":"2024-03-04T08:56:30.650505+0000","last_became_peered":"2024-03-04T08:56:30.650505+0000","last_unstale":"2024-03-04T08:56:37.084181+0000","last_undegraded":"2024-03-04T08:56:37.084181+0000","last_fullsized":"2024-03-04T08:56:37.084181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:53:36.877201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.844452+0000","last_change":"2024-03-04T08:56:30.646870+0000","last_active":"2024-03-04T08:56:35.844452+0000","last_peered":"2024-03-04T08:56:35.844452+0000","last_clean":"2024-03-04T08:56:35.844452+0000","last_became_active":"2024-03-04T08:56:30.646369+0000","last_became_peered":"2024-03-04T08:56:30.646369+0000","last_unstale":"2024-03-04T08:56:35.844452+0000","last_undegraded":"2024-03-04T08:56:35.844452+0000","last_fullsized":"2024-03-04T08:56:35.844452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:25:34.969089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.769070+0000","last_change":"2024-03-04T08:56:30.640854+0000","last_active":"2024-03-04T08:56:35.769070+0000","last_peered":"2024-03-04T08:56:35.769070+0000","last_clean":"2024-03-04T08:56:35.769070+0000","last_became_active":"2024-03-04T08:56:30.640484+0000","last_became_peered":"2024-03-04T08:56:30.640484+0000","last_unstale":"2024-03-04T08:56:35.769070+0000","last_undegraded":"2024-03-04T08:56:35.769070+0000","last_fullsized":"2024-03-04T08:56:35.769070+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:16:01.298720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:36.551079+0000","last_change":"2024-03-04T08:56:30.648593+0000","last_active":"2024-03-04T08:56:36.551079+0000","last_peered":"2024-03-04T08:56:36.551079+0000","last_clean":"2024-03-04T08:56:36.551079+0000","last_became_active":"2024-03-04T08:56:30.648287+0000","last_became_peered":"2024-03-04T08:56:30.648287+0000","last_unstale":"2024-03-04T08:56:36.551079+0000","last_undegraded":"2024-03-04T08:56:36.551079+0000","last_fullsized":"2024-03-04T08:56:36.551079+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:04:52.525357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.768566+0000","last_change":"2024-03-04T08:56:30.640284+0000","last_active":"2024-03-04T08:56:35.768566+0000","last_peered":"2024-03-04T08:56:35.768566+0000","last_clean":"2024-03-04T08:56:35.768566+0000","last_became_active":"2024-03-04T08:56:30.639968+0000","last_became_peered":"2024-03-04T08:56:30.639968+0000","last_unstale":"2024-03-04T08:56:35.768566+0000","last_undegraded":"2024-03-04T08:56:35.768566+0000","last_fullsized":"2024-03-04T08:56:35.768566+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:37:26.063318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:35.670931+0000","last_change":"2024-03-04T08:56:30.649547+0000","last_active":"2024-03-04T08:56:35.670931+0000","last_peered":"2024-03-04T08:56:35.670931+0000","last_clean":"2024-03-04T08:56:35.670931+0000","last_became_active":"2024-03-04T08:56:30.649170+0000","last_became_peered":"2024-03-04T08:56:30.649170+0000","last_unstale":"2024-03-04T08:56:35.670931+0000","last_undegraded":"2024-03-04T08:56:35.670931+0000","last_fullsized":"2024-03-04T08:56:35.670931+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:02:22.805496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.083961+0000","last_change":"2024-03-04T08:56:30.650892+0000","last_active":"2024-03-04T08:56:37.083961+0000","last_peered":"2024-03-04T08:56:37.083961+0000","last_clean":"2024-03-04T08:56:37.083961+0000","last_became_active":"2024-03-04T08:56:30.650442+0000","last_became_peered":"2024-03-04T08:56:30.650442+0000","last_unstale":"2024-03-04T08:56:37.083961+0000","last_undegraded":"2024-03-04T08:56:37.083961+0000","last_fullsized":"2024-03-04T08:56:37.083961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:40:38.217900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T08:56:37.084330+0000","last_change":"2024-03-04T08:56:30.647282+0000","last_active":"2024-03-04T08:56:37.084330+0000","last_peered":"2024-03-04T08:56:37.084330+0000","last_clean":"2024-03-04T08:56:37.084330+0000","last_became_active":"2024-03-04T08:56:30.646916+0000","last_became_peered":"2024-03-04T08:56:30.646916+0000","last_unstale":"2024-03-04T08:56:37.084330+0000","last_undegraded":"2024-03-04T08:56:37.084330+0000","last_fullsized":"2024-03-04T08:56:37.084330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:36:11.714223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":8,"num_read_kb":9,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":33219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318292,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":486056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579928,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1080460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743454,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1007616,"data_stored":486638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005095,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":486660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299433,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1601536,"data_stored":1077006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462959,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":486056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":509197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":486146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":489932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149761,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":486160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888124,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":486078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724599,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28592,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716560,"statfs":{"total":95995035648,"available":95965757440,"internally_reserved":0,"allocated":1613824,"data_stored":1080702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":23141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T08:56:58.780 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-04T08:56:58.781 INFO:tasks.cephadm:Setting up client nodes... 2024-03-04T08:56:58.781 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-04T08:56:59.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:58 smithi150 ceph-mon[36807]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:56:59.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:58 smithi150 ceph-mon[36807]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:56:59.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:58 smithi150 ceph-mon[36807]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-04T08:56:59.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:58 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1193363184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:59.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:58 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2648067688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:59.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:58 smithi158 ceph-mon[37241]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:56:59.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:58 smithi158 ceph-mon[37241]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:56:59.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:58 smithi158 ceph-mon[37241]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-04T08:56:59.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:58 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1193363184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:59.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:58 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2648067688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:59.058 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:56:59.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:58 smithi071 ceph-mon[32789]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:56:59.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:58 smithi071 ceph-mon[32789]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:56:59.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:58 smithi071 ceph-mon[32789]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-03-04T08:56:59.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:58 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1193363184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:56:59.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:58 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2648067688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:56:59.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:59 smithi071 ceph-mon[32789]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T08:56:59.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:59 smithi071 ceph-mon[32789]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 255 B/s wr, 0 op/s 2024-03-04T08:56:59.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:56:59 smithi071 ceph-mon[32789]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T08:57:00.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:59 smithi150 ceph-mon[36807]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T08:57:00.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:59 smithi150 ceph-mon[36807]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 255 B/s wr, 0 op/s 2024-03-04T08:57:00.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:56:59 smithi150 ceph-mon[36807]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T08:57:00.056 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:59 smithi158 ceph-mon[37241]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T08:57:00.057 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:59 smithi158 ceph-mon[37241]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 255 B/s wr, 0 op/s 2024-03-04T08:57:00.057 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:56:59 smithi158 ceph-mon[37241]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T08:57:00.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.857+0000 7ff75627c700 1 -- 172.21.15.71:0/3463879727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff750102ba0 msgr2=0x7ff750104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:00.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.857+0000 7ff75627c700 1 --2- 172.21.15.71:0/3463879727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff750102ba0 0x7ff750104f90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff740009990 tx=0x7ff740009ca0 comp rx=0 tx=0).stop 2024-03-04T08:57:00.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.857+0000 7ff75627c700 1 -- 172.21.15.71:0/3463879727 shutdown_connections 2024-03-04T08:57:00.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.857+0000 7ff75627c700 1 --2- 172.21.15.71:0/3463879727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff750107e00 0x7ff75010a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:00.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.857+0000 7ff75627c700 1 --2- 172.21.15.71:0/3463879727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff750102ba0 0x7ff750104f90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:00.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.857+0000 7ff75627c700 1 --2- 172.21.15.71:0/3463879727 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff7501054d0 0x7ff7501078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:00.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.857+0000 7ff75627c700 1 -- 172.21.15.71:0/3463879727 >> 172.21.15.71:0/3463879727 conn(0x7ff7500fc5d0 msgr2=0x7ff7500fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:00.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.857+0000 7ff75627c700 1 -- 172.21.15.71:0/3463879727 shutdown_connections 2024-03-04T08:57:00.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.857+0000 7ff75627c700 1 -- 172.21.15.71:0/3463879727 wait complete. 2024-03-04T08:57:00.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.858+0000 7ff75627c700 1 Processor -- start 2024-03-04T08:57:00.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.858+0000 7ff75627c700 1 -- start start 2024-03-04T08:57:00.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.858+0000 7ff75627c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff750102ba0 0x7ff750102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:00.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.858+0000 7ff75627c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff7501054d0 0x7ff750100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:00.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.858+0000 7ff75627c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff750107e00 0x7ff750101090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:00.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.858+0000 7ff75627c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75010dd20 con 0x7ff750102ba0 2024-03-04T08:57:00.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.858+0000 7ff75627c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75010dba0 con 0x7ff750107e00 2024-03-04T08:57:00.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.858+0000 7ff75627c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75010dea0 con 0x7ff7501054d0 2024-03-04T08:57:00.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.858+0000 7ff74ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff750102ba0 0x7ff750102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:00.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.858+0000 7ff74ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff750102ba0 0x7ff750102500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52354/0 (socket says 172.21.15.71:52354) 2024-03-04T08:57:00.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.858+0000 7ff74ffff700 1 -- 172.21.15.71:0/2452979190 learned_addr learned my addr 172.21.15.71:0/2452979190 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:57:00.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.858+0000 7ff754819700 1 --2- 172.21.15.71:0/2452979190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff750107e00 0x7ff750101090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:00.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.858+0000 7ff74f7fe700 1 --2- 172.21.15.71:0/2452979190 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff7501054d0 0x7ff750100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:00.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.858+0000 7ff74ffff700 1 -- 172.21.15.71:0/2452979190 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff7501054d0 msgr2=0x7ff750100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:00.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.858+0000 7ff74ffff700 1 --2- 172.21.15.71:0/2452979190 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff7501054d0 0x7ff750100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:00.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.858+0000 7ff74ffff700 1 -- 172.21.15.71:0/2452979190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff750107e00 msgr2=0x7ff750101090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:00.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.858+0000 7ff74ffff700 1 --2- 172.21.15.71:0/2452979190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff750107e00 0x7ff750101090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:00.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.858+0000 7ff74ffff700 1 -- 172.21.15.71:0/2452979190 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff750101940 con 0x7ff750102ba0 2024-03-04T08:57:00.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.858+0000 7ff74f7fe700 1 --2- 172.21.15.71:0/2452979190 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff7501054d0 0x7ff750100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:57:00.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.859+0000 7ff74ffff700 1 --2- 172.21.15.71:0/2452979190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff750102ba0 0x7ff750102500 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff74000b2e0 tx=0x7ff74000bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:00.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.859+0000 7ff74d7fa700 1 -- 172.21.15.71:0/2452979190 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff740027070 con 0x7ff750102ba0 2024-03-04T08:57:00.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.859+0000 7ff74d7fa700 1 -- 172.21.15.71:0/2452979190 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff740022470 con 0x7ff750102ba0 2024-03-04T08:57:00.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.859+0000 7ff75627c700 1 -- 172.21.15.71:0/2452979190 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7501a2390 con 0x7ff750102ba0 2024-03-04T08:57:00.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.859+0000 7ff74d7fa700 1 -- 172.21.15.71:0/2452979190 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74000f6c0 con 0x7ff750102ba0 2024-03-04T08:57:00.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.859+0000 7ff75627c700 1 -- 172.21.15.71:0/2452979190 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7501a27e0 con 0x7ff750102ba0 2024-03-04T08:57:00.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.860+0000 7ff75627c700 1 -- 172.21.15.71:0/2452979190 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7500505d0 con 0x7ff750102ba0 2024-03-04T08:57:00.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.861+0000 7ff74d7fa700 1 -- 172.21.15.71:0/2452979190 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7ff74002a070 con 0x7ff750102ba0 2024-03-04T08:57:00.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.861+0000 7ff74d7fa700 1 --2- 172.21.15.71:0/2452979190 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff7380a1fd0 0x7ff7380a4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:00.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.861+0000 7ff74f7fe700 1 --2- 172.21.15.71:0/2452979190 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff7380a1fd0 0x7ff7380a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:00.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.862+0000 7ff74d7fa700 1 -- 172.21.15.71:0/2452979190 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff7400cf160 con 0x7ff750102ba0 2024-03-04T08:57:00.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.862+0000 7ff74f7fe700 1 --2- 172.21.15.71:0/2452979190 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff7380a1fd0 0x7ff7380a4490 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff74400bcf0 tx=0x7ff74400b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:00.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:00.866+0000 7ff74d7fa700 1 -- 172.21.15.71:0/2452979190 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff74005e280 con 0x7ff750102ba0 2024-03-04T08:57:01.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:01.137+0000 7ff75627c700 1 -- 172.21.15.71:0/2452979190 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7ff7500008d0 con 0x7ff750102ba0 2024-03-04T08:57:01.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:01.140+0000 7ff74d7fa700 1 -- 172.21.15.71:0/2452979190 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7ff740013070 con 0x7ff750102ba0 2024-03-04T08:57:01.136 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2024-03-04T08:57:01.136 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQDdjOVlzyxOCBAAoKRVZ/JAihTfjuiD8wbtCw== 2024-03-04T08:57:01.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:01.142+0000 7ff75627c700 1 -- 172.21.15.71:0/2452979190 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff7380a1fd0 msgr2=0x7ff7380a4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:01.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:01.142+0000 7ff75627c700 1 --2- 172.21.15.71:0/2452979190 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff7380a1fd0 0x7ff7380a4490 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff74400bcf0 tx=0x7ff74400b2c0 comp rx=0 tx=0).stop 2024-03-04T08:57:01.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:01.142+0000 7ff75627c700 1 -- 172.21.15.71:0/2452979190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff750102ba0 msgr2=0x7ff750102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:01.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:01.142+0000 7ff75627c700 1 --2- 172.21.15.71:0/2452979190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff750102ba0 0x7ff750102500 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff74000b2e0 tx=0x7ff74000bbb0 comp rx=0 tx=0).stop 2024-03-04T08:57:01.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:01.143+0000 7ff75627c700 1 -- 172.21.15.71:0/2452979190 shutdown_connections 2024-03-04T08:57:01.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:01.143+0000 7ff75627c700 1 --2- 172.21.15.71:0/2452979190 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff7380a1fd0 0x7ff7380a4490 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:01.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:01.143+0000 7ff75627c700 1 --2- 172.21.15.71:0/2452979190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff750107e00 0x7ff750101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:01.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:01.143+0000 7ff75627c700 1 --2- 172.21.15.71:0/2452979190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff750102ba0 0x7ff750102500 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:01.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:01.143+0000 7ff75627c700 1 --2- 172.21.15.71:0/2452979190 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff7501054d0 0x7ff750100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:01.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:01.143+0000 7ff75627c700 1 -- 172.21.15.71:0/2452979190 >> 172.21.15.71:0/2452979190 conn(0x7ff7500fc5d0 msgr2=0x7ff7501034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:01.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:01.143+0000 7ff75627c700 1 -- 172.21.15.71:0/2452979190 shutdown_connections 2024-03-04T08:57:01.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:01.143+0000 7ff75627c700 1 -- 172.21.15.71:0/2452979190 wait complete. 2024-03-04T08:57:01.831 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-04T08:57:01.831 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-04T08:57:01.831 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-04T08:57:01.874 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-04T08:57:01.874 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-04T08:57:01.874 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph mgr dump --format=json 2024-03-04T08:57:02.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:01 smithi150 ceph-mon[36807]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 341 B/s wr, 1 op/s 2024-03-04T08:57:02.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:01 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2452979190' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T08:57:02.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:01 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2452979190' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-04T08:57:02.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:01 smithi158 ceph-mon[37241]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 341 B/s wr, 1 op/s 2024-03-04T08:57:02.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:01 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2452979190' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T08:57:02.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:01 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2452979190' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-04T08:57:02.165 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:02.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:01 smithi071 ceph-mon[32789]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 341 B/s wr, 1 op/s 2024-03-04T08:57:02.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:01 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2452979190' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T08:57:02.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:01 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2452979190' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-04T08:57:03.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.837+0000 7f3efc1b2700 1 -- 172.21.15.71:0/2253811322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ef4101960 msgr2=0x7f3ef410de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:03.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.837+0000 7f3efc1b2700 1 --2- 172.21.15.71:0/2253811322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ef4101960 0x7f3ef410de40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0009990 tx=0x7f3ef0009ca0 comp rx=0 tx=0).stop 2024-03-04T08:57:03.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.837+0000 7f3efc1b2700 1 -- 172.21.15.71:0/2253811322 shutdown_connections 2024-03-04T08:57:03.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.837+0000 7f3efc1b2700 1 --2- 172.21.15.71:0/2253811322 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3ef4107000 0x7f3ef41073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:03.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.837+0000 7f3efc1b2700 1 --2- 172.21.15.71:0/2253811322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ef4101960 0x7f3ef410de40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:03.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.837+0000 7f3efc1b2700 1 --2- 172.21.15.71:0/2253811322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ef4100fc0 0x7f3ef4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:03.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.837+0000 7f3efc1b2700 1 -- 172.21.15.71:0/2253811322 >> 172.21.15.71:0/2253811322 conn(0x7f3ef4077ef0 msgr2=0x7f3ef4078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:03.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.837+0000 7f3efc1b2700 1 -- 172.21.15.71:0/2253811322 shutdown_connections 2024-03-04T08:57:03.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.837+0000 7f3efc1b2700 1 -- 172.21.15.71:0/2253811322 wait complete. 2024-03-04T08:57:03.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.838+0000 7f3efc1b2700 1 Processor -- start 2024-03-04T08:57:03.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.838+0000 7f3efc1b2700 1 -- start start 2024-03-04T08:57:03.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.838+0000 7f3efc1b2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ef4100fc0 0x7f3ef419e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:03.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.838+0000 7f3efc1b2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ef4101960 0x7f3ef419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:03.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.838+0000 7f3efc1b2700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3ef4107000 0x7f3ef4198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:03.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.838+0000 7f3efc1b2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef4111280 con 0x7f3ef4100fc0 2024-03-04T08:57:03.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.838+0000 7f3efc1b2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef4111100 con 0x7f3ef4101960 2024-03-04T08:57:03.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.838+0000 7f3efc1b2700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef4111400 con 0x7f3ef4107000 2024-03-04T08:57:03.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.838+0000 7f3ef9f4e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ef4100fc0 0x7f3ef419e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:03.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.838+0000 7f3ef9f4e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ef4100fc0 0x7f3ef419e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52370/0 (socket says 172.21.15.71:52370) 2024-03-04T08:57:03.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.838+0000 7f3ef9f4e700 1 -- 172.21.15.71:0/2883075401 learned_addr learned my addr 172.21.15.71:0/2883075401 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:57:03.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.838+0000 7f3ef974d700 1 --2- 172.21.15.71:0/2883075401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ef4101960 0x7f3ef419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:03.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.838+0000 7f3efa74f700 1 --2- 172.21.15.71:0/2883075401 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3ef4107000 0x7f3ef4198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:03.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.838+0000 7f3ef9f4e700 1 -- 172.21.15.71:0/2883075401 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3ef4107000 msgr2=0x7f3ef4198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:03.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.838+0000 7f3ef9f4e700 1 --2- 172.21.15.71:0/2883075401 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3ef4107000 0x7f3ef4198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:03.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.838+0000 7f3ef9f4e700 1 -- 172.21.15.71:0/2883075401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ef4101960 msgr2=0x7f3ef419e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:03.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.838+0000 7f3ef9f4e700 1 --2- 172.21.15.71:0/2883075401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ef4101960 0x7f3ef419e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:03.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.838+0000 7f3ef9f4e700 1 -- 172.21.15.71:0/2883075401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef4198d60 con 0x7f3ef4100fc0 2024-03-04T08:57:03.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.838+0000 7f3efa74f700 1 --2- 172.21.15.71:0/2883075401 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3ef4107000 0x7f3ef4198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:57:03.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.839+0000 7f3ef9f4e700 1 --2- 172.21.15.71:0/2883075401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ef4100fc0 0x7f3ef419e440 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3eec007f10 tx=0x7f3eec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:03.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.839+0000 7f3eeaffd700 1 -- 172.21.15.71:0/2883075401 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec00dbd0 con 0x7f3ef4100fc0 2024-03-04T08:57:03.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.839+0000 7f3eeaffd700 1 -- 172.21.15.71:0/2883075401 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3eec00dd70 con 0x7f3ef4100fc0 2024-03-04T08:57:03.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.839+0000 7f3eeaffd700 1 -- 172.21.15.71:0/2883075401 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec0133f0 con 0x7f3ef4100fc0 2024-03-04T08:57:03.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.839+0000 7f3efc1b2700 1 -- 172.21.15.71:0/2883075401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef4199050 con 0x7f3ef4100fc0 2024-03-04T08:57:03.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.839+0000 7f3efc1b2700 1 -- 172.21.15.71:0/2883075401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef41a5310 con 0x7f3ef4100fc0 2024-03-04T08:57:03.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.840+0000 7f3efc1b2700 1 -- 172.21.15.71:0/2883075401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef4068d80 con 0x7f3ef4100fc0 2024-03-04T08:57:03.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.841+0000 7f3eeaffd700 1 -- 172.21.15.71:0/2883075401 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f3eec01a020 con 0x7f3ef4100fc0 2024-03-04T08:57:03.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.841+0000 7f3eeaffd700 1 --2- 172.21.15.71:0/2883075401 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3ee00a1ea0 0x7f3ee00a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:03.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.841+0000 7f3ef974d700 1 --2- 172.21.15.71:0/2883075401 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3ee00a1ea0 0x7f3ee00a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:03.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.841+0000 7f3eeaffd700 1 -- 172.21.15.71:0/2883075401 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f3eec0cd950 con 0x7f3ef4100fc0 2024-03-04T08:57:03.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.842+0000 7f3ef974d700 1 --2- 172.21.15.71:0/2883075401 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3ee00a1ea0 0x7f3ee00a4360 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3ef406adc0 tx=0x7f3ee4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:03.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:03.846+0000 7f3eeaffd700 1 -- 172.21.15.71:0/2883075401 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3eec010070 con 0x7f3ef4100fc0 2024-03-04T08:57:03.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:03 smithi071 ceph-mon[32789]: pgmap v341: 129 pgs: 129 active+clean; 622 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.3 KiB/s wr, 3 op/s 2024-03-04T08:57:04.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:03 smithi150 ceph-mon[36807]: pgmap v341: 129 pgs: 129 active+clean; 622 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.3 KiB/s wr, 3 op/s 2024-03-04T08:57:04.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:03 smithi158 ceph-mon[37241]: pgmap v341: 129 pgs: 129 active+clean; 622 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.3 KiB/s wr, 3 op/s 2024-03-04T08:57:04.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:04.098+0000 7f3efc1b2700 1 -- 172.21.15.71:0/2883075401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f3ef41a54e0 con 0x7f3ef4100fc0 2024-03-04T08:57:04.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:04.103+0000 7f3eeaffd700 1 -- 172.21.15.71:0/2883075401 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+261470 (secure 0 0 0) 0x7f3eec09a9e0 con 0x7f3ef4100fc0 2024-03-04T08:57:04.103 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:57:04.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:04.106+0000 7f3efc1b2700 1 -- 172.21.15.71:0/2883075401 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3ee00a1ea0 msgr2=0x7f3ee00a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:04.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:04.106+0000 7f3efc1b2700 1 --2- 172.21.15.71:0/2883075401 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3ee00a1ea0 0x7f3ee00a4360 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3ef406adc0 tx=0x7f3ee4009260 comp rx=0 tx=0).stop 2024-03-04T08:57:04.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:04.106+0000 7f3efc1b2700 1 -- 172.21.15.71:0/2883075401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ef4100fc0 msgr2=0x7f3ef419e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:04.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:04.106+0000 7f3efc1b2700 1 --2- 172.21.15.71:0/2883075401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ef4100fc0 0x7f3ef419e440 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3eec007f10 tx=0x7f3eec00d3b0 comp rx=0 tx=0).stop 2024-03-04T08:57:04.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:04.106+0000 7f3efc1b2700 1 -- 172.21.15.71:0/2883075401 shutdown_connections 2024-03-04T08:57:04.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:04.106+0000 7f3efc1b2700 1 --2- 172.21.15.71:0/2883075401 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3ee00a1ea0 0x7f3ee00a4360 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:04.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:04.106+0000 7f3efc1b2700 1 --2- 172.21.15.71:0/2883075401 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3ef4107000 0x7f3ef4198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:04.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:04.106+0000 7f3efc1b2700 1 --2- 172.21.15.71:0/2883075401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ef4100fc0 0x7f3ef419e440 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:04.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:04.106+0000 7f3efc1b2700 1 --2- 172.21.15.71:0/2883075401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ef4101960 0x7f3ef419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:04.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:04.106+0000 7f3efc1b2700 1 -- 172.21.15.71:0/2883075401 >> 172.21.15.71:0/2883075401 conn(0x7f3ef4077ef0 msgr2=0x7f3ef41051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:04.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:04.106+0000 7f3efc1b2700 1 -- 172.21.15.71:0/2883075401 shutdown_connections 2024-03-04T08:57:04.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:04.106+0000 7f3efc1b2700 1 -- 172.21.15.71:0/2883075401 wait complete. 2024-03-04T08:57:04.722 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2901868399},{"type":"v1","addr":"172.21.15.71:6801","nonce":2901868399}]},"active_addr":"172.21.15.71:6801/2901868399","active_change":"2024-03-04T08:48:22.594050+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24121,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24125,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.71:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":2792362717}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":1289352375}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":4186644246}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":1087047482}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":2585024353}]}]} 2024-03-04T08:57:04.727 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-04T08:57:04.727 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-04T08:57:04.727 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph osd dump --format=json 2024-03-04T08:57:04.970 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:05.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:04 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2883075401' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-04T08:57:05.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:04 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2883075401' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-04T08:57:05.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:04 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2883075401' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-04T08:57:06.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:05 smithi150 ceph-mon[36807]: pgmap v342: 129 pgs: 129 active+clean; 622 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.2 KiB/s wr, 3 op/s 2024-03-04T08:57:06.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:05 smithi158 ceph-mon[37241]: pgmap v342: 129 pgs: 129 active+clean; 622 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.2 KiB/s wr, 3 op/s 2024-03-04T08:57:06.103 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:05 smithi071 ceph-mon[32789]: pgmap v342: 129 pgs: 129 active+clean; 622 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.2 KiB/s wr, 3 op/s 2024-03-04T08:57:06.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.662+0000 7fdfa1f85700 1 -- 172.21.15.71:0/3691396378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf9c1080f0 msgr2=0x7fdf9c1084d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:06.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.662+0000 7fdfa1f85700 1 --2- 172.21.15.71:0/3691396378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf9c1080f0 0x7fdf9c1084d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00b0d0 tx=0x7fdf8c00b3e0 comp rx=0 tx=0).stop 2024-03-04T08:57:06.660 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.662+0000 7fdfa1f85700 1 -- 172.21.15.71:0/3691396378 shutdown_connections 2024-03-04T08:57:06.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.662+0000 7fdfa1f85700 1 --2- 172.21.15.71:0/3691396378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf9c102ad0 0x7fdf9c10ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:06.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.662+0000 7fdfa1f85700 1 --2- 172.21.15.71:0/3691396378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf9c1080f0 0x7fdf9c1084d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:06.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.662+0000 7fdfa1f85700 1 --2- 172.21.15.71:0/3691396378 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf9c102110 0x7fdf9c102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:06.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.662+0000 7fdfa1f85700 1 -- 172.21.15.71:0/3691396378 >> 172.21.15.71:0/3691396378 conn(0x7fdf9c0791f0 msgr2=0x7fdf9c079600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:06.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.662+0000 7fdfa1f85700 1 -- 172.21.15.71:0/3691396378 shutdown_connections 2024-03-04T08:57:06.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.662+0000 7fdfa1f85700 1 -- 172.21.15.71:0/3691396378 wait complete. 2024-03-04T08:57:06.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.662+0000 7fdfa1f85700 1 Processor -- start 2024-03-04T08:57:06.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.663+0000 7fdfa1f85700 1 -- start start 2024-03-04T08:57:06.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.663+0000 7fdfa1f85700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf9c102110 0x7fdf9c19e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:06.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.663+0000 7fdfa1f85700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf9c102ad0 0x7fdf9c19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:06.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.663+0000 7fdfa1f85700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf9c1080f0 0x7fdf9c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:06.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.663+0000 7fdfa1f85700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c1123f0 con 0x7fdf9c102ad0 2024-03-04T08:57:06.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.663+0000 7fdfa1f85700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c112270 con 0x7fdf9c1080f0 2024-03-04T08:57:06.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.663+0000 7fdfa1f85700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c112570 con 0x7fdf9c102110 2024-03-04T08:57:06.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.663+0000 7fdf9affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf9c102ad0 0x7fdf9c19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:06.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.663+0000 7fdf9affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf9c102ad0 0x7fdf9c19e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47642/0 (socket says 172.21.15.71:47642) 2024-03-04T08:57:06.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.663+0000 7fdf9affd700 1 -- 172.21.15.71:0/1081099295 learned_addr learned my addr 172.21.15.71:0/1081099295 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:57:06.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.663+0000 7fdf9bfff700 1 --2- 172.21.15.71:0/1081099295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf9c1080f0 0x7fdf9c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:06.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.663+0000 7fdf9b7fe700 1 --2- 172.21.15.71:0/1081099295 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf9c102110 0x7fdf9c19e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:06.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.663+0000 7fdf9affd700 1 -- 172.21.15.71:0/1081099295 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf9c102110 msgr2=0x7fdf9c19e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:06.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.663+0000 7fdf9affd700 1 --2- 172.21.15.71:0/1081099295 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf9c102110 0x7fdf9c19e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:06.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.663+0000 7fdf9affd700 1 -- 172.21.15.71:0/1081099295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf9c1080f0 msgr2=0x7fdf9c198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:06.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.663+0000 7fdf9affd700 1 --2- 172.21.15.71:0/1081099295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf9c1080f0 0x7fdf9c198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:06.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.663+0000 7fdf9affd700 1 -- 172.21.15.71:0/1081099295 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9c198dd0 con 0x7fdf9c102ad0 2024-03-04T08:57:06.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.663+0000 7fdf9b7fe700 1 --2- 172.21.15.71:0/1081099295 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf9c102110 0x7fdf9c19e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:57:06.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.664+0000 7fdf9affd700 1 --2- 172.21.15.71:0/1081099295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf9c102ad0 0x7fdf9c19e9a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000b530 tx=0x7fdf9000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:06.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.664+0000 7fdf98ff9700 1 -- 172.21.15.71:0/1081099295 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf90015020 con 0x7fdf9c102ad0 2024-03-04T08:57:06.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.664+0000 7fdf98ff9700 1 -- 172.21.15.71:0/1081099295 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf90011d10 con 0x7fdf9c102ad0 2024-03-04T08:57:06.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.664+0000 7fdfa1f85700 1 -- 172.21.15.71:0/1081099295 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9c1990c0 con 0x7fdf9c102ad0 2024-03-04T08:57:06.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.664+0000 7fdf98ff9700 1 -- 172.21.15.71:0/1081099295 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9000f2e0 con 0x7fdf9c102ad0 2024-03-04T08:57:06.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.664+0000 7fdfa1f85700 1 -- 172.21.15.71:0/1081099295 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9c1a53b0 con 0x7fdf9c102ad0 2024-03-04T08:57:06.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.665+0000 7fdfa1f85700 1 -- 172.21.15.71:0/1081099295 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf9c068d80 con 0x7fdf9c102ad0 2024-03-04T08:57:06.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.666+0000 7fdf98ff9700 1 -- 172.21.15.71:0/1081099295 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fdf90018020 con 0x7fdf9c102ad0 2024-03-04T08:57:06.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.666+0000 7fdf98ff9700 1 --2- 172.21.15.71:0/1081099295 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fdf840a1f20 0x7fdf840a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:06.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.667+0000 7fdf9b7fe700 1 --2- 172.21.15.71:0/1081099295 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fdf840a1f20 0x7fdf840a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:06.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.667+0000 7fdf98ff9700 1 -- 172.21.15.71:0/1081099295 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fdf900cdc40 con 0x7fdf9c102ad0 2024-03-04T08:57:06.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.667+0000 7fdf9b7fe700 1 --2- 172.21.15.71:0/1081099295 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fdf840a1f20 0x7fdf840a43e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c009850 tx=0x7fdf8c009780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:06.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.670+0000 7fdf98ff9700 1 -- 172.21.15.71:0/1081099295 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdf9001f020 con 0x7fdf9c102ad0 2024-03-04T08:57:06.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.872+0000 7fdfa1f85700 1 -- 172.21.15.71:0/1081099295 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf9c02d9b0 con 0x7fdf9c102ad0 2024-03-04T08:57:06.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.874+0000 7fdf98ff9700 1 -- 172.21.15.71:0/1081099295 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24153 (secure 0 0 0) 0x7fdf9009ade0 con 0x7fdf9c102ad0 2024-03-04T08:57:06.871 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:57:06.871 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":91,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","created":"2024-03-04T08:47:28.546102+0000","modified":"2024-03-04T08:56:35.007858+0000","last_up_change":"2024-03-04T08:55:30.744961+0000","last_in_change":"2024-03-04T08:55:11.570990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T08:51:14.933487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T08:56:28.253074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T08:56:29.543622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T08:56:32.884220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7e43b47d-8888-4631-bc96-8a96fcf8cd46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6803","nonce":1173641530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6805","nonce":1173641530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6809","nonce":1173641530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6807","nonce":1173641530}]},"public_addr":"172.21.15.71:6803/1173641530","cluster_addr":"172.21.15.71:6805/1173641530","heartbeat_back_addr":"172.21.15.71:6809/1173641530","heartbeat_front_addr":"172.21.15.71:6807/1173641530","state":["exists","up"]},{"osd":1,"uuid":"1209f214-f2e2-4209-bf69-4528a43a2123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6801","nonce":3383948110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6803","nonce":3383948110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6807","nonce":3383948110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6805","nonce":3383948110}]},"public_addr":"172.21.15.150:6801/3383948110","cluster_addr":"172.21.15.150:6803/3383948110","heartbeat_back_addr":"172.21.15.150:6807/3383948110","heartbeat_front_addr":"172.21.15.150:6805/3383948110","state":["exists","up"]},{"osd":2,"uuid":"b88c3082-93a9-4c9b-9221-a7105a44c91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6801","nonce":586944144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6803","nonce":586944144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6807","nonce":586944144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6805","nonce":586944144}]},"public_addr":"172.21.15.158:6801/586944144","cluster_addr":"172.21.15.158:6803/586944144","heartbeat_back_addr":"172.21.15.158:6807/586944144","heartbeat_front_addr":"172.21.15.158:6805/586944144","state":["exists","up"]},{"osd":3,"uuid":"e4fc7723-fd73-40b2-b9d0-7f9bda282ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6811","nonce":1311903262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6813","nonce":1311903262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6817","nonce":1311903262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6815","nonce":1311903262}]},"public_addr":"172.21.15.71:6811/1311903262","cluster_addr":"172.21.15.71:6813/1311903262","heartbeat_back_addr":"172.21.15.71:6817/1311903262","heartbeat_front_addr":"172.21.15.71:6815/1311903262","state":["exists","up"]},{"osd":4,"uuid":"e7eb72ac-f066-4ccd-b2bf-22a6a26daf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6809","nonce":2300837573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6811","nonce":2300837573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6815","nonce":2300837573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6813","nonce":2300837573}]},"public_addr":"172.21.15.150:6809/2300837573","cluster_addr":"172.21.15.150:6811/2300837573","heartbeat_back_addr":"172.21.15.150:6815/2300837573","heartbeat_front_addr":"172.21.15.150:6813/2300837573","state":["exists","up"]},{"osd":5,"uuid":"9b295840-e104-4a66-9cc4-7020627c1765","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6809","nonce":1922694045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6811","nonce":1922694045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6815","nonce":1922694045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6813","nonce":1922694045}]},"public_addr":"172.21.15.158:6809/1922694045","cluster_addr":"172.21.15.158:6811/1922694045","heartbeat_back_addr":"172.21.15.158:6815/1922694045","heartbeat_front_addr":"172.21.15.158:6813/1922694045","state":["exists","up"]},{"osd":6,"uuid":"6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6819","nonce":3848842707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6821","nonce":3848842707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6825","nonce":3848842707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6823","nonce":3848842707}]},"public_addr":"172.21.15.71:6819/3848842707","cluster_addr":"172.21.15.71:6821/3848842707","heartbeat_back_addr":"172.21.15.71:6825/3848842707","heartbeat_front_addr":"172.21.15.71:6823/3848842707","state":["exists","up"]},{"osd":7,"uuid":"6160a40a-d30f-4569-949c-0afb1bb3d133","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6817","nonce":1328330305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6819","nonce":1328330305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6823","nonce":1328330305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6821","nonce":1328330305}]},"public_addr":"172.21.15.150:6817/1328330305","cluster_addr":"172.21.15.150:6819/1328330305","heartbeat_back_addr":"172.21.15.150:6823/1328330305","heartbeat_front_addr":"172.21.15.150:6821/1328330305","state":["exists","up"]},{"osd":8,"uuid":"95ed2341-84a4-465c-9948-ac667680ce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6817","nonce":2195304577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6819","nonce":2195304577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6823","nonce":2195304577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6821","nonce":2195304577}]},"public_addr":"172.21.15.158:6817/2195304577","cluster_addr":"172.21.15.158:6819/2195304577","heartbeat_back_addr":"172.21.15.158:6823/2195304577","heartbeat_front_addr":"172.21.15.158:6821/2195304577","state":["exists","up"]},{"osd":9,"uuid":"088f6419-63a0-4768-afae-dbb8a5f5a565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6827","nonce":1913779347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6829","nonce":1913779347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6833","nonce":1913779347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6831","nonce":1913779347}]},"public_addr":"172.21.15.71:6827/1913779347","cluster_addr":"172.21.15.71:6829/1913779347","heartbeat_back_addr":"172.21.15.71:6833/1913779347","heartbeat_front_addr":"172.21.15.71:6831/1913779347","state":["exists","up"]},{"osd":10,"uuid":"18c85e24-068d-4f1e-a3b8-f7915b363234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6825","nonce":3295420996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6827","nonce":3295420996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6831","nonce":3295420996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6829","nonce":3295420996}]},"public_addr":"172.21.15.150:6825/3295420996","cluster_addr":"172.21.15.150:6827/3295420996","heartbeat_back_addr":"172.21.15.150:6831/3295420996","heartbeat_front_addr":"172.21.15.150:6829/3295420996","state":["exists","up"]},{"osd":11,"uuid":"4f6b3ddf-c482-4740-b1fd-de836e047c2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6825","nonce":3332083865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6827","nonce":3332083865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6831","nonce":3332083865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6829","nonce":3332083865}]},"public_addr":"172.21.15.158:6825/3332083865","cluster_addr":"172.21.15.158:6827/3332083865","heartbeat_back_addr":"172.21.15.158:6831/3332083865","heartbeat_front_addr":"172.21.15.158:6829/3332083865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:05.997832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:29.314235+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:08.819465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:41.194275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:06.032479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:38.814498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:13.067960+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:38.568479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:03.416713+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:38.349479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:04.096945+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:29.425250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1287712358":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/3770452860":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/718743094":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/1904717532":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:6800/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6800/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/2862184313":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/3621754630":"2024-03-05T08:48:22.593931+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T08:57:06.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.877+0000 7fdfa1f85700 1 -- 172.21.15.71:0/1081099295 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fdf840a1f20 msgr2=0x7fdf840a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:06.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.877+0000 7fdfa1f85700 1 --2- 172.21.15.71:0/1081099295 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fdf840a1f20 0x7fdf840a43e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c009850 tx=0x7fdf8c009780 comp rx=0 tx=0).stop 2024-03-04T08:57:06.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.877+0000 7fdfa1f85700 1 -- 172.21.15.71:0/1081099295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf9c102ad0 msgr2=0x7fdf9c19e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:06.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.877+0000 7fdfa1f85700 1 --2- 172.21.15.71:0/1081099295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf9c102ad0 0x7fdf9c19e9a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000b530 tx=0x7fdf9000b840 comp rx=0 tx=0).stop 2024-03-04T08:57:06.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.877+0000 7fdfa1f85700 1 -- 172.21.15.71:0/1081099295 shutdown_connections 2024-03-04T08:57:06.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.877+0000 7fdfa1f85700 1 --2- 172.21.15.71:0/1081099295 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fdf840a1f20 0x7fdf840a43e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:06.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.877+0000 7fdfa1f85700 1 --2- 172.21.15.71:0/1081099295 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdf9c102110 0x7fdf9c19e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:06.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.877+0000 7fdfa1f85700 1 --2- 172.21.15.71:0/1081099295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf9c1080f0 0x7fdf9c198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:06.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.877+0000 7fdfa1f85700 1 --2- 172.21.15.71:0/1081099295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf9c102ad0 0x7fdf9c19e9a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:06.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.877+0000 7fdfa1f85700 1 -- 172.21.15.71:0/1081099295 >> 172.21.15.71:0/1081099295 conn(0x7fdf9c0791f0 msgr2=0x7fdf9c100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:06.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.877+0000 7fdfa1f85700 1 -- 172.21.15.71:0/1081099295 shutdown_connections 2024-03-04T08:57:06.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:06.877+0000 7fdfa1f85700 1 -- 172.21.15.71:0/1081099295 wait complete. 2024-03-04T08:57:07.532 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-04T08:57:07.533 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph osd dump --format=json 2024-03-04T08:57:07.774 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:08.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:07 smithi150 ceph-mon[36807]: pgmap v343: 129 pgs: 129 active+clean; 638 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5.9 KiB/s wr, 5 op/s 2024-03-04T08:57:08.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:07 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1081099295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:57:08.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:07 smithi071 ceph-mon[32789]: pgmap v343: 129 pgs: 129 active+clean; 638 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5.9 KiB/s wr, 5 op/s 2024-03-04T08:57:08.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:07 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1081099295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:57:08.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:07 smithi158 ceph-mon[37241]: pgmap v343: 129 pgs: 129 active+clean; 638 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5.9 KiB/s wr, 5 op/s 2024-03-04T08:57:08.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:07 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1081099295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:57:09.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.463+0000 7ff290269700 1 -- 172.21.15.71:0/1044843675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff288100c60 msgr2=0x7ff288101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:09.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.463+0000 7ff290269700 1 --2- 172.21.15.71:0/1044843675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff288100c60 0x7ff288101040 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff2840099c0 tx=0x7ff284009cd0 comp rx=0 tx=0).stop 2024-03-04T08:57:09.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.464+0000 7ff290269700 1 -- 172.21.15.71:0/1044843675 shutdown_connections 2024-03-04T08:57:09.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.464+0000 7ff290269700 1 --2- 172.21.15.71:0/1044843675 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff288107480 0x7ff28810f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:09.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.464+0000 7ff290269700 1 --2- 172.21.15.71:0/1044843675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff288100c60 0x7ff288101040 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:09.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.464+0000 7ff290269700 1 --2- 172.21.15.71:0/1044843675 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff288101610 0x7ff288106d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:09.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.464+0000 7ff290269700 1 -- 172.21.15.71:0/1044843675 >> 172.21.15.71:0/1044843675 conn(0x7ff288076320 msgr2=0x7ff288076730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:09.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.464+0000 7ff290269700 1 -- 172.21.15.71:0/1044843675 shutdown_connections 2024-03-04T08:57:09.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.464+0000 7ff290269700 1 -- 172.21.15.71:0/1044843675 wait complete. 2024-03-04T08:57:09.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.464+0000 7ff290269700 1 Processor -- start 2024-03-04T08:57:09.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.464+0000 7ff290269700 1 -- start start 2024-03-04T08:57:09.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.464+0000 7ff290269700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff288100c60 0x7ff28819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:09.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.464+0000 7ff290269700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff288101610 0x7ff28819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:09.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.464+0000 7ff290269700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff288107480 0x7ff288198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:09.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.464+0000 7ff290269700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff288112460 con 0x7ff288101610 2024-03-04T08:57:09.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.464+0000 7ff290269700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2881122e0 con 0x7ff288100c60 2024-03-04T08:57:09.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.464+0000 7ff290269700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2881125e0 con 0x7ff288107480 2024-03-04T08:57:09.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.465+0000 7ff28d804700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff288101610 0x7ff28819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:09.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.465+0000 7ff28d804700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff288101610 0x7ff28819eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47660/0 (socket says 172.21.15.71:47660) 2024-03-04T08:57:09.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.465+0000 7ff28d804700 1 -- 172.21.15.71:0/2487184747 learned_addr learned my addr 172.21.15.71:0/2487184747 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:57:09.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.465+0000 7ff28e005700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff288100c60 0x7ff28819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:09.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.465+0000 7ff28e806700 1 --2- 172.21.15.71:0/2487184747 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff288107480 0x7ff288198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:09.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.465+0000 7ff28d804700 1 -- 172.21.15.71:0/2487184747 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff288107480 msgr2=0x7ff288198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:09.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.465+0000 7ff28d804700 1 --2- 172.21.15.71:0/2487184747 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff288107480 0x7ff288198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:09.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.465+0000 7ff28d804700 1 -- 172.21.15.71:0/2487184747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff288100c60 msgr2=0x7ff28819e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:09.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.465+0000 7ff28d804700 1 --2- 172.21.15.71:0/2487184747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff288100c60 0x7ff28819e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:09.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.465+0000 7ff28d804700 1 -- 172.21.15.71:0/2487184747 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff288199040 con 0x7ff288101610 2024-03-04T08:57:09.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.465+0000 7ff28e005700 1 --2- 172.21.15.71:0/2487184747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff288100c60 0x7ff28819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:57:09.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.465+0000 7ff28d804700 1 --2- 172.21.15.71:0/2487184747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff288101610 0x7ff28819eb80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff27800c8c0 tx=0x7ff27800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:09.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.465+0000 7ff27f7fe700 1 -- 172.21.15.71:0/2487184747 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278004220 con 0x7ff288101610 2024-03-04T08:57:09.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.465+0000 7ff27f7fe700 1 -- 172.21.15.71:0/2487184747 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff278004d10 con 0x7ff288101610 2024-03-04T08:57:09.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.465+0000 7ff290269700 1 -- 172.21.15.71:0/2487184747 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff288199330 con 0x7ff288101610 2024-03-04T08:57:09.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.465+0000 7ff27f7fe700 1 -- 172.21.15.71:0/2487184747 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2780038e0 con 0x7ff288101610 2024-03-04T08:57:09.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.465+0000 7ff290269700 1 -- 172.21.15.71:0/2487184747 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2881a55c0 con 0x7ff288101610 2024-03-04T08:57:09.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.466+0000 7ff290269700 1 -- 172.21.15.71:0/2487184747 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff28806a330 con 0x7ff288101610 2024-03-04T08:57:09.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.467+0000 7ff27f7fe700 1 -- 172.21.15.71:0/2487184747 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7ff278004ec0 con 0x7ff288101610 2024-03-04T08:57:09.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.468+0000 7ff27f7fe700 1 --2- 172.21.15.71:0/2487184747 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff2740a1ea0 0x7ff2740a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:09.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.468+0000 7ff28e005700 1 --2- 172.21.15.71:0/2487184747 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff2740a1ea0 0x7ff2740a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:09.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.468+0000 7ff27f7fe700 1 -- 172.21.15.71:0/2487184747 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff2780ce050 con 0x7ff288101610 2024-03-04T08:57:09.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.468+0000 7ff28e005700 1 --2- 172.21.15.71:0/2487184747 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff2740a1ea0 0x7ff2740a4360 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff284000c00 tx=0x7ff284005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:09.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.472+0000 7ff27f7fe700 1 -- 172.21.15.71:0/2487184747 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff278014030 con 0x7ff288101610 2024-03-04T08:57:09.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.665+0000 7ff290269700 1 -- 172.21.15.71:0/2487184747 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff288199c40 con 0x7ff288101610 2024-03-04T08:57:09.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.666+0000 7ff27f7fe700 1 -- 172.21.15.71:0/2487184747 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24153 (secure 0 0 0) 0x7ff27809b0e0 con 0x7ff288101610 2024-03-04T08:57:09.663 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:57:09.663 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":91,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","created":"2024-03-04T08:47:28.546102+0000","modified":"2024-03-04T08:56:35.007858+0000","last_up_change":"2024-03-04T08:55:30.744961+0000","last_in_change":"2024-03-04T08:55:11.570990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T08:51:14.933487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T08:56:28.253074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T08:56:29.543622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T08:56:32.884220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7e43b47d-8888-4631-bc96-8a96fcf8cd46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6803","nonce":1173641530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6805","nonce":1173641530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6809","nonce":1173641530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6807","nonce":1173641530}]},"public_addr":"172.21.15.71:6803/1173641530","cluster_addr":"172.21.15.71:6805/1173641530","heartbeat_back_addr":"172.21.15.71:6809/1173641530","heartbeat_front_addr":"172.21.15.71:6807/1173641530","state":["exists","up"]},{"osd":1,"uuid":"1209f214-f2e2-4209-bf69-4528a43a2123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6801","nonce":3383948110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6803","nonce":3383948110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6807","nonce":3383948110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6805","nonce":3383948110}]},"public_addr":"172.21.15.150:6801/3383948110","cluster_addr":"172.21.15.150:6803/3383948110","heartbeat_back_addr":"172.21.15.150:6807/3383948110","heartbeat_front_addr":"172.21.15.150:6805/3383948110","state":["exists","up"]},{"osd":2,"uuid":"b88c3082-93a9-4c9b-9221-a7105a44c91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6801","nonce":586944144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6803","nonce":586944144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6807","nonce":586944144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6805","nonce":586944144}]},"public_addr":"172.21.15.158:6801/586944144","cluster_addr":"172.21.15.158:6803/586944144","heartbeat_back_addr":"172.21.15.158:6807/586944144","heartbeat_front_addr":"172.21.15.158:6805/586944144","state":["exists","up"]},{"osd":3,"uuid":"e4fc7723-fd73-40b2-b9d0-7f9bda282ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6811","nonce":1311903262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6813","nonce":1311903262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6817","nonce":1311903262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6815","nonce":1311903262}]},"public_addr":"172.21.15.71:6811/1311903262","cluster_addr":"172.21.15.71:6813/1311903262","heartbeat_back_addr":"172.21.15.71:6817/1311903262","heartbeat_front_addr":"172.21.15.71:6815/1311903262","state":["exists","up"]},{"osd":4,"uuid":"e7eb72ac-f066-4ccd-b2bf-22a6a26daf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6809","nonce":2300837573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6811","nonce":2300837573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6815","nonce":2300837573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6813","nonce":2300837573}]},"public_addr":"172.21.15.150:6809/2300837573","cluster_addr":"172.21.15.150:6811/2300837573","heartbeat_back_addr":"172.21.15.150:6815/2300837573","heartbeat_front_addr":"172.21.15.150:6813/2300837573","state":["exists","up"]},{"osd":5,"uuid":"9b295840-e104-4a66-9cc4-7020627c1765","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6809","nonce":1922694045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6811","nonce":1922694045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6815","nonce":1922694045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6813","nonce":1922694045}]},"public_addr":"172.21.15.158:6809/1922694045","cluster_addr":"172.21.15.158:6811/1922694045","heartbeat_back_addr":"172.21.15.158:6815/1922694045","heartbeat_front_addr":"172.21.15.158:6813/1922694045","state":["exists","up"]},{"osd":6,"uuid":"6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6819","nonce":3848842707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6821","nonce":3848842707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6825","nonce":3848842707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6823","nonce":3848842707}]},"public_addr":"172.21.15.71:6819/3848842707","cluster_addr":"172.21.15.71:6821/3848842707","heartbeat_back_addr":"172.21.15.71:6825/3848842707","heartbeat_front_addr":"172.21.15.71:6823/3848842707","state":["exists","up"]},{"osd":7,"uuid":"6160a40a-d30f-4569-949c-0afb1bb3d133","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6817","nonce":1328330305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6819","nonce":1328330305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6823","nonce":1328330305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6821","nonce":1328330305}]},"public_addr":"172.21.15.150:6817/1328330305","cluster_addr":"172.21.15.150:6819/1328330305","heartbeat_back_addr":"172.21.15.150:6823/1328330305","heartbeat_front_addr":"172.21.15.150:6821/1328330305","state":["exists","up"]},{"osd":8,"uuid":"95ed2341-84a4-465c-9948-ac667680ce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6817","nonce":2195304577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6819","nonce":2195304577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6823","nonce":2195304577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6821","nonce":2195304577}]},"public_addr":"172.21.15.158:6817/2195304577","cluster_addr":"172.21.15.158:6819/2195304577","heartbeat_back_addr":"172.21.15.158:6823/2195304577","heartbeat_front_addr":"172.21.15.158:6821/2195304577","state":["exists","up"]},{"osd":9,"uuid":"088f6419-63a0-4768-afae-dbb8a5f5a565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6827","nonce":1913779347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6829","nonce":1913779347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6833","nonce":1913779347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6831","nonce":1913779347}]},"public_addr":"172.21.15.71:6827/1913779347","cluster_addr":"172.21.15.71:6829/1913779347","heartbeat_back_addr":"172.21.15.71:6833/1913779347","heartbeat_front_addr":"172.21.15.71:6831/1913779347","state":["exists","up"]},{"osd":10,"uuid":"18c85e24-068d-4f1e-a3b8-f7915b363234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6825","nonce":3295420996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6827","nonce":3295420996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6831","nonce":3295420996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6829","nonce":3295420996}]},"public_addr":"172.21.15.150:6825/3295420996","cluster_addr":"172.21.15.150:6827/3295420996","heartbeat_back_addr":"172.21.15.150:6831/3295420996","heartbeat_front_addr":"172.21.15.150:6829/3295420996","state":["exists","up"]},{"osd":11,"uuid":"4f6b3ddf-c482-4740-b1fd-de836e047c2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6825","nonce":3332083865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6827","nonce":3332083865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6831","nonce":3332083865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6829","nonce":3332083865}]},"public_addr":"172.21.15.158:6825/3332083865","cluster_addr":"172.21.15.158:6827/3332083865","heartbeat_back_addr":"172.21.15.158:6831/3332083865","heartbeat_front_addr":"172.21.15.158:6829/3332083865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:05.997832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:29.314235+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:08.819465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:41.194275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:06.032479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:38.814498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:13.067960+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:38.568479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:03.416713+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:38.349479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:04.096945+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:29.425250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1287712358":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/3770452860":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/718743094":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/1904717532":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:6800/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6800/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/2862184313":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/3621754630":"2024-03-05T08:48:22.593931+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T08:57:09.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.669+0000 7ff290269700 1 -- 172.21.15.71:0/2487184747 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff2740a1ea0 msgr2=0x7ff2740a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:09.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.669+0000 7ff290269700 1 --2- 172.21.15.71:0/2487184747 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff2740a1ea0 0x7ff2740a4360 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff284000c00 tx=0x7ff284005fb0 comp rx=0 tx=0).stop 2024-03-04T08:57:09.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.669+0000 7ff290269700 1 -- 172.21.15.71:0/2487184747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff288101610 msgr2=0x7ff28819eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:09.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.669+0000 7ff290269700 1 --2- 172.21.15.71:0/2487184747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff288101610 0x7ff28819eb80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff27800c8c0 tx=0x7ff27800cbd0 comp rx=0 tx=0).stop 2024-03-04T08:57:09.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.669+0000 7ff290269700 1 -- 172.21.15.71:0/2487184747 shutdown_connections 2024-03-04T08:57:09.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.669+0000 7ff290269700 1 --2- 172.21.15.71:0/2487184747 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff2740a1ea0 0x7ff2740a4360 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:09.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.669+0000 7ff290269700 1 --2- 172.21.15.71:0/2487184747 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff288107480 0x7ff288198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:09.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.669+0000 7ff290269700 1 --2- 172.21.15.71:0/2487184747 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff288100c60 0x7ff28819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:09.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.669+0000 7ff290269700 1 --2- 172.21.15.71:0/2487184747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff288101610 0x7ff28819eb80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:09.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.669+0000 7ff290269700 1 -- 172.21.15.71:0/2487184747 >> 172.21.15.71:0/2487184747 conn(0x7ff288076320 msgr2=0x7ff288100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:09.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.669+0000 7ff290269700 1 -- 172.21.15.71:0/2487184747 shutdown_connections 2024-03-04T08:57:09.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:09.669+0000 7ff290269700 1 -- 172.21.15.71:0/2487184747 wait complete. 2024-03-04T08:57:09.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:09 smithi071 ceph-mon[32789]: pgmap v344: 129 pgs: 129 active+clean; 638 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.9 KiB/s wr, 4 op/s 2024-03-04T08:57:09.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:09 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2487184747' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:57:10.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:09 smithi158 ceph-mon[37241]: pgmap v344: 129 pgs: 129 active+clean; 638 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.9 KiB/s wr, 4 op/s 2024-03-04T08:57:10.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:09 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2487184747' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:57:10.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:09 smithi150 ceph-mon[36807]: pgmap v344: 129 pgs: 129 active+clean; 638 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.9 KiB/s wr, 4 op/s 2024-03-04T08:57:10.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:09 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2487184747' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:57:10.290 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-04T08:57:10.290 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-04T08:57:10.291 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-04T08:57:10.291 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-04T08:57:10.291 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-04T08:57:10.292 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-04T08:57:10.292 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-04T08:57:10.292 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-04T08:57:10.293 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-04T08:57:10.293 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-04T08:57:10.294 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-04T08:57:10.294 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-04T08:57:10.906 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:10.952 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:12.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:11 smithi158 ceph-mon[37241]: pgmap v345: 129 pgs: 129 active+clean; 638 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.8 KiB/s wr, 4 op/s 2024-03-04T08:57:12.088 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:11 smithi071 ceph-mon[32789]: pgmap v345: 129 pgs: 129 active+clean; 638 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.8 KiB/s wr, 4 op/s 2024-03-04T08:57:12.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:11 smithi150 ceph-mon[36807]: pgmap v345: 129 pgs: 129 active+clean; 638 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.8 KiB/s wr, 4 op/s 2024-03-04T08:57:12.417 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:12.418 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:12.419 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:12.419 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:12.424 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:12.424 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:12.424 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:12.429 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:12.857 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:12.858 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:14.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:13 smithi158 ceph-mon[37241]: pgmap v346: 129 pgs: 129 active+clean; 638 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-03-04T08:57:14.055 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:13 smithi071 ceph-mon[32789]: pgmap v346: 129 pgs: 129 active+clean; 638 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-03-04T08:57:14.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.163+0000 7f0aa1536700 1 -- 172.21.15.71:0/1028261901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a9c1080f0 msgr2=0x7f0a9c1084d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:14.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.163+0000 7f0aa1536700 1 --2- 172.21.15.71:0/1028261901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a9c1080f0 0x7f0a9c1084d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0099c0 tx=0x7f0a8c009cd0 comp rx=0 tx=0).stop 2024-03-04T08:57:14.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.163+0000 7f0aa1536700 1 -- 172.21.15.71:0/1028261901 shutdown_connections 2024-03-04T08:57:14.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.163+0000 7f0aa1536700 1 --2- 172.21.15.71:0/1028261901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a9c06b1d0 0x7f0a9c10ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:14.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.163+0000 7f0aa1536700 1 --2- 172.21.15.71:0/1028261901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a9c1080f0 0x7f0a9c1084d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:14.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.163+0000 7f0aa1536700 1 --2- 172.21.15.71:0/1028261901 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0a9c06a810 0x7f0a9c06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:14.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.163+0000 7f0aa1536700 1 -- 172.21.15.71:0/1028261901 >> 172.21.15.71:0/1028261901 conn(0x7f0a9c077fd0 msgr2=0x7f0a9c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:14.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.163+0000 7f0aa1536700 1 -- 172.21.15.71:0/1028261901 shutdown_connections 2024-03-04T08:57:14.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.163+0000 7f0aa1536700 1 -- 172.21.15.71:0/1028261901 wait complete. 2024-03-04T08:57:14.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.163+0000 7f0aa1536700 1 Processor -- start 2024-03-04T08:57:14.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.163+0000 7f0aa1536700 1 -- start start 2024-03-04T08:57:14.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.164+0000 7f0aa1536700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a9c06a810 0x7f0a9c101070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:14.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.164+0000 7f0aa1536700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a9c06b1d0 0x7f0a9c1015b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:14.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.164+0000 7f0aa1536700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0a9c1080f0 0x7f0a9c101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:14.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.164+0000 7f0aa1536700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c112370 con 0x7f0a9c06a810 2024-03-04T08:57:14.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.164+0000 7f0aa1536700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c1121f0 con 0x7f0a9c06b1d0 2024-03-04T08:57:14.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.164+0000 7f0aa1536700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c1124f0 con 0x7f0a9c1080f0 2024-03-04T08:57:14.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.164+0000 7f0a9affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a9c06a810 0x7f0a9c101070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:14.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.164+0000 7f0a9affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a9c06a810 0x7f0a9c101070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47672/0 (socket says 172.21.15.71:47672) 2024-03-04T08:57:14.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.164+0000 7f0a9affd700 1 -- 172.21.15.71:0/3686682039 learned_addr learned my addr 172.21.15.71:0/3686682039 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:57:14.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.164+0000 7f0a9a7fc700 1 --2- 172.21.15.71:0/3686682039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a9c06b1d0 0x7f0a9c1015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:14.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.164+0000 7f0a9b7fe700 1 --2- 172.21.15.71:0/3686682039 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0a9c1080f0 0x7f0a9c101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:14.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.164+0000 7f0a9affd700 1 -- 172.21.15.71:0/3686682039 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0a9c1080f0 msgr2=0x7f0a9c101af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:14.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.164+0000 7f0a9affd700 1 --2- 172.21.15.71:0/3686682039 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0a9c1080f0 0x7f0a9c101af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:14.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.164+0000 7f0a9affd700 1 -- 172.21.15.71:0/3686682039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a9c06b1d0 msgr2=0x7f0a9c1015b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:14.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.164+0000 7f0a9affd700 1 --2- 172.21.15.71:0/3686682039 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0a9c06b1d0 0x7f0a9c1015b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:14.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.164+0000 7f0a9affd700 1 -- 172.21.15.71:0/3686682039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c1a6d10 con 0x7f0a9c06a810 2024-03-04T08:57:14.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.164+0000 7f0a9affd700 1 --2- 172.21.15.71:0/3686682039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a9c06a810 0x7f0a9c101070 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0059d0 tx=0x7f0a8c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:14.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.164+0000 7f0a83fff700 1 -- 172.21.15.71:0/3686682039 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c027070 con 0x7f0a9c06a810 2024-03-04T08:57:14.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.164+0000 7f0a83fff700 1 -- 172.21.15.71:0/3686682039 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a8c003680 con 0x7f0a9c06a810 2024-03-04T08:57:14.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.164+0000 7f0a83fff700 1 -- 172.21.15.71:0/3686682039 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c017650 con 0x7f0a9c06a810 2024-03-04T08:57:14.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.164+0000 7f0aa1536700 1 -- 172.21.15.71:0/3686682039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9c1a6fa0 con 0x7f0a9c06a810 2024-03-04T08:57:14.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.165+0000 7f0aa1536700 1 -- 172.21.15.71:0/3686682039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9c1a73f0 con 0x7f0a9c06a810 2024-03-04T08:57:14.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.165+0000 7f0aa1536700 1 -- 172.21.15.71:0/3686682039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0a9c062790 con 0x7f0a9c06a810 2024-03-04T08:57:14.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.166+0000 7f0a83fff700 1 -- 172.21.15.71:0/3686682039 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f0a8c02a070 con 0x7f0a9c06a810 2024-03-04T08:57:14.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.167+0000 7f0a83fff700 1 --2- 172.21.15.71:0/3686682039 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f0a840a1f20 0x7f0a840a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:14.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.167+0000 7f0a9a7fc700 1 --2- 172.21.15.71:0/3686682039 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f0a840a1f20 0x7f0a840a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:14.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.167+0000 7f0a83fff700 1 -- 172.21.15.71:0/3686682039 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f0a8c0cf380 con 0x7f0a9c06a810 2024-03-04T08:57:14.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.167+0000 7f0a83fff700 1 --2- 172.21.15.71:0/3686682039 >> [v2:172.21.15.150:6800/3383948110,v1:172.21.15.150:6801/3383948110] conn(0x7f0a840acbe0 0x7f0a840af020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:14.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.167+0000 7f0a83fff700 1 -- 172.21.15.71:0/3686682039 --> [v2:172.21.15.150:6800/3383948110,v1:172.21.15.150:6801/3383948110] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0a8c010ea0 con 0x7f0a840acbe0 2024-03-04T08:57:14.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.167+0000 7f0a83fff700 1 -- 172.21.15.71:0/3686682039 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a8c0cf7d0 con 0x7f0a9c06a810 2024-03-04T08:57:14.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.168+0000 7f0a9b7fe700 1 --2- 172.21.15.71:0/3686682039 >> [v2:172.21.15.150:6800/3383948110,v1:172.21.15.150:6801/3383948110] conn(0x7f0a840acbe0 0x7f0a840af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:14.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.168+0000 7f0a9a7fc700 1 --2- 172.21.15.71:0/3686682039 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f0a840a1f20 0x7f0a840a43e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c0ff5a0 tx=0x7f0a9000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:14.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.168+0000 7f0a9b7fe700 1 --2- 172.21.15.71:0/3686682039 >> [v2:172.21.15.150:6800/3383948110,v1:172.21.15.150:6801/3383948110] conn(0x7f0a840acbe0 0x7f0a840af020 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:14.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.169+0000 7f0a83fff700 1 -- 172.21.15.71:0/3686682039 <== osd.1 v2:172.21.15.150:6800/3383948110 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0a8c010ea0 con 0x7f0a840acbe0 2024-03-04T08:57:14.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.189+0000 7f0aa1536700 1 -- 172.21.15.71:0/3686682039 --> [v2:172.21.15.150:6800/3383948110,v1:172.21.15.150:6801/3383948110] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0a9c062c10 con 0x7f0a840acbe0 2024-03-04T08:57:14.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.189+0000 7f0a83fff700 1 -- 172.21.15.71:0/3686682039 <== osd.1 v2:172.21.15.150:6800/3383948110 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0a9c062c10 con 0x7f0a840acbe0 2024-03-04T08:57:14.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.190+0000 7f0aa1536700 1 -- 172.21.15.71:0/3686682039 >> [v2:172.21.15.150:6800/3383948110,v1:172.21.15.150:6801/3383948110] conn(0x7f0a840acbe0 msgr2=0x7f0a840af020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:14.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.190+0000 7f0aa1536700 1 --2- 172.21.15.71:0/3686682039 >> [v2:172.21.15.150:6800/3383948110,v1:172.21.15.150:6801/3383948110] conn(0x7f0a840acbe0 0x7f0a840af020 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:14.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.190+0000 7f0aa1536700 1 -- 172.21.15.71:0/3686682039 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f0a840a1f20 msgr2=0x7f0a840a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:14.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.190+0000 7f0aa1536700 1 --2- 172.21.15.71:0/3686682039 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f0a840a1f20 0x7f0a840a43e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c0ff5a0 tx=0x7f0a9000a300 comp rx=0 tx=0).stop 2024-03-04T08:57:14.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.190+0000 7f0aa1536700 1 -- 172.21.15.71:0/3686682039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a9c06a810 msgr2=0x7f0a9c101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:14.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.190+0000 7f0aa1536700 1 --2- 172.21.15.71:0/3686682039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a9c06a810 0x7f0a9c101070 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0059d0 tx=0x7f0a8c00fa00 comp rx=0 tx=0).stop 2024-03-04T08:57:14.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.190+0000 7f0a9b7fe700 1 -- 172.21.15.71:0/3686682039 reap_dead start 2024-03-04T08:57:14.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.190+0000 7f0aa1536700 1 -- 172.21.15.71:0/3686682039 shutdown_connections 2024-03-04T08:57:14.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.190+0000 7f0aa1536700 1 -- 172.21.15.71:0/3686682039 >> 172.21.15.71:0/3686682039 conn(0x7f0a9c077fd0 msgr2=0x7f0a9c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:14.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.190+0000 7f0aa1536700 1 -- 172.21.15.71:0/3686682039 shutdown_connections 2024-03-04T08:57:14.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:14.190+0000 7f0aa1536700 1 -- 172.21.15.71:0/3686682039 wait complete. 2024-03-04T08:57:14.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:13 smithi150 ceph-mon[36807]: pgmap v346: 129 pgs: 129 active+clean; 638 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-03-04T08:57:16.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:15 smithi158 ceph-mon[37241]: pgmap v347: 129 pgs: 129 active+clean; 638 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-03-04T08:57:16.080 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:15 smithi071 ceph-mon[32789]: pgmap v347: 129 pgs: 129 active+clean; 638 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-03-04T08:57:16.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:15 smithi150 ceph-mon[36807]: pgmap v347: 129 pgs: 129 active+clean; 638 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-03-04T08:57:18.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:17 smithi158 ceph-mon[37241]: pgmap v348: 129 pgs: 129 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.9 KiB/s wr, 3 op/s 2024-03-04T08:57:18.080 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:17 smithi071 ceph-mon[32789]: pgmap v348: 129 pgs: 129 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.9 KiB/s wr, 3 op/s 2024-03-04T08:57:18.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:17 smithi150 ceph-mon[36807]: pgmap v348: 129 pgs: 129 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.9 KiB/s wr, 3 op/s 2024-03-04T08:57:20.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:19 smithi158 ceph-mon[37241]: pgmap v349: 129 pgs: 129 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-03-04T08:57:20.127 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:19 smithi071 ceph-mon[32789]: pgmap v349: 129 pgs: 129 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-03-04T08:57:20.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:19 smithi150 ceph-mon[36807]: pgmap v349: 129 pgs: 129 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-03-04T08:57:21.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.497+0000 7fb14cebb700 1 -- 172.21.15.71:0/61323243 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1481080f0 msgr2=0x7fb1481084d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:21.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.497+0000 7fb14cebb700 1 --2- 172.21.15.71:0/61323243 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1481080f0 0x7fb1481084d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb1380099c0 tx=0x7fb138009cd0 comp rx=0 tx=0).stop 2024-03-04T08:57:21.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.497+0000 7fb14cebb700 1 -- 172.21.15.71:0/61323243 shutdown_connections 2024-03-04T08:57:21.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.497+0000 7fb14cebb700 1 --2- 172.21.15.71:0/61323243 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb148102ad0 0x7fb14810ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:21.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.497+0000 7fb14cebb700 1 --2- 172.21.15.71:0/61323243 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1481080f0 0x7fb1481084d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:21.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.497+0000 7fb14cebb700 1 --2- 172.21.15.71:0/61323243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb148102110 0x7fb148102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:21.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.497+0000 7fb14cebb700 1 -- 172.21.15.71:0/61323243 >> 172.21.15.71:0/61323243 conn(0x7fb1480791f0 msgr2=0x7fb148079600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:21.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.497+0000 7fb14cebb700 1 -- 172.21.15.71:0/61323243 shutdown_connections 2024-03-04T08:57:21.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.497+0000 7fb14cebb700 1 -- 172.21.15.71:0/61323243 wait complete. 2024-03-04T08:57:21.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.497+0000 7fb14cebb700 1 Processor -- start 2024-03-04T08:57:21.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.498+0000 7fb14cebb700 1 -- start start 2024-03-04T08:57:21.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.498+0000 7fb14cebb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb148102110 0x7fb14819e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:21.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.498+0000 7fb14cebb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb148102ad0 0x7fb14819e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:21.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.498+0000 7fb14cebb700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb1481080f0 0x7fb148198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:21.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.498+0000 7fb14cebb700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb148112370 con 0x7fb148102ad0 2024-03-04T08:57:21.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.498+0000 7fb14cebb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1481121f0 con 0x7fb148102110 2024-03-04T08:57:21.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.498+0000 7fb14cebb700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1481124f0 con 0x7fb1481080f0 2024-03-04T08:57:21.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.498+0000 7fb145d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb148102ad0 0x7fb14819e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:21.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.498+0000 7fb145d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb148102ad0 0x7fb14819e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47302/0 (socket says 172.21.15.71:47302) 2024-03-04T08:57:21.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.498+0000 7fb145d9b700 1 -- 172.21.15.71:0/3643701370 learned_addr learned my addr 172.21.15.71:0/3643701370 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:57:21.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.498+0000 7fb14659c700 1 --2- 172.21.15.71:0/3643701370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb148102110 0x7fb14819e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:21.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.498+0000 7fb146d9d700 1 --2- 172.21.15.71:0/3643701370 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb1481080f0 0x7fb148198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:21.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.498+0000 7fb145d9b700 1 -- 172.21.15.71:0/3643701370 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb1481080f0 msgr2=0x7fb148198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:21.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.498+0000 7fb145d9b700 1 --2- 172.21.15.71:0/3643701370 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb1481080f0 0x7fb148198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:21.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.498+0000 7fb145d9b700 1 -- 172.21.15.71:0/3643701370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb148102110 msgr2=0x7fb14819e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:21.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.498+0000 7fb145d9b700 1 --2- 172.21.15.71:0/3643701370 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb148102110 0x7fb14819e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:21.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.498+0000 7fb145d9b700 1 -- 172.21.15.71:0/3643701370 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb148198d70 con 0x7fb148102ad0 2024-03-04T08:57:21.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.498+0000 7fb145d9b700 1 --2- 172.21.15.71:0/3643701370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb148102ad0 0x7fb14819e990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00d980 tx=0x7fb13c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:21.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.499+0000 7fb1377fe700 1 -- 172.21.15.71:0/3643701370 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c016070 con 0x7fb148102ad0 2024-03-04T08:57:21.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.499+0000 7fb14cebb700 1 -- 172.21.15.71:0/3643701370 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb148199060 con 0x7fb148102ad0 2024-03-04T08:57:21.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.499+0000 7fb1377fe700 1 -- 172.21.15.71:0/3643701370 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb13c004d10 con 0x7fb148102ad0 2024-03-04T08:57:21.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.499+0000 7fb1377fe700 1 -- 172.21.15.71:0/3643701370 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c0055f0 con 0x7fb148102ad0 2024-03-04T08:57:21.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.499+0000 7fb14cebb700 1 -- 172.21.15.71:0/3643701370 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1481a5320 con 0x7fb148102ad0 2024-03-04T08:57:21.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.500+0000 7fb14cebb700 1 -- 172.21.15.71:0/3643701370 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb148062850 con 0x7fb148102ad0 2024-03-04T08:57:21.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.501+0000 7fb1377fe700 1 -- 172.21.15.71:0/3643701370 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fb13c004750 con 0x7fb148102ad0 2024-03-04T08:57:21.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.502+0000 7fb1377fe700 1 --2- 172.21.15.71:0/3643701370 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb1300a1ea0 0x7fb1300a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:21.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.502+0000 7fb14659c700 1 --2- 172.21.15.71:0/3643701370 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb1300a1ea0 0x7fb1300a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:21.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.502+0000 7fb1377fe700 1 -- 172.21.15.71:0/3643701370 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb13c0ce640 con 0x7fb148102ad0 2024-03-04T08:57:21.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.502+0000 7fb1377fe700 1 --2- 172.21.15.71:0/3643701370 >> [v2:172.21.15.158:6800/586944144,v1:172.21.15.158:6801/586944144] conn(0x7fb1300acb60 0x7fb1300aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:21.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.502+0000 7fb1377fe700 1 -- 172.21.15.71:0/3643701370 --> [v2:172.21.15.158:6800/586944144,v1:172.21.15.158:6801/586944144] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb13c015ec0 con 0x7fb1300acb60 2024-03-04T08:57:21.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.502+0000 7fb1377fe700 1 -- 172.21.15.71:0/3643701370 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb13c0cea80 con 0x7fb148102ad0 2024-03-04T08:57:21.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.502+0000 7fb14659c700 1 --2- 172.21.15.71:0/3643701370 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb1300a1ea0 0x7fb1300a4360 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb138000c00 tx=0x7fb138005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:21.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.503+0000 7fb146d9d700 1 --2- 172.21.15.71:0/3643701370 >> [v2:172.21.15.158:6800/586944144,v1:172.21.15.158:6801/586944144] conn(0x7fb1300acb60 0x7fb1300aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:21.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.503+0000 7fb146d9d700 1 --2- 172.21.15.71:0/3643701370 >> [v2:172.21.15.158:6800/586944144,v1:172.21.15.158:6801/586944144] conn(0x7fb1300acb60 0x7fb1300aefa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:21.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.504+0000 7fb1377fe700 1 -- 172.21.15.71:0/3643701370 <== osd.2 v2:172.21.15.158:6800/586944144 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb13c015ec0 con 0x7fb1300acb60 2024-03-04T08:57:21.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.524+0000 7fb14cebb700 1 -- 172.21.15.71:0/3643701370 --> [v2:172.21.15.158:6800/586944144,v1:172.21.15.158:6801/586944144] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb148062cd0 con 0x7fb1300acb60 2024-03-04T08:57:21.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.525+0000 7fb1377fe700 1 -- 172.21.15.71:0/3643701370 <== osd.2 v2:172.21.15.158:6800/586944144 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb148062cd0 con 0x7fb1300acb60 2024-03-04T08:57:21.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.525+0000 7fb14cebb700 1 -- 172.21.15.71:0/3643701370 >> [v2:172.21.15.158:6800/586944144,v1:172.21.15.158:6801/586944144] conn(0x7fb1300acb60 msgr2=0x7fb1300aefa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:21.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.525+0000 7fb14cebb700 1 --2- 172.21.15.71:0/3643701370 >> [v2:172.21.15.158:6800/586944144,v1:172.21.15.158:6801/586944144] conn(0x7fb1300acb60 0x7fb1300aefa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:21.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.525+0000 7fb14cebb700 1 -- 172.21.15.71:0/3643701370 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb1300a1ea0 msgr2=0x7fb1300a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:21.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.525+0000 7fb14cebb700 1 --2- 172.21.15.71:0/3643701370 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb1300a1ea0 0x7fb1300a4360 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb138000c00 tx=0x7fb138005fb0 comp rx=0 tx=0).stop 2024-03-04T08:57:21.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.525+0000 7fb14cebb700 1 -- 172.21.15.71:0/3643701370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb148102ad0 msgr2=0x7fb14819e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:21.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.525+0000 7fb14cebb700 1 --2- 172.21.15.71:0/3643701370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb148102ad0 0x7fb14819e990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00d980 tx=0x7fb13c00dc90 comp rx=0 tx=0).stop 2024-03-04T08:57:21.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.525+0000 7fb146d9d700 1 -- 172.21.15.71:0/3643701370 reap_dead start 2024-03-04T08:57:21.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.525+0000 7fb14cebb700 1 -- 172.21.15.71:0/3643701370 shutdown_connections 2024-03-04T08:57:21.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.525+0000 7fb14cebb700 1 -- 172.21.15.71:0/3643701370 >> 172.21.15.71:0/3643701370 conn(0x7fb1480791f0 msgr2=0x7fb148100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:21.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.526+0000 7fb14cebb700 1 -- 172.21.15.71:0/3643701370 shutdown_connections 2024-03-04T08:57:21.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:21.526+0000 7fb14cebb700 1 -- 172.21.15.71:0/3643701370 wait complete. 2024-03-04T08:57:22.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:21 smithi158 ceph-mon[37241]: pgmap v350: 129 pgs: 129 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-03-04T08:57:22.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:21 smithi071 ceph-mon[32789]: pgmap v350: 129 pgs: 129 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-03-04T08:57:22.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:21 smithi150 ceph-mon[36807]: pgmap v350: 129 pgs: 129 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-03-04T08:57:23.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:22 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-04T08:57:23.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:22 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-03-04T08:57:23.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:22 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-03-04T08:57:23.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:22 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-03-04T08:57:23.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:22 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-03-04T08:57:23.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:22 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-03-04T08:57:23.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:22 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]: dispatch 2024-03-04T08:57:23.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:22 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]: dispatch 2024-03-04T08:57:23.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:22 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-03-04T08:57:23.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:22 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-03-04T08:57:23.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:22 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-04T08:57:23.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:22 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-03-04T08:57:23.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:22 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-03-04T08:57:23.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:22 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-03-04T08:57:23.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:22 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-03-04T08:57:23.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:22 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-03-04T08:57:23.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:22 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]: dispatch 2024-03-04T08:57:23.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:22 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]: dispatch 2024-03-04T08:57:23.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:22 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-03-04T08:57:23.175 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:22 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-03-04T08:57:23.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:22 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-04T08:57:23.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:22 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-03-04T08:57:23.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:22 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-03-04T08:57:23.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:22 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-03-04T08:57:23.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:22 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-03-04T08:57:23.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:22 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-03-04T08:57:23.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:22 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]: dispatch 2024-03-04T08:57:23.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:22 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]: dispatch 2024-03-04T08:57:23.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:22 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-03-04T08:57:23.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:22 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-03-04T08:57:23.434 INFO:teuthology.orchestra.run.smithi071.stdout:55834574932 2024-03-04T08:57:23.434 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-04T08:57:23.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:23 smithi071 ceph-mon[32789]: pgmap v351: 129 pgs: 129 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 2 op/s 2024-03-04T08:57:23.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:23 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-04T08:57:23.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:23 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-03-04T08:57:23.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:23 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-03-04T08:57:23.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:23 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-03-04T08:57:23.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:23 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-03-04T08:57:23.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:23 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-03-04T08:57:23.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:23 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-03-04T08:57:23.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:23 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]': finished 2024-03-04T08:57:23.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:23 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-03-04T08:57:23.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:23 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-03-04T08:57:23.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:23 smithi071 ceph-mon[32789]: osdmap e92: 12 total, 12 up, 12 in 2024-03-04T08:57:24.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:23 smithi150 ceph-mon[36807]: pgmap v351: 129 pgs: 129 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 2 op/s 2024-03-04T08:57:24.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:23 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-04T08:57:24.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:23 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-03-04T08:57:24.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:23 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-03-04T08:57:24.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:23 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-03-04T08:57:24.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:23 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-03-04T08:57:24.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:23 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-03-04T08:57:24.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:23 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-03-04T08:57:24.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:23 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]': finished 2024-03-04T08:57:24.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:23 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-03-04T08:57:24.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:23 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-03-04T08:57:24.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:23 smithi150 ceph-mon[36807]: osdmap e92: 12 total, 12 up, 12 in 2024-03-04T08:57:24.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:23 smithi158 ceph-mon[37241]: pgmap v351: 129 pgs: 129 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 2 op/s 2024-03-04T08:57:24.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:23 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-04T08:57:24.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:23 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-03-04T08:57:24.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:23 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-03-04T08:57:24.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:23 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-03-04T08:57:24.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:23 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-03-04T08:57:24.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:23 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-03-04T08:57:24.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:23 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-03-04T08:57:24.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:23 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]': finished 2024-03-04T08:57:24.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:23 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-03-04T08:57:24.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:23 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-03-04T08:57:24.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:23 smithi158 ceph-mon[37241]: osdmap e92: 12 total, 12 up, 12 in 2024-03-04T08:57:24.993 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:24 smithi071 ceph-mon[32789]: osdmap e93: 12 total, 12 up, 12 in 2024-03-04T08:57:25.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:24 smithi150 ceph-mon[36807]: osdmap e93: 12 total, 12 up, 12 in 2024-03-04T08:57:25.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:24 smithi158 ceph-mon[37241]: osdmap e93: 12 total, 12 up, 12 in 2024-03-04T08:57:25.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.706+0000 7fdaa7cfa700 1 -- 172.21.15.71:0/470163531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaa0107000 msgr2=0x7fdaa01073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:25.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.706+0000 7fdaa7cfa700 1 --2- 172.21.15.71:0/470163531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaa0107000 0x7fdaa01073e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fda9c009990 tx=0x7fda9c009ca0 comp rx=0 tx=0).stop 2024-03-04T08:57:25.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.706+0000 7fdaa7cfa700 1 -- 172.21.15.71:0/470163531 shutdown_connections 2024-03-04T08:57:25.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.706+0000 7fdaa7cfa700 1 --2- 172.21.15.71:0/470163531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa0101960 0x7fdaa010de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:25.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.706+0000 7fdaa7cfa700 1 --2- 172.21.15.71:0/470163531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaa0107000 0x7fdaa01073e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:25.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.706+0000 7fdaa7cfa700 1 --2- 172.21.15.71:0/470163531 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdaa0100fc0 0x7fdaa0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:25.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.706+0000 7fdaa7cfa700 1 -- 172.21.15.71:0/470163531 >> 172.21.15.71:0/470163531 conn(0x7fdaa0077ef0 msgr2=0x7fdaa0078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:25.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.706+0000 7fdaa7cfa700 1 -- 172.21.15.71:0/470163531 shutdown_connections 2024-03-04T08:57:25.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.706+0000 7fdaa7cfa700 1 -- 172.21.15.71:0/470163531 wait complete. 2024-03-04T08:57:25.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.706+0000 7fdaa7cfa700 1 Processor -- start 2024-03-04T08:57:25.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.706+0000 7fdaa7cfa700 1 -- start start 2024-03-04T08:57:25.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.706+0000 7fdaa7cfa700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdaa0100fc0 0x7fdaa0199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:25.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.707+0000 7fdaa7cfa700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaa0101960 0x7fdaa0199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:25.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.707+0000 7fdaa7cfa700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa0107000 0x7fdaa01a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:25.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.707+0000 7fdaa7cfa700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa0111240 con 0x7fdaa0101960 2024-03-04T08:57:25.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.707+0000 7fdaa7cfa700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa01110c0 con 0x7fdaa0107000 2024-03-04T08:57:25.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.707+0000 7fdaa7cfa700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa01113c0 con 0x7fdaa0100fc0 2024-03-04T08:57:25.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.707+0000 7fdaa5295700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaa0101960 0x7fdaa0199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:25.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.707+0000 7fdaa5295700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaa0101960 0x7fdaa0199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45326/0 (socket says 172.21.15.71:45326) 2024-03-04T08:57:25.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.707+0000 7fdaa5295700 1 -- 172.21.15.71:0/1604460165 learned_addr learned my addr 172.21.15.71:0/1604460165 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:57:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.707+0000 7fdaa6297700 1 --2- 172.21.15.71:0/1604460165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa0107000 0x7fdaa01a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.707+0000 7fdaa5a96700 1 --2- 172.21.15.71:0/1604460165 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdaa0100fc0 0x7fdaa0199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.707+0000 7fdaa5295700 1 -- 172.21.15.71:0/1604460165 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdaa0100fc0 msgr2=0x7fdaa0199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.707+0000 7fdaa5295700 1 --2- 172.21.15.71:0/1604460165 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdaa0100fc0 0x7fdaa0199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.707+0000 7fdaa5295700 1 -- 172.21.15.71:0/1604460165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa0107000 msgr2=0x7fdaa01a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.707+0000 7fdaa5295700 1 --2- 172.21.15.71:0/1604460165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa0107000 0x7fdaa01a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.707+0000 7fdaa5295700 1 -- 172.21.15.71:0/1604460165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa01a4c20 con 0x7fdaa0101960 2024-03-04T08:57:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.707+0000 7fdaa6297700 1 --2- 172.21.15.71:0/1604460165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdaa0107000 0x7fdaa01a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:57:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.707+0000 7fdaa5295700 1 --2- 172.21.15.71:0/1604460165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaa0101960 0x7fdaa0199ba0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fda9000d950 tx=0x7fda9000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.707+0000 7fda96ffd700 1 -- 172.21.15.71:0/1604460165 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90016070 con 0x7fdaa0101960 2024-03-04T08:57:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.707+0000 7fda96ffd700 1 -- 172.21.15.71:0/1604460165 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda9000fd70 con 0x7fdaa0101960 2024-03-04T08:57:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.707+0000 7fdaa7cfa700 1 -- 172.21.15.71:0/1604460165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa01a4e50 con 0x7fdaa0101960 2024-03-04T08:57:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.707+0000 7fda96ffd700 1 -- 172.21.15.71:0/1604460165 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9000e610 con 0x7fdaa0101960 2024-03-04T08:57:25.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.707+0000 7fdaa7cfa700 1 -- 172.21.15.71:0/1604460165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa01a5300 con 0x7fdaa0101960 2024-03-04T08:57:25.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.708+0000 7fdaa7cfa700 1 -- 172.21.15.71:0/1604460165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdaa0062790 con 0x7fdaa0101960 2024-03-04T08:57:25.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.709+0000 7fda96ffd700 1 -- 172.21.15.71:0/1604460165 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fda90019070 con 0x7fdaa0101960 2024-03-04T08:57:25.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.710+0000 7fda96ffd700 1 --2- 172.21.15.71:0/1604460165 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fda8c0a1ea0 0x7fda8c0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:25.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.710+0000 7fdaa5a96700 1 --2- 172.21.15.71:0/1604460165 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fda8c0a1ea0 0x7fda8c0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:25.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.710+0000 7fda96ffd700 1 -- 172.21.15.71:0/1604460165 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fda900ce2a0 con 0x7fdaa0101960 2024-03-04T08:57:25.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.710+0000 7fda96ffd700 1 --2- 172.21.15.71:0/1604460165 >> [v2:172.21.15.158:6824/3332083865,v1:172.21.15.158:6825/3332083865] conn(0x7fda8c0ad000 0x7fda8c0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:25.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.710+0000 7fda96ffd700 1 -- 172.21.15.71:0/1604460165 --> [v2:172.21.15.158:6824/3332083865,v1:172.21.15.158:6825/3332083865] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fda90014e80 con 0x7fda8c0ad000 2024-03-04T08:57:25.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.710+0000 7fda96ffd700 1 -- 172.21.15.71:0/1604460165 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda900ce6f0 con 0x7fdaa0101960 2024-03-04T08:57:25.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.710+0000 7fdaa5a96700 1 --2- 172.21.15.71:0/1604460165 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fda8c0a1ea0 0x7fda8c0a4360 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00b2e0 tx=0x7fda9c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:25.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.710+0000 7fdaa6297700 1 --2- 172.21.15.71:0/1604460165 >> [v2:172.21.15.158:6824/3332083865,v1:172.21.15.158:6825/3332083865] conn(0x7fda8c0ad000 0x7fda8c0af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:25.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.711+0000 7fdaa6297700 1 --2- 172.21.15.71:0/1604460165 >> [v2:172.21.15.158:6824/3332083865,v1:172.21.15.158:6825/3332083865] conn(0x7fda8c0ad000 0x7fda8c0af4a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:25.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.712+0000 7fda96ffd700 1 -- 172.21.15.71:0/1604460165 <== osd.11 v2:172.21.15.158:6824/3332083865 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fda90014e80 con 0x7fda8c0ad000 2024-03-04T08:57:25.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.730+0000 7fdaa7cfa700 1 -- 172.21.15.71:0/1604460165 --> [v2:172.21.15.158:6824/3332083865,v1:172.21.15.158:6825/3332083865] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdaa0062c10 con 0x7fda8c0ad000 2024-03-04T08:57:25.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.731+0000 7fda96ffd700 1 -- 172.21.15.71:0/1604460165 <== osd.11 v2:172.21.15.158:6824/3332083865 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fdaa0062c10 con 0x7fda8c0ad000 2024-03-04T08:57:25.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.732+0000 7fdaa7cfa700 1 -- 172.21.15.71:0/1604460165 >> [v2:172.21.15.158:6824/3332083865,v1:172.21.15.158:6825/3332083865] conn(0x7fda8c0ad000 msgr2=0x7fda8c0af4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:25.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.732+0000 7fdaa7cfa700 1 --2- 172.21.15.71:0/1604460165 >> [v2:172.21.15.158:6824/3332083865,v1:172.21.15.158:6825/3332083865] conn(0x7fda8c0ad000 0x7fda8c0af4a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:25.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.732+0000 7fdaa7cfa700 1 -- 172.21.15.71:0/1604460165 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fda8c0a1ea0 msgr2=0x7fda8c0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:25.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.732+0000 7fdaa7cfa700 1 --2- 172.21.15.71:0/1604460165 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fda8c0a1ea0 0x7fda8c0a4360 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00b2e0 tx=0x7fda9c005960 comp rx=0 tx=0).stop 2024-03-04T08:57:25.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.732+0000 7fdaa7cfa700 1 -- 172.21.15.71:0/1604460165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaa0101960 msgr2=0x7fdaa0199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:25.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.732+0000 7fdaa7cfa700 1 --2- 172.21.15.71:0/1604460165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaa0101960 0x7fdaa0199ba0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fda9000d950 tx=0x7fda9000dc60 comp rx=0 tx=0).stop 2024-03-04T08:57:25.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.732+0000 7fdaa6297700 1 -- 172.21.15.71:0/1604460165 reap_dead start 2024-03-04T08:57:25.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.732+0000 7fdaa7cfa700 1 -- 172.21.15.71:0/1604460165 shutdown_connections 2024-03-04T08:57:25.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.732+0000 7fdaa7cfa700 1 -- 172.21.15.71:0/1604460165 >> 172.21.15.71:0/1604460165 conn(0x7fdaa0077ef0 msgr2=0x7fdaa00ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:25.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.732+0000 7fdaa7cfa700 1 -- 172.21.15.71:0/1604460165 shutdown_connections 2024-03-04T08:57:25.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:25.732+0000 7fdaa7cfa700 1 -- 172.21.15.71:0/1604460165 wait complete. 2024-03-04T08:57:25.897 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:25 smithi071 ceph-mon[32789]: pgmap v354: 129 pgs: 2 peering, 127 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s 2024-03-04T08:57:26.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:25 smithi150 ceph-mon[36807]: pgmap v354: 129 pgs: 2 peering, 127 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s 2024-03-04T08:57:26.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:25 smithi158 ceph-mon[37241]: pgmap v354: 129 pgs: 2 peering, 127 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s 2024-03-04T08:57:26.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.494+0000 7f26d8f39700 1 -- 172.21.15.71:0/2640875644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26d41080f0 msgr2=0x7f26d41084d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:26.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.494+0000 7f26d8f39700 1 --2- 172.21.15.71:0/2640875644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26d41080f0 0x7f26d41084d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f26c40099c0 tx=0x7f26c4009cd0 comp rx=0 tx=0).stop 2024-03-04T08:57:26.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.494+0000 7f26d8f39700 1 -- 172.21.15.71:0/2640875644 shutdown_connections 2024-03-04T08:57:26.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.494+0000 7f26d8f39700 1 --2- 172.21.15.71:0/2640875644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26d4102ad0 0x7f26d410ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:26.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.494+0000 7f26d8f39700 1 --2- 172.21.15.71:0/2640875644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26d41080f0 0x7f26d41084d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:26.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.494+0000 7f26d8f39700 1 --2- 172.21.15.71:0/2640875644 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26d4102110 0x7f26d4102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:26.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.494+0000 7f26d8f39700 1 -- 172.21.15.71:0/2640875644 >> 172.21.15.71:0/2640875644 conn(0x7f26d40791f0 msgr2=0x7f26d4079600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:26.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.494+0000 7f26d8f39700 1 -- 172.21.15.71:0/2640875644 shutdown_connections 2024-03-04T08:57:26.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.494+0000 7f26d8f39700 1 -- 172.21.15.71:0/2640875644 wait complete. 2024-03-04T08:57:26.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.495+0000 7f26d8f39700 1 Processor -- start 2024-03-04T08:57:26.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.495+0000 7f26d8f39700 1 -- start start 2024-03-04T08:57:26.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.495+0000 7f26d8f39700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26d4102110 0x7f26d40743c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:26.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.495+0000 7f26d8f39700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26d4102ad0 0x7f26d4074900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:26.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.495+0000 7f26d8f39700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26d41080f0 0x7f26d4074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:26.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.495+0000 7f26d8f39700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d4112370 con 0x7f26d41080f0 2024-03-04T08:57:26.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.495+0000 7f26d8f39700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d41121f0 con 0x7f26d4102110 2024-03-04T08:57:26.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.495+0000 7f26d8f39700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d41124f0 con 0x7f26d4102ad0 2024-03-04T08:57:26.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.496+0000 7f26d2d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26d41080f0 0x7f26d4074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:26.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.496+0000 7f26d2d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26d41080f0 0x7f26d4074e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45330/0 (socket says 172.21.15.71:45330) 2024-03-04T08:57:26.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.496+0000 7f26d259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26d4102110 0x7f26d40743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:26.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.496+0000 7f26d2d9d700 1 -- 172.21.15.71:0/13657704 learned_addr learned my addr 172.21.15.71:0/13657704 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:57:26.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.496+0000 7f26d1d9b700 1 --2- 172.21.15.71:0/13657704 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26d4102ad0 0x7f26d4074900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:26.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.496+0000 7f26d2d9d700 1 -- 172.21.15.71:0/13657704 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26d4102ad0 msgr2=0x7f26d4074900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:26.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.496+0000 7f26d2d9d700 1 --2- 172.21.15.71:0/13657704 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f26d4102ad0 0x7f26d4074900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:26.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.496+0000 7f26d2d9d700 1 -- 172.21.15.71:0/13657704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26d4102110 msgr2=0x7f26d40743c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:26.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.496+0000 7f26d2d9d700 1 --2- 172.21.15.71:0/13657704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26d4102110 0x7f26d40743c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:26.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.496+0000 7f26d2d9d700 1 -- 172.21.15.71:0/13657704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d41a6cc0 con 0x7f26d41080f0 2024-03-04T08:57:26.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.496+0000 7f26d259c700 1 --2- 172.21.15.71:0/13657704 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26d4102110 0x7f26d40743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:57:26.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.496+0000 7f26d2d9d700 1 --2- 172.21.15.71:0/13657704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26d41080f0 0x7f26d4074e40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00ef30 tx=0x7f26cc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:26.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.496+0000 7f26c37fe700 1 -- 172.21.15.71:0/13657704 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc0092e0 con 0x7f26d41080f0 2024-03-04T08:57:26.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.496+0000 7f26c37fe700 1 -- 172.21.15.71:0/13657704 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26cc004bb0 con 0x7f26d41080f0 2024-03-04T08:57:26.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.496+0000 7f26c37fe700 1 -- 172.21.15.71:0/13657704 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc004280 con 0x7f26d41080f0 2024-03-04T08:57:26.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.496+0000 7f26d8f39700 1 -- 172.21.15.71:0/13657704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d41a6f50 con 0x7f26d41080f0 2024-03-04T08:57:26.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.496+0000 7f26d8f39700 1 -- 172.21.15.71:0/13657704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d41a7460 con 0x7f26d41080f0 2024-03-04T08:57:26.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.497+0000 7f26d8f39700 1 -- 172.21.15.71:0/13657704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f26d4062850 con 0x7f26d41080f0 2024-03-04T08:57:26.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.498+0000 7f26c37fe700 1 -- 172.21.15.71:0/13657704 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f26cc01b030 con 0x7f26d41080f0 2024-03-04T08:57:26.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.499+0000 7f26c37fe700 1 --2- 172.21.15.71:0/13657704 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f26bc0a1f20 0x7f26bc0a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:26.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.499+0000 7f26d259c700 1 --2- 172.21.15.71:0/13657704 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f26bc0a1f20 0x7f26bc0a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:26.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.499+0000 7f26c37fe700 1 -- 172.21.15.71:0/13657704 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f26cc0ce5e0 con 0x7f26d41080f0 2024-03-04T08:57:26.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.499+0000 7f26c37fe700 1 --2- 172.21.15.71:0/13657704 >> [v2:172.21.15.71:6802/1173641530,v1:172.21.15.71:6803/1173641530] conn(0x7f26bc0ad080 0x7f26bc0af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:26.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.499+0000 7f26c37fe700 1 -- 172.21.15.71:0/13657704 --> [v2:172.21.15.71:6802/1173641530,v1:172.21.15.71:6803/1173641530] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f26cc013ec0 con 0x7f26bc0ad080 2024-03-04T08:57:26.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.499+0000 7f26c37fe700 1 -- 172.21.15.71:0/13657704 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26cc0cea30 con 0x7f26d41080f0 2024-03-04T08:57:26.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.499+0000 7f26d1d9b700 1 --2- 172.21.15.71:0/13657704 >> [v2:172.21.15.71:6802/1173641530,v1:172.21.15.71:6803/1173641530] conn(0x7f26bc0ad080 0x7f26bc0af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:26.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.499+0000 7f26d259c700 1 --2- 172.21.15.71:0/13657704 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f26bc0a1f20 0x7f26bc0a43e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f26c4000c00 tx=0x7f26c4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:26.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.500+0000 7f26d1d9b700 1 --2- 172.21.15.71:0/13657704 >> [v2:172.21.15.71:6802/1173641530,v1:172.21.15.71:6803/1173641530] conn(0x7f26bc0ad080 0x7f26bc0af520 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:26.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.500+0000 7f26c37fe700 1 -- 172.21.15.71:0/13657704 <== osd.0 v2:172.21.15.71:6802/1173641530 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f26cc013ec0 con 0x7f26bc0ad080 2024-03-04T08:57:26.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.517+0000 7f26d8f39700 1 -- 172.21.15.71:0/13657704 --> [v2:172.21.15.71:6802/1173641530,v1:172.21.15.71:6803/1173641530] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f26d4062cd0 con 0x7f26bc0ad080 2024-03-04T08:57:26.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.517+0000 7f26c37fe700 1 -- 172.21.15.71:0/13657704 <== osd.0 v2:172.21.15.71:6802/1173641530 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f26d4062cd0 con 0x7f26bc0ad080 2024-03-04T08:57:26.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.518+0000 7f26d8f39700 1 -- 172.21.15.71:0/13657704 >> [v2:172.21.15.71:6802/1173641530,v1:172.21.15.71:6803/1173641530] conn(0x7f26bc0ad080 msgr2=0x7f26bc0af520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:26.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.518+0000 7f26d8f39700 1 --2- 172.21.15.71:0/13657704 >> [v2:172.21.15.71:6802/1173641530,v1:172.21.15.71:6803/1173641530] conn(0x7f26bc0ad080 0x7f26bc0af520 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:26.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.518+0000 7f26d8f39700 1 -- 172.21.15.71:0/13657704 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f26bc0a1f20 msgr2=0x7f26bc0a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:26.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.518+0000 7f26d8f39700 1 --2- 172.21.15.71:0/13657704 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f26bc0a1f20 0x7f26bc0a43e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f26c4000c00 tx=0x7f26c4005fb0 comp rx=0 tx=0).stop 2024-03-04T08:57:26.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.518+0000 7f26d8f39700 1 -- 172.21.15.71:0/13657704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26d41080f0 msgr2=0x7f26d4074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:26.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.518+0000 7f26d8f39700 1 --2- 172.21.15.71:0/13657704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26d41080f0 0x7f26d4074e40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00ef30 tx=0x7f26cc00c3a0 comp rx=0 tx=0).stop 2024-03-04T08:57:26.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.518+0000 7f26d2d9d700 1 -- 172.21.15.71:0/13657704 reap_dead start 2024-03-04T08:57:26.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.518+0000 7f26d8f39700 1 -- 172.21.15.71:0/13657704 shutdown_connections 2024-03-04T08:57:26.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.518+0000 7f26d8f39700 1 -- 172.21.15.71:0/13657704 >> 172.21.15.71:0/13657704 conn(0x7f26d40791f0 msgr2=0x7f26d4100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:26.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.518+0000 7f26d8f39700 1 -- 172.21.15.71:0/13657704 shutdown_connections 2024-03-04T08:57:26.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.518+0000 7f26d8f39700 1 -- 172.21.15.71:0/13657704 wait complete. 2024-03-04T08:57:26.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.924+0000 7f942b77c700 1 -- 172.21.15.71:0/798385046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9424102170 msgr2=0x7f94241025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:26.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.924+0000 7f942b77c700 1 --2- 172.21.15.71:0/798385046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9424102170 0x7f94241025f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9420009990 tx=0x7f9420009ca0 comp rx=0 tx=0).stop 2024-03-04T08:57:26.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.924+0000 7f942b77c700 1 -- 172.21.15.71:0/798385046 shutdown_connections 2024-03-04T08:57:26.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.924+0000 7f942b77c700 1 --2- 172.21.15.71:0/798385046 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9424102b30 0x7f942410ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:26.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.924+0000 7f942b77c700 1 --2- 172.21.15.71:0/798385046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94241081e0 0x7f94241085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:26.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.924+0000 7f942b77c700 1 --2- 172.21.15.71:0/798385046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9424102170 0x7f94241025f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:26.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.924+0000 7f942b77c700 1 -- 172.21.15.71:0/798385046 >> 172.21.15.71:0/798385046 conn(0x7f9424079190 msgr2=0x7f94240795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:26.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.924+0000 7f942b77c700 1 -- 172.21.15.71:0/798385046 shutdown_connections 2024-03-04T08:57:26.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.924+0000 7f942b77c700 1 -- 172.21.15.71:0/798385046 wait complete. 2024-03-04T08:57:26.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.925+0000 7f942b77c700 1 Processor -- start 2024-03-04T08:57:26.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.925+0000 7f942b77c700 1 -- start start 2024-03-04T08:57:26.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.925+0000 7f942b77c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9424102170 0x7f942419e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:26.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.925+0000 7f942b77c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9424102b30 0x7f942419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:26.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.925+0000 7f942b77c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94241081e0 0x7f9424198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:26.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.925+0000 7f942b77c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9424112430 con 0x7f94241081e0 2024-03-04T08:57:26.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.925+0000 7f942b77c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94241122b0 con 0x7f9424102b30 2024-03-04T08:57:26.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.925+0000 7f942b77c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94241125b0 con 0x7f9424102170 2024-03-04T08:57:26.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.925+0000 7f9429d19700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94241081e0 0x7f9424198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:26.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.926+0000 7f9428d17700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9424102b30 0x7f942419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:26.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.926+0000 7f9429d19700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94241081e0 0x7f9424198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45342/0 (socket says 172.21.15.71:45342) 2024-03-04T08:57:26.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.926+0000 7f9429d19700 1 -- 172.21.15.71:0/1470535521 learned_addr learned my addr 172.21.15.71:0/1470535521 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:57:26.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.926+0000 7f9429518700 1 --2- 172.21.15.71:0/1470535521 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9424102170 0x7f942419e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:26.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.926+0000 7f9429d19700 1 -- 172.21.15.71:0/1470535521 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9424102170 msgr2=0x7f942419e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:26.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.926+0000 7f9429d19700 1 --2- 172.21.15.71:0/1470535521 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9424102170 0x7f942419e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:26.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.926+0000 7f9429d19700 1 -- 172.21.15.71:0/1470535521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9424102b30 msgr2=0x7f942419eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:26.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.926+0000 7f9429d19700 1 --2- 172.21.15.71:0/1470535521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9424102b30 0x7f942419eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:26.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.926+0000 7f9429d19700 1 -- 172.21.15.71:0/1470535521 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9424198ff0 con 0x7f94241081e0 2024-03-04T08:57:26.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.926+0000 7f9428d17700 1 --2- 172.21.15.71:0/1470535521 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9424102b30 0x7f942419eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:57:26.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.926+0000 7f9429d19700 1 --2- 172.21.15.71:0/1470535521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94241081e0 0x7f9424198710 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f941c00ada0 tx=0x7f941c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:26.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.926+0000 7f941a7fc700 1 -- 172.21.15.71:0/1470535521 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c017020 con 0x7f94241081e0 2024-03-04T08:57:26.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.926+0000 7f941a7fc700 1 -- 172.21.15.71:0/1470535521 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f941c00cdb0 con 0x7f94241081e0 2024-03-04T08:57:26.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.926+0000 7f942b77c700 1 -- 172.21.15.71:0/1470535521 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94241992e0 con 0x7f94241081e0 2024-03-04T08:57:26.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.926+0000 7f941a7fc700 1 -- 172.21.15.71:0/1470535521 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c0053e0 con 0x7f94241081e0 2024-03-04T08:57:26.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.926+0000 7f942b77c700 1 -- 172.21.15.71:0/1470535521 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94241a55c0 con 0x7f94241081e0 2024-03-04T08:57:26.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.927+0000 7f942b77c700 1 -- 172.21.15.71:0/1470535521 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9424062790 con 0x7f94241081e0 2024-03-04T08:57:26.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.928+0000 7f941a7fc700 1 -- 172.21.15.71:0/1470535521 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f941c00c8d0 con 0x7f94241081e0 2024-03-04T08:57:26.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.929+0000 7f941a7fc700 1 --2- 172.21.15.71:0/1470535521 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f94100a1f20 0x7f94100a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:26.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.929+0000 7f9429518700 1 --2- 172.21.15.71:0/1470535521 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f94100a1f20 0x7f94100a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:26.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.929+0000 7f941a7fc700 1 -- 172.21.15.71:0/1470535521 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f941c0ce330 con 0x7f94241081e0 2024-03-04T08:57:26.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.929+0000 7f941a7fc700 1 --2- 172.21.15.71:0/1470535521 >> [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347] conn(0x7f94100ad080 0x7f94100af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:26.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.929+0000 7f941a7fc700 1 -- 172.21.15.71:0/1470535521 --> [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f941c01ee50 con 0x7f94100ad080 2024-03-04T08:57:26.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.929+0000 7f941a7fc700 1 -- 172.21.15.71:0/1470535521 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f941c0ce780 con 0x7f94241081e0 2024-03-04T08:57:26.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.929+0000 7f9428d17700 1 --2- 172.21.15.71:0/1470535521 >> [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347] conn(0x7f94100ad080 0x7f94100af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:26.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.929+0000 7f9429518700 1 --2- 172.21.15.71:0/1470535521 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f94100a1f20 0x7f94100a43e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9414009be0 tx=0x7f9414009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:26.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.929+0000 7f9428d17700 1 --2- 172.21.15.71:0/1470535521 >> [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347] conn(0x7f94100ad080 0x7f94100af520 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:26.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.930+0000 7f941a7fc700 1 -- 172.21.15.71:0/1470535521 <== osd.9 v2:172.21.15.71:6826/1913779347 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f941c01ee50 con 0x7f94100ad080 2024-03-04T08:57:26.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.946+0000 7f942b77c700 1 -- 172.21.15.71:0/1470535521 --> [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9424062c10 con 0x7f94100ad080 2024-03-04T08:57:26.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.946+0000 7f941a7fc700 1 -- 172.21.15.71:0/1470535521 <== osd.9 v2:172.21.15.71:6826/1913779347 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9424062c10 con 0x7f94100ad080 2024-03-04T08:57:26.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.947+0000 7f942b77c700 1 -- 172.21.15.71:0/1470535521 >> [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347] conn(0x7f94100ad080 msgr2=0x7f94100af520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:26.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.947+0000 7f942b77c700 1 --2- 172.21.15.71:0/1470535521 >> [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347] conn(0x7f94100ad080 0x7f94100af520 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:26.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.947+0000 7f942b77c700 1 -- 172.21.15.71:0/1470535521 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f94100a1f20 msgr2=0x7f94100a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:26.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.947+0000 7f942b77c700 1 --2- 172.21.15.71:0/1470535521 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f94100a1f20 0x7f94100a43e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9414009be0 tx=0x7f9414009260 comp rx=0 tx=0).stop 2024-03-04T08:57:26.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.947+0000 7f942b77c700 1 -- 172.21.15.71:0/1470535521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94241081e0 msgr2=0x7f9424198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:26.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.947+0000 7f942b77c700 1 --2- 172.21.15.71:0/1470535521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94241081e0 0x7f9424198710 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f941c00ada0 tx=0x7f941c00c330 comp rx=0 tx=0).stop 2024-03-04T08:57:26.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.947+0000 7f9429d19700 1 -- 172.21.15.71:0/1470535521 reap_dead start 2024-03-04T08:57:26.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.947+0000 7f942b77c700 1 -- 172.21.15.71:0/1470535521 shutdown_connections 2024-03-04T08:57:26.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.947+0000 7f942b77c700 1 -- 172.21.15.71:0/1470535521 >> 172.21.15.71:0/1470535521 conn(0x7f9424079190 msgr2=0x7f9424100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:26.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.947+0000 7f942b77c700 1 -- 172.21.15.71:0/1470535521 shutdown_connections 2024-03-04T08:57:26.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:26.947+0000 7f942b77c700 1 -- 172.21.15.71:0/1470535521 wait complete. 2024-03-04T08:57:27.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.241+0000 7fe852b24700 1 -- 172.21.15.71:0/1977846853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe84c101960 msgr2=0x7fe84c10de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:27.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.241+0000 7fe852b24700 1 --2- 172.21.15.71:0/1977846853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe84c101960 0x7fe84c10de00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe84400b0a0 tx=0x7fe84400b3b0 comp rx=0 tx=0).stop 2024-03-04T08:57:27.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.241+0000 7fe852b24700 1 -- 172.21.15.71:0/1977846853 shutdown_connections 2024-03-04T08:57:27.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.241+0000 7fe852b24700 1 --2- 172.21.15.71:0/1977846853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe84c101960 0x7fe84c10de00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:27.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.241+0000 7fe852b24700 1 --2- 172.21.15.71:0/1977846853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe84c107050 0x7fe84c107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:27.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.241+0000 7fe852b24700 1 --2- 172.21.15.71:0/1977846853 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe84c100fa0 0x7fe84c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:27.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.241+0000 7fe852b24700 1 -- 172.21.15.71:0/1977846853 >> 172.21.15.71:0/1977846853 conn(0x7fe84c077fd0 msgr2=0x7fe84c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:27.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.241+0000 7fe852b24700 1 -- 172.21.15.71:0/1977846853 shutdown_connections 2024-03-04T08:57:27.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.241+0000 7fe852b24700 1 -- 172.21.15.71:0/1977846853 wait complete. 2024-03-04T08:57:27.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.242+0000 7fe852b24700 1 Processor -- start 2024-03-04T08:57:27.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.242+0000 7fe852b24700 1 -- start start 2024-03-04T08:57:27.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.242+0000 7fe852b24700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe84c100fa0 0x7fe84c198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:27.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.242+0000 7fe852b24700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe84c101960 0x7fe84c198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:27.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.242+0000 7fe852b24700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe84c107050 0x7fe84c19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:27.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.242+0000 7fe852b24700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c111260 con 0x7fe84c100fa0 2024-03-04T08:57:27.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.242+0000 7fe852b24700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c1110e0 con 0x7fe84c107050 2024-03-04T08:57:27.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.242+0000 7fe852b24700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c1113e0 con 0x7fe84c101960 2024-03-04T08:57:27.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.242+0000 7fe8508c0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe84c100fa0 0x7fe84c198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:27.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.242+0000 7fe8508c0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe84c100fa0 0x7fe84c198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45364/0 (socket says 172.21.15.71:45364) 2024-03-04T08:57:27.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.242+0000 7fe8508c0700 1 -- 172.21.15.71:0/123360701 learned_addr learned my addr 172.21.15.71:0/123360701 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:57:27.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.242+0000 7fe8510c1700 1 --2- 172.21.15.71:0/123360701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe84c107050 0x7fe84c19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:27.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.242+0000 7fe84bfff700 1 --2- 172.21.15.71:0/123360701 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe84c101960 0x7fe84c198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:27.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.242+0000 7fe8508c0700 1 -- 172.21.15.71:0/123360701 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe84c101960 msgr2=0x7fe84c198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:27.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.242+0000 7fe8508c0700 1 --2- 172.21.15.71:0/123360701 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe84c101960 0x7fe84c198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:27.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.242+0000 7fe8508c0700 1 -- 172.21.15.71:0/123360701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe84c107050 msgr2=0x7fe84c19cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:27.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.242+0000 7fe8508c0700 1 --2- 172.21.15.71:0/123360701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe84c107050 0x7fe84c19cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:27.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.243+0000 7fe8508c0700 1 -- 172.21.15.71:0/123360701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84c19d730 con 0x7fe84c100fa0 2024-03-04T08:57:27.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.243+0000 7fe84bfff700 1 --2- 172.21.15.71:0/123360701 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe84c101960 0x7fe84c198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:57:27.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.243+0000 7fe8508c0700 1 --2- 172.21.15.71:0/123360701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe84c100fa0 0x7fe84c198720 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe84000eb30 tx=0x7fe84000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:27.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.243+0000 7fe849ffb700 1 -- 172.21.15.71:0/123360701 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84000cc90 con 0x7fe84c100fa0 2024-03-04T08:57:27.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.243+0000 7fe849ffb700 1 -- 172.21.15.71:0/123360701 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe840004500 con 0x7fe84c100fa0 2024-03-04T08:57:27.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.243+0000 7fe852b24700 1 -- 172.21.15.71:0/123360701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84c19da20 con 0x7fe84c100fa0 2024-03-04T08:57:27.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.243+0000 7fe849ffb700 1 -- 172.21.15.71:0/123360701 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe840010430 con 0x7fe84c100fa0 2024-03-04T08:57:27.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.243+0000 7fe852b24700 1 -- 172.21.15.71:0/123360701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe84c1a52a0 con 0x7fe84c100fa0 2024-03-04T08:57:27.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.244+0000 7fe852b24700 1 -- 172.21.15.71:0/123360701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe84c0ff870 con 0x7fe84c100fa0 2024-03-04T08:57:27.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.245+0000 7fe849ffb700 1 -- 172.21.15.71:0/123360701 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fe840003f70 con 0x7fe84c100fa0 2024-03-04T08:57:27.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.245+0000 7fe849ffb700 1 --2- 172.21.15.71:0/123360701 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe83c0a1ea0 0x7fe83c0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:27.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.246+0000 7fe84bfff700 1 --2- 172.21.15.71:0/123360701 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe83c0a1ea0 0x7fe83c0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:27.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.246+0000 7fe849ffb700 1 -- 172.21.15.71:0/123360701 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fe8400ce870 con 0x7fe84c100fa0 2024-03-04T08:57:27.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.246+0000 7fe849ffb700 1 --2- 172.21.15.71:0/123360701 >> [v2:172.21.15.150:6816/1328330305,v1:172.21.15.150:6817/1328330305] conn(0x7fe83c0ad000 0x7fe83c0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:27.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.246+0000 7fe849ffb700 1 -- 172.21.15.71:0/123360701 --> [v2:172.21.15.150:6816/1328330305,v1:172.21.15.150:6817/1328330305] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe84000be50 con 0x7fe83c0ad000 2024-03-04T08:57:27.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.246+0000 7fe849ffb700 1 -- 172.21.15.71:0/123360701 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8400cecc0 con 0x7fe84c100fa0 2024-03-04T08:57:27.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.246+0000 7fe84bfff700 1 --2- 172.21.15.71:0/123360701 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe83c0a1ea0 0x7fe83c0a4360 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe84c100f10 tx=0x7fe838009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:27.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.246+0000 7fe8510c1700 1 --2- 172.21.15.71:0/123360701 >> [v2:172.21.15.150:6816/1328330305,v1:172.21.15.150:6817/1328330305] conn(0x7fe83c0ad000 0x7fe83c0af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:27.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.246+0000 7fe8510c1700 1 --2- 172.21.15.71:0/123360701 >> [v2:172.21.15.150:6816/1328330305,v1:172.21.15.150:6817/1328330305] conn(0x7fe83c0ad000 0x7fe83c0af4a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:27.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.247+0000 7fe849ffb700 1 -- 172.21.15.71:0/123360701 <== osd.7 v2:172.21.15.150:6816/1328330305 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe84000be50 con 0x7fe83c0ad000 2024-03-04T08:57:27.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.263+0000 7fe852b24700 1 -- 172.21.15.71:0/123360701 --> [v2:172.21.15.150:6816/1328330305,v1:172.21.15.150:6817/1328330305] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe84c102690 con 0x7fe83c0ad000 2024-03-04T08:57:27.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.263+0000 7fe849ffb700 1 -- 172.21.15.71:0/123360701 <== osd.7 v2:172.21.15.150:6816/1328330305 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe84c102690 con 0x7fe83c0ad000 2024-03-04T08:57:27.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.264+0000 7fe852b24700 1 -- 172.21.15.71:0/123360701 >> [v2:172.21.15.150:6816/1328330305,v1:172.21.15.150:6817/1328330305] conn(0x7fe83c0ad000 msgr2=0x7fe83c0af4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:27.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.264+0000 7fe852b24700 1 --2- 172.21.15.71:0/123360701 >> [v2:172.21.15.150:6816/1328330305,v1:172.21.15.150:6817/1328330305] conn(0x7fe83c0ad000 0x7fe83c0af4a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:27.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.264+0000 7fe852b24700 1 -- 172.21.15.71:0/123360701 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe83c0a1ea0 msgr2=0x7fe83c0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:27.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.264+0000 7fe852b24700 1 --2- 172.21.15.71:0/123360701 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe83c0a1ea0 0x7fe83c0a4360 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe84c100f10 tx=0x7fe838009260 comp rx=0 tx=0).stop 2024-03-04T08:57:27.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.264+0000 7fe852b24700 1 -- 172.21.15.71:0/123360701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe84c100fa0 msgr2=0x7fe84c198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:27.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.264+0000 7fe852b24700 1 --2- 172.21.15.71:0/123360701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe84c100fa0 0x7fe84c198720 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe84000eb30 tx=0x7fe84000ee40 comp rx=0 tx=0).stop 2024-03-04T08:57:27.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.264+0000 7fe8510c1700 1 -- 172.21.15.71:0/123360701 reap_dead start 2024-03-04T08:57:27.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.264+0000 7fe852b24700 1 -- 172.21.15.71:0/123360701 shutdown_connections 2024-03-04T08:57:27.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.264+0000 7fe852b24700 1 -- 172.21.15.71:0/123360701 >> 172.21.15.71:0/123360701 conn(0x7fe84c077fd0 msgr2=0x7fe84c0fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:27.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.264+0000 7fe852b24700 1 -- 172.21.15.71:0/123360701 shutdown_connections 2024-03-04T08:57:27.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.264+0000 7fe852b24700 1 -- 172.21.15.71:0/123360701 wait complete. 2024-03-04T08:57:28.110 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:27 smithi071 ceph-mon[32789]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s; 7.7 KiB/s, 0 objects/s recovering 2024-03-04T08:57:28.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:27 smithi150 ceph-mon[36807]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s; 7.7 KiB/s, 0 objects/s recovering 2024-03-04T08:57:28.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.477+0000 7ff0b2b59700 1 -- 172.21.15.71:0/3010094640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0ac107130 msgr2=0x7ff0ac107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:28.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.477+0000 7ff0b2b59700 1 --2- 172.21.15.71:0/3010094640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0ac107130 0x7ff0ac107510 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009990 tx=0x7ff0a0009ca0 comp rx=0 tx=0).stop 2024-03-04T08:57:28.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.477+0000 7ff0b2b59700 1 -- 172.21.15.71:0/3010094640 shutdown_connections 2024-03-04T08:57:28.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.477+0000 7ff0b2b59700 1 --2- 172.21.15.71:0/3010094640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0ac101070 0x7ff0ac10efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:28.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.477+0000 7ff0b2b59700 1 --2- 172.21.15.71:0/3010094640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0ac107130 0x7ff0ac107510 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:28.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.477+0000 7ff0b2b59700 1 --2- 172.21.15.71:0/3010094640 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff0ac1006b0 0x7ff0ac100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:28.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.477+0000 7ff0b2b59700 1 -- 172.21.15.71:0/3010094640 >> 172.21.15.71:0/3010094640 conn(0x7ff0ac0775c0 msgr2=0x7ff0ac0fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:28.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.477+0000 7ff0b2b59700 1 -- 172.21.15.71:0/3010094640 shutdown_connections 2024-03-04T08:57:28.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.477+0000 7ff0b2b59700 1 -- 172.21.15.71:0/3010094640 wait complete. 2024-03-04T08:57:28.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.477+0000 7ff0b2b59700 1 Processor -- start 2024-03-04T08:57:28.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.478+0000 7ff0b2b59700 1 -- start start 2024-03-04T08:57:28.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.478+0000 7ff0b2b59700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff0ac1006b0 0x7ff0ac101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:28.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.478+0000 7ff0b2b59700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0ac101070 0x7ff0ac101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:28.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.478+0000 7ff0b2b59700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0ac107130 0x7ff0ac102310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:28.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.478+0000 7ff0b2b59700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac112510 con 0x7ff0ac101070 2024-03-04T08:57:28.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.478+0000 7ff0b2b59700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac112390 con 0x7ff0ac107130 2024-03-04T08:57:28.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.478+0000 7ff0b2b59700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac112690 con 0x7ff0ac1006b0 2024-03-04T08:57:28.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.478+0000 7ff0abfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0ac101070 0x7ff0ac101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:28.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.478+0000 7ff0abfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0ac101070 0x7ff0ac101dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45380/0 (socket says 172.21.15.71:45380) 2024-03-04T08:57:28.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.478+0000 7ff0abfff700 1 -- 172.21.15.71:0/4210491138 learned_addr learned my addr 172.21.15.71:0/4210491138 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:57:28.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.478+0000 7ff0b10f6700 1 --2- 172.21.15.71:0/4210491138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0ac107130 0x7ff0ac102310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:28.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.478+0000 7ff0b08f5700 1 --2- 172.21.15.71:0/4210491138 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff0ac1006b0 0x7ff0ac101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:28.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.478+0000 7ff0abfff700 1 -- 172.21.15.71:0/4210491138 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff0ac1006b0 msgr2=0x7ff0ac101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:28.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.478+0000 7ff0abfff700 1 --2- 172.21.15.71:0/4210491138 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff0ac1006b0 0x7ff0ac101890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:28.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.478+0000 7ff0abfff700 1 -- 172.21.15.71:0/4210491138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0ac107130 msgr2=0x7ff0ac102310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:28.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.478+0000 7ff0abfff700 1 --2- 172.21.15.71:0/4210491138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0ac107130 0x7ff0ac102310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:28.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.478+0000 7ff0abfff700 1 -- 172.21.15.71:0/4210491138 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ac1a6ed0 con 0x7ff0ac101070 2024-03-04T08:57:28.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.478+0000 7ff0b08f5700 1 --2- 172.21.15.71:0/4210491138 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff0ac1006b0 0x7ff0ac101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:57:28.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.478+0000 7ff0b10f6700 1 --2- 172.21.15.71:0/4210491138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0ac107130 0x7ff0ac102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:57:28.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.478+0000 7ff0abfff700 1 --2- 172.21.15.71:0/4210491138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0ac101070 0x7ff0ac101dd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff09800e9c0 tx=0x7ff09800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:28.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.478+0000 7ff0a9ffb700 1 -- 172.21.15.71:0/4210491138 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09800cbd0 con 0x7ff0ac101070 2024-03-04T08:57:28.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.478+0000 7ff0b2b59700 1 -- 172.21.15.71:0/4210491138 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0ac1a71c0 con 0x7ff0ac101070 2024-03-04T08:57:28.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.478+0000 7ff0a9ffb700 1 -- 172.21.15.71:0/4210491138 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff09800cd70 con 0x7ff0ac101070 2024-03-04T08:57:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.478+0000 7ff0a9ffb700 1 -- 172.21.15.71:0/4210491138 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff098010430 con 0x7ff0ac101070 2024-03-04T08:57:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.479+0000 7ff0b2b59700 1 -- 172.21.15.71:0/4210491138 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0ac1a76b0 con 0x7ff0ac101070 2024-03-04T08:57:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.479+0000 7ff0b2b59700 1 -- 172.21.15.71:0/4210491138 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff0ac0627d0 con 0x7ff0ac101070 2024-03-04T08:57:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.480+0000 7ff0a9ffb700 1 -- 172.21.15.71:0/4210491138 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7ff098004750 con 0x7ff0ac101070 2024-03-04T08:57:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.481+0000 7ff0a9ffb700 1 --2- 172.21.15.71:0/4210491138 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff09c0a1ea0 0x7ff09c0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.481+0000 7ff0b08f5700 1 --2- 172.21.15.71:0/4210491138 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff09c0a1ea0 0x7ff09c0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.481+0000 7ff0a9ffb700 1 -- 172.21.15.71:0/4210491138 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7ff0980ceb80 con 0x7ff0ac101070 2024-03-04T08:57:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.481+0000 7ff0a9ffb700 1 --2- 172.21.15.71:0/4210491138 >> [v2:172.21.15.71:6818/3848842707,v1:172.21.15.71:6819/3848842707] conn(0x7ff09c0ad000 0x7ff09c0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.481+0000 7ff0a9ffb700 1 -- 172.21.15.71:0/4210491138 --> [v2:172.21.15.71:6818/3848842707,v1:172.21.15.71:6819/3848842707] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff09c0afb70 con 0x7ff09c0ad000 2024-03-04T08:57:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.481+0000 7ff0a9ffb700 1 -- 172.21.15.71:0/4210491138 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0980cefd0 con 0x7ff0ac101070 2024-03-04T08:57:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.481+0000 7ff0b10f6700 1 --2- 172.21.15.71:0/4210491138 >> [v2:172.21.15.71:6818/3848842707,v1:172.21.15.71:6819/3848842707] conn(0x7ff09c0ad000 0x7ff09c0af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.481+0000 7ff0b08f5700 1 --2- 172.21.15.71:0/4210491138 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff09c0a1ea0 0x7ff09c0a4360 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000b2e0 tx=0x7ff0a001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.482+0000 7ff0b10f6700 1 --2- 172.21.15.71:0/4210491138 >> [v2:172.21.15.71:6818/3848842707,v1:172.21.15.71:6819/3848842707] conn(0x7ff09c0ad000 0x7ff09c0af4a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.482+0000 7ff0a9ffb700 1 -- 172.21.15.71:0/4210491138 <== osd.6 v2:172.21.15.71:6818/3848842707 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff09c0afb70 con 0x7ff09c0ad000 2024-03-04T08:57:28.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.498+0000 7ff0b2b59700 1 -- 172.21.15.71:0/4210491138 --> [v2:172.21.15.71:6818/3848842707,v1:172.21.15.71:6819/3848842707] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff0ac062bd0 con 0x7ff09c0ad000 2024-03-04T08:57:28.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.499+0000 7ff0a9ffb700 1 -- 172.21.15.71:0/4210491138 <== osd.6 v2:172.21.15.71:6818/3848842707 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff0ac062bd0 con 0x7ff09c0ad000 2024-03-04T08:57:28.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.499+0000 7ff0b2b59700 1 -- 172.21.15.71:0/4210491138 >> [v2:172.21.15.71:6818/3848842707,v1:172.21.15.71:6819/3848842707] conn(0x7ff09c0ad000 msgr2=0x7ff09c0af4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:28.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.499+0000 7ff0b2b59700 1 --2- 172.21.15.71:0/4210491138 >> [v2:172.21.15.71:6818/3848842707,v1:172.21.15.71:6819/3848842707] conn(0x7ff09c0ad000 0x7ff09c0af4a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:28.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.499+0000 7ff0b2b59700 1 -- 172.21.15.71:0/4210491138 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff09c0a1ea0 msgr2=0x7ff09c0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:28.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.499+0000 7ff0b2b59700 1 --2- 172.21.15.71:0/4210491138 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff09c0a1ea0 0x7ff09c0a4360 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000b2e0 tx=0x7ff0a001a040 comp rx=0 tx=0).stop 2024-03-04T08:57:28.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.499+0000 7ff0b2b59700 1 -- 172.21.15.71:0/4210491138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0ac101070 msgr2=0x7ff0ac101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:28.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.499+0000 7ff0b2b59700 1 --2- 172.21.15.71:0/4210491138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0ac101070 0x7ff0ac101dd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff09800e9c0 tx=0x7ff09800ed80 comp rx=0 tx=0).stop 2024-03-04T08:57:28.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.499+0000 7ff0b10f6700 1 -- 172.21.15.71:0/4210491138 reap_dead start 2024-03-04T08:57:28.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.499+0000 7ff0b2b59700 1 -- 172.21.15.71:0/4210491138 shutdown_connections 2024-03-04T08:57:28.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.499+0000 7ff0b2b59700 1 -- 172.21.15.71:0/4210491138 >> 172.21.15.71:0/4210491138 conn(0x7ff0ac0775c0 msgr2=0x7ff0ac105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:28.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.499+0000 7ff0b2b59700 1 -- 172.21.15.71:0/4210491138 shutdown_connections 2024-03-04T08:57:28.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:27.499+0000 7ff0b2b59700 1 -- 172.21.15.71:0/4210491138 wait complete. 2024-03-04T08:57:28.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:27 smithi158 ceph-mon[37241]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s; 7.7 KiB/s, 0 objects/s recovering 2024-03-04T08:57:28.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.502+0000 7f12a83d8700 1 -- 172.21.15.71:0/239503341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a0106000 msgr2=0x7f12a01063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:28.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.502+0000 7f12a83d8700 1 --2- 172.21.15.71:0/239503341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a0106000 0x7f12a01063e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f129c009990 tx=0x7f129c009ca0 comp rx=0 tx=0).stop 2024-03-04T08:57:28.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.502+0000 7f12a83d8700 1 -- 172.21.15.71:0/239503341 shutdown_connections 2024-03-04T08:57:28.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.502+0000 7f12a83d8700 1 --2- 172.21.15.71:0/239503341 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f12a01024d0 0x7f12a010dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:28.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.502+0000 7f12a83d8700 1 --2- 172.21.15.71:0/239503341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a0106000 0x7f12a01063e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:28.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.502+0000 7f12a83d8700 1 --2- 172.21.15.71:0/239503341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12a0101b10 0x7f12a0101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:28.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.502+0000 7f12a83d8700 1 -- 172.21.15.71:0/239503341 >> 172.21.15.71:0/239503341 conn(0x7f12a0076300 msgr2=0x7f12a0076710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:28.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.502+0000 7f12a83d8700 1 -- 172.21.15.71:0/239503341 shutdown_connections 2024-03-04T08:57:28.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.502+0000 7f12a83d8700 1 -- 172.21.15.71:0/239503341 wait complete. 2024-03-04T08:57:28.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.502+0000 7f12a83d8700 1 Processor -- start 2024-03-04T08:57:28.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.503+0000 7f12a83d8700 1 -- start start 2024-03-04T08:57:28.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.503+0000 7f12a83d8700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f12a0101b10 0x7f12a019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:28.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.503+0000 7f12a83d8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a01024d0 0x7f12a019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:28.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.503+0000 7f12a83d8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12a0106000 0x7f12a01986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:28.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.503+0000 7f12a83d8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a0111380 con 0x7f12a01024d0 2024-03-04T08:57:28.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.503+0000 7f12a83d8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a0111200 con 0x7f12a0106000 2024-03-04T08:57:28.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.503+0000 7f12a83d8700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a0111500 con 0x7f12a0101b10 2024-03-04T08:57:28.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.503+0000 7f12a5973700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a01024d0 0x7f12a019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:28.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.503+0000 7f12a5973700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a01024d0 0x7f12a019eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45396/0 (socket says 172.21.15.71:45396) 2024-03-04T08:57:28.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.503+0000 7f12a5973700 1 -- 172.21.15.71:0/1356954703 learned_addr learned my addr 172.21.15.71:0/1356954703 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:57:28.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.503+0000 7f12a6975700 1 --2- 172.21.15.71:0/1356954703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12a0106000 0x7f12a01986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:28.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.503+0000 7f12a6174700 1 --2- 172.21.15.71:0/1356954703 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f12a0101b10 0x7f12a019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:28.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.503+0000 7f12a5973700 1 -- 172.21.15.71:0/1356954703 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f12a0101b10 msgr2=0x7f12a019e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:28.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.503+0000 7f12a5973700 1 --2- 172.21.15.71:0/1356954703 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f12a0101b10 0x7f12a019e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:28.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.503+0000 7f12a5973700 1 -- 172.21.15.71:0/1356954703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12a0106000 msgr2=0x7f12a01986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:28.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.503+0000 7f12a5973700 1 --2- 172.21.15.71:0/1356954703 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12a0106000 0x7f12a01986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:28.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.503+0000 7f12a5973700 1 -- 172.21.15.71:0/1356954703 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a0198f90 con 0x7f12a01024d0 2024-03-04T08:57:28.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.503+0000 7f12a6174700 1 --2- 172.21.15.71:0/1356954703 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f12a0101b10 0x7f12a019e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:57:28.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.503+0000 7f12a5973700 1 --2- 172.21.15.71:0/1356954703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a01024d0 0x7f12a019eb20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f129000e9a0 tx=0x7f129000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:28.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.503+0000 7f12977fe700 1 -- 172.21.15.71:0/1356954703 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129000cbd0 con 0x7f12a01024d0 2024-03-04T08:57:28.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.503+0000 7f12977fe700 1 -- 172.21.15.71:0/1356954703 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f129000cd70 con 0x7f12a01024d0 2024-03-04T08:57:28.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.503+0000 7f12a83d8700 1 -- 172.21.15.71:0/1356954703 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a0199280 con 0x7f12a01024d0 2024-03-04T08:57:28.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.503+0000 7f12977fe700 1 -- 172.21.15.71:0/1356954703 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1290018750 con 0x7f12a01024d0 2024-03-04T08:57:28.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.504+0000 7f12a83d8700 1 -- 172.21.15.71:0/1356954703 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a01a5560 con 0x7f12a01024d0 2024-03-04T08:57:28.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.504+0000 7f12a83d8700 1 -- 172.21.15.71:0/1356954703 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f12a0100900 con 0x7f12a01024d0 2024-03-04T08:57:28.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.505+0000 7f12977fe700 1 -- 172.21.15.71:0/1356954703 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f1290010430 con 0x7f12a01024d0 2024-03-04T08:57:28.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.506+0000 7f12977fe700 1 --2- 172.21.15.71:0/1356954703 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f128c0a1ea0 0x7f128c0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:28.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.506+0000 7f12a6174700 1 --2- 172.21.15.71:0/1356954703 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f128c0a1ea0 0x7f128c0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:28.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.506+0000 7f12977fe700 1 -- 172.21.15.71:0/1356954703 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f12900ceda0 con 0x7f12a01024d0 2024-03-04T08:57:28.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.506+0000 7f12977fe700 1 --2- 172.21.15.71:0/1356954703 >> [v2:172.21.15.158:6816/2195304577,v1:172.21.15.158:6817/2195304577] conn(0x7f128c0acff0 0x7f128c0af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:28.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.506+0000 7f12977fe700 1 -- 172.21.15.71:0/1356954703 --> [v2:172.21.15.158:6816/2195304577,v1:172.21.15.158:6817/2195304577] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f129000ae50 con 0x7f128c0acff0 2024-03-04T08:57:28.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.506+0000 7f12977fe700 1 -- 172.21.15.71:0/1356954703 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12900cf1f0 con 0x7f12a01024d0 2024-03-04T08:57:28.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.507+0000 7f12a6174700 1 --2- 172.21.15.71:0/1356954703 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f128c0a1ea0 0x7f128c0a4360 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f129c00b2e0 tx=0x7f129c005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:28.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.507+0000 7f12a6975700 1 --2- 172.21.15.71:0/1356954703 >> [v2:172.21.15.158:6816/2195304577,v1:172.21.15.158:6817/2195304577] conn(0x7f128c0acff0 0x7f128c0af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:28.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.507+0000 7f12a6975700 1 --2- 172.21.15.71:0/1356954703 >> [v2:172.21.15.158:6816/2195304577,v1:172.21.15.158:6817/2195304577] conn(0x7f128c0acff0 0x7f128c0af490 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:28.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.508+0000 7f12977fe700 1 -- 172.21.15.71:0/1356954703 <== osd.8 v2:172.21.15.158:6816/2195304577 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f129000ae50 con 0x7f128c0acff0 2024-03-04T08:57:28.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.524+0000 7f12a83d8700 1 -- 172.21.15.71:0/1356954703 --> [v2:172.21.15.158:6816/2195304577,v1:172.21.15.158:6817/2195304577] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f12a006a390 con 0x7f128c0acff0 2024-03-04T08:57:28.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.524+0000 7f12977fe700 1 -- 172.21.15.71:0/1356954703 <== osd.8 v2:172.21.15.158:6816/2195304577 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f12a006a390 con 0x7f128c0acff0 2024-03-04T08:57:28.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.525+0000 7f12a83d8700 1 -- 172.21.15.71:0/1356954703 >> [v2:172.21.15.158:6816/2195304577,v1:172.21.15.158:6817/2195304577] conn(0x7f128c0acff0 msgr2=0x7f128c0af490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:28.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.525+0000 7f12a83d8700 1 --2- 172.21.15.71:0/1356954703 >> [v2:172.21.15.158:6816/2195304577,v1:172.21.15.158:6817/2195304577] conn(0x7f128c0acff0 0x7f128c0af490 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:28.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.525+0000 7f12a83d8700 1 -- 172.21.15.71:0/1356954703 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f128c0a1ea0 msgr2=0x7f128c0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:28.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.525+0000 7f12a83d8700 1 --2- 172.21.15.71:0/1356954703 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f128c0a1ea0 0x7f128c0a4360 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f129c00b2e0 tx=0x7f129c005e80 comp rx=0 tx=0).stop 2024-03-04T08:57:28.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.525+0000 7f12a83d8700 1 -- 172.21.15.71:0/1356954703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a01024d0 msgr2=0x7f12a019eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:28.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.525+0000 7f12a83d8700 1 --2- 172.21.15.71:0/1356954703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12a01024d0 0x7f12a019eb20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f129000e9a0 tx=0x7f129000ecb0 comp rx=0 tx=0).stop 2024-03-04T08:57:28.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.525+0000 7f12a6975700 1 -- 172.21.15.71:0/1356954703 reap_dead start 2024-03-04T08:57:28.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.525+0000 7f12a83d8700 1 -- 172.21.15.71:0/1356954703 shutdown_connections 2024-03-04T08:57:28.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.525+0000 7f12a83d8700 1 -- 172.21.15.71:0/1356954703 >> 172.21.15.71:0/1356954703 conn(0x7f12a0076300 msgr2=0x7f12a0105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:28.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.525+0000 7f12a83d8700 1 -- 172.21.15.71:0/1356954703 shutdown_connections 2024-03-04T08:57:28.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:28.525+0000 7f12a83d8700 1 -- 172.21.15.71:0/1356954703 wait complete. 2024-03-04T08:57:29.265 INFO:teuthology.orchestra.run.smithi071.stdout:201863462966 2024-03-04T08:57:29.265 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-04T08:57:29.656 INFO:teuthology.orchestra.run.smithi071.stdout:296352743461 2024-03-04T08:57:29.656 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-04T08:57:29.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:29 smithi071 ceph-mon[32789]: pgmap v356: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s; 7.7 KiB/s, 0 objects/s recovering 2024-03-04T08:57:30.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:29 smithi150 ceph-mon[36807]: pgmap v356: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s; 7.7 KiB/s, 0 objects/s recovering 2024-03-04T08:57:30.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:29 smithi158 ceph-mon[37241]: pgmap v356: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s; 7.7 KiB/s, 0 objects/s recovering 2024-03-04T08:57:31.104 INFO:teuthology.orchestra.run.smithi071.stdout:223338299440 2024-03-04T08:57:31.105 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-04T08:57:31.483 INFO:teuthology.orchestra.run.smithi071.stdout:352187318298 2024-03-04T08:57:31.483 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-04T08:57:31.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.668+0000 7fe50e6ad700 1 -- 172.21.15.71:0/3484732963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5081006b0 msgr2=0x7fe508100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:31.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.668+0000 7fe50e6ad700 1 --2- 172.21.15.71:0/3484732963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5081006b0 0x7fe508100b30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009990 tx=0x7fe4fc009ca0 comp rx=0 tx=0).stop 2024-03-04T08:57:31.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.668+0000 7fe50e6ad700 1 -- 172.21.15.71:0/3484732963 shutdown_connections 2024-03-04T08:57:31.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.668+0000 7fe50e6ad700 1 --2- 172.21.15.71:0/3484732963 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe508101070 0x7fe50810efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:31.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.668+0000 7fe50e6ad700 1 --2- 172.21.15.71:0/3484732963 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe508107130 0x7fe508107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:31.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.668+0000 7fe50e6ad700 1 --2- 172.21.15.71:0/3484732963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5081006b0 0x7fe508100b30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:31.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.668+0000 7fe50e6ad700 1 -- 172.21.15.71:0/3484732963 >> 172.21.15.71:0/3484732963 conn(0x7fe5080775c0 msgr2=0x7fe5080fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:31.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.669+0000 7fe50e6ad700 1 -- 172.21.15.71:0/3484732963 shutdown_connections 2024-03-04T08:57:31.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.669+0000 7fe50e6ad700 1 -- 172.21.15.71:0/3484732963 wait complete. 2024-03-04T08:57:31.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.669+0000 7fe50e6ad700 1 Processor -- start 2024-03-04T08:57:31.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.669+0000 7fe50e6ad700 1 -- start start 2024-03-04T08:57:31.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.670+0000 7fe50e6ad700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5081006b0 0x7fe50810e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:31.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.670+0000 7fe50e6ad700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe508101070 0x7fe50810e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:31.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.670+0000 7fe50e6ad700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe508107130 0x7fe5081a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:31.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.670+0000 7fe50e6ad700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe508112540 con 0x7fe5081006b0 2024-03-04T08:57:31.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.670+0000 7fe50e6ad700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5081123c0 con 0x7fe508101070 2024-03-04T08:57:31.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.670+0000 7fe50e6ad700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5081126c0 con 0x7fe508107130 2024-03-04T08:57:31.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.670+0000 7fe507fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5081006b0 0x7fe50810e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:31.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.670+0000 7fe50cc4a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe508107130 0x7fe5081a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:31.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.670+0000 7fe50cc4a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe508107130 0x7fe5081a3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.158:3300/0 says I am v2:172.21.15.71:44052/0 (socket says 172.21.15.71:44052) 2024-03-04T08:57:31.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.670+0000 7fe50cc4a700 1 -- 172.21.15.71:0/3683395273 learned_addr learned my addr 172.21.15.71:0/3683395273 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:57:31.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.670+0000 7fe5077fe700 1 --2- 172.21.15.71:0/3683395273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe508101070 0x7fe50810e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:31.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.671+0000 7fe50cc4a700 1 -- 172.21.15.71:0/3683395273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe508101070 msgr2=0x7fe50810e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:31.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.671+0000 7fe50cc4a700 1 --2- 172.21.15.71:0/3683395273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe508101070 0x7fe50810e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:31.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.671+0000 7fe50cc4a700 1 -- 172.21.15.71:0/3683395273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5081006b0 msgr2=0x7fe50810e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:31.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.671+0000 7fe50cc4a700 1 --2- 172.21.15.71:0/3683395273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5081006b0 0x7fe50810e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:31.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.671+0000 7fe50cc4a700 1 -- 172.21.15.71:0/3683395273 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5081a3fb0 con 0x7fe508107130 2024-03-04T08:57:31.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.671+0000 7fe5077fe700 1 --2- 172.21.15.71:0/3683395273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe508101070 0x7fe50810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:57:31.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.671+0000 7fe50cc4a700 1 --2- 172.21.15.71:0/3683395273 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe508107130 0x7fe5081a3870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe500007ec0 tx=0x7fe50000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:31.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.671+0000 7fe5057fa700 1 -- 172.21.15.71:0/3683395273 <== mon.1 v2:172.21.15.158:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50001a070 con 0x7fe508107130 2024-03-04T08:57:31.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.671+0000 7fe50e6ad700 1 -- 172.21.15.71:0/3683395273 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5081a42a0 con 0x7fe508107130 2024-03-04T08:57:31.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.671+0000 7fe5057fa700 1 -- 172.21.15.71:0/3683395273 <== mon.1 v2:172.21.15.158:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe50000ddd0 con 0x7fe508107130 2024-03-04T08:57:31.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.671+0000 7fe5057fa700 1 -- 172.21.15.71:0/3683395273 <== mon.1 v2:172.21.15.158:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5000127f0 con 0x7fe508107130 2024-03-04T08:57:31.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.671+0000 7fe50e6ad700 1 -- 172.21.15.71:0/3683395273 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5081abb20 con 0x7fe508107130 2024-03-04T08:57:31.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.672+0000 7fe50e6ad700 1 -- 172.21.15.71:0/3683395273 --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe508105950 con 0x7fe508107130 2024-03-04T08:57:31.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.674+0000 7fe5057fa700 1 -- 172.21.15.71:0/3683395273 <== mon.1 v2:172.21.15.158:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fe50001d070 con 0x7fe508107130 2024-03-04T08:57:31.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.675+0000 7fe5057fa700 1 --2- 172.21.15.71:0/3683395273 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe4f00a1f20 0x7fe4f00a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:31.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.675+0000 7fe5057fa700 1 -- 172.21.15.71:0/3683395273 <== mon.1 v2:172.21.15.158:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fe5000ce330 con 0x7fe508107130 2024-03-04T08:57:31.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.675+0000 7fe507fff700 1 --2- 172.21.15.71:0/3683395273 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe4f00a1f20 0x7fe4f00a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:31.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.675+0000 7fe5057fa700 1 --2- 172.21.15.71:0/3683395273 >> [v2:172.21.15.158:6808/1922694045,v1:172.21.15.158:6809/1922694045] conn(0x7fe4f00ad080 0x7fe4f00af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:31.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.675+0000 7fe5057fa700 1 -- 172.21.15.71:0/3683395273 --> [v2:172.21.15.158:6808/1922694045,v1:172.21.15.158:6809/1922694045] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe4f00afbf0 con 0x7fe4f00ad080 2024-03-04T08:57:31.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.675+0000 7fe5057fa700 1 -- 172.21.15.71:0/3683395273 <== mon.1 v2:172.21.15.158:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5000ce770 con 0x7fe508107130 2024-03-04T08:57:31.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.675+0000 7fe5077fe700 1 --2- 172.21.15.71:0/3683395273 >> [v2:172.21.15.158:6808/1922694045,v1:172.21.15.158:6809/1922694045] conn(0x7fe4f00ad080 0x7fe4f00af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:31.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.675+0000 7fe507fff700 1 --2- 172.21.15.71:0/3683395273 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe4f00a1f20 0x7fe4f00a43e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8005950 tx=0x7fe4f800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:31.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.676+0000 7fe5077fe700 1 --2- 172.21.15.71:0/3683395273 >> [v2:172.21.15.158:6808/1922694045,v1:172.21.15.158:6809/1922694045] conn(0x7fe4f00ad080 0x7fe4f00af520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:31.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.677+0000 7fe5057fa700 1 -- 172.21.15.71:0/3683395273 <== osd.5 v2:172.21.15.158:6808/1922694045 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe4f00afbf0 con 0x7fe4f00ad080 2024-03-04T08:57:31.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.693+0000 7fe50e6ad700 1 -- 172.21.15.71:0/3683395273 --> [v2:172.21.15.158:6808/1922694045,v1:172.21.15.158:6809/1922694045] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe508105b90 con 0x7fe4f00ad080 2024-03-04T08:57:31.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.693+0000 7fe5057fa700 1 -- 172.21.15.71:0/3683395273 <== osd.5 v2:172.21.15.158:6808/1922694045 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe508105b90 con 0x7fe4f00ad080 2024-03-04T08:57:31.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.694+0000 7fe50e6ad700 1 -- 172.21.15.71:0/3683395273 >> [v2:172.21.15.158:6808/1922694045,v1:172.21.15.158:6809/1922694045] conn(0x7fe4f00ad080 msgr2=0x7fe4f00af520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:31.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.694+0000 7fe50e6ad700 1 --2- 172.21.15.71:0/3683395273 >> [v2:172.21.15.158:6808/1922694045,v1:172.21.15.158:6809/1922694045] conn(0x7fe4f00ad080 0x7fe4f00af520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:31.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.694+0000 7fe50e6ad700 1 -- 172.21.15.71:0/3683395273 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe4f00a1f20 msgr2=0x7fe4f00a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:31.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.694+0000 7fe50e6ad700 1 --2- 172.21.15.71:0/3683395273 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe4f00a1f20 0x7fe4f00a43e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8005950 tx=0x7fe4f800b410 comp rx=0 tx=0).stop 2024-03-04T08:57:31.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.694+0000 7fe50e6ad700 1 -- 172.21.15.71:0/3683395273 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe508107130 msgr2=0x7fe5081a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:31.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.694+0000 7fe50e6ad700 1 --2- 172.21.15.71:0/3683395273 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe508107130 0x7fe5081a3870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe500007ec0 tx=0x7fe50000d3b0 comp rx=0 tx=0).stop 2024-03-04T08:57:31.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.694+0000 7fe50cc4a700 1 -- 172.21.15.71:0/3683395273 reap_dead start 2024-03-04T08:57:31.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.694+0000 7fe50e6ad700 1 -- 172.21.15.71:0/3683395273 shutdown_connections 2024-03-04T08:57:31.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.694+0000 7fe50e6ad700 1 -- 172.21.15.71:0/3683395273 >> 172.21.15.71:0/3683395273 conn(0x7fe5080775c0 msgr2=0x7fe508104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:31.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.694+0000 7fe50e6ad700 1 -- 172.21.15.71:0/3683395273 shutdown_connections 2024-03-04T08:57:31.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:31.694+0000 7fe50e6ad700 1 -- 172.21.15.71:0/3683395273 wait complete. 2024-03-04T08:57:31.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:31 smithi071 ceph-mon[32789]: pgmap v357: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s, 0 objects/s recovering 2024-03-04T08:57:31.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:31 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:57:32.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:31 smithi150 ceph-mon[36807]: pgmap v357: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s, 0 objects/s recovering 2024-03-04T08:57:32.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:31 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:57:32.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:31 smithi158 ceph-mon[37241]: pgmap v357: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s, 0 objects/s recovering 2024-03-04T08:57:32.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:31 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:57:32.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.328+0000 7f018395a700 1 -- 172.21.15.71:0/4285303661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f017c10c7b0 msgr2=0x7f017c10ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:32.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.328+0000 7f018395a700 1 --2- 172.21.15.71:0/4285303661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f017c10c7b0 0x7f017c10ebe0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f017400b0a0 tx=0x7f017400b3b0 comp rx=0 tx=0).stop 2024-03-04T08:57:32.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.328+0000 7f018395a700 1 -- 172.21.15.71:0/4285303661 shutdown_connections 2024-03-04T08:57:32.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.328+0000 7f018395a700 1 --2- 172.21.15.71:0/4285303661 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f017c1033d0 0x7f017c1037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:32.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.328+0000 7f018395a700 1 --2- 172.21.15.71:0/4285303661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f017c10c7b0 0x7f017c10ebe0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:32.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.328+0000 7f018395a700 1 --2- 172.21.15.71:0/4285303661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f017c103cf0 0x7f017c10c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:32.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.328+0000 7f018395a700 1 -- 172.21.15.71:0/4285303661 >> 172.21.15.71:0/4285303661 conn(0x7f017c0780f0 msgr2=0x7f017c078500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:32.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.328+0000 7f018395a700 1 -- 172.21.15.71:0/4285303661 shutdown_connections 2024-03-04T08:57:32.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.328+0000 7f018395a700 1 -- 172.21.15.71:0/4285303661 wait complete. 2024-03-04T08:57:32.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.329+0000 7f018395a700 1 Processor -- start 2024-03-04T08:57:32.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.329+0000 7f018395a700 1 -- start start 2024-03-04T08:57:32.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.329+0000 7f018395a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f017c1033d0 0x7f017c19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:32.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.329+0000 7f018395a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f017c103cf0 0x7f017c19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:32.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.329+0000 7f018395a700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f017c10c7b0 0x7f017c198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:32.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.329+0000 7f018395a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017c112480 con 0x7f017c103cf0 2024-03-04T08:57:32.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.329+0000 7f018395a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017c112300 con 0x7f017c1033d0 2024-03-04T08:57:32.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.329+0000 7f018395a700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017c112600 con 0x7f017c10c7b0 2024-03-04T08:57:32.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.329+0000 7f0180ef5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f017c103cf0 0x7f017c19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:32.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.329+0000 7f0180ef5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f017c103cf0 0x7f017c19eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45456/0 (socket says 172.21.15.71:45456) 2024-03-04T08:57:32.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.329+0000 7f0180ef5700 1 -- 172.21.15.71:0/824151171 learned_addr learned my addr 172.21.15.71:0/824151171 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:57:32.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.329+0000 7f0181ef7700 1 --2- 172.21.15.71:0/824151171 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f017c10c7b0 0x7f017c198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:32.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.329+0000 7f01816f6700 1 --2- 172.21.15.71:0/824151171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f017c1033d0 0x7f017c19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:32.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.329+0000 7f0180ef5700 1 -- 172.21.15.71:0/824151171 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f017c10c7b0 msgr2=0x7f017c198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:32.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.329+0000 7f0180ef5700 1 --2- 172.21.15.71:0/824151171 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f017c10c7b0 0x7f017c198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:32.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.329+0000 7f0180ef5700 1 -- 172.21.15.71:0/824151171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f017c1033d0 msgr2=0x7f017c19e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:32.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.329+0000 7f0180ef5700 1 --2- 172.21.15.71:0/824151171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f017c1033d0 0x7f017c19e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:32.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.329+0000 7f0180ef5700 1 -- 172.21.15.71:0/824151171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f017c199000 con 0x7f017c103cf0 2024-03-04T08:57:32.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.330+0000 7f0180ef5700 1 --2- 172.21.15.71:0/824151171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f017c103cf0 0x7f017c19eb90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f016c00e970 tx=0x7f016c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:32.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.330+0000 7f01727fc700 1 -- 172.21.15.71:0/824151171 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c004a30 con 0x7f017c103cf0 2024-03-04T08:57:32.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.330+0000 7f018395a700 1 -- 172.21.15.71:0/824151171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f017c1992f0 con 0x7f017c103cf0 2024-03-04T08:57:32.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.330+0000 7f01727fc700 1 -- 172.21.15.71:0/824151171 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f016c00ccc0 con 0x7f017c103cf0 2024-03-04T08:57:32.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.330+0000 7f01727fc700 1 -- 172.21.15.71:0/824151171 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c005650 con 0x7f017c103cf0 2024-03-04T08:57:32.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.330+0000 7f018395a700 1 -- 172.21.15.71:0/824151171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f017c1a55d0 con 0x7f017c103cf0 2024-03-04T08:57:32.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.332+0000 7f018395a700 1 -- 172.21.15.71:0/824151171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f017c062790 con 0x7f017c103cf0 2024-03-04T08:57:32.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.333+0000 7f01727fc700 1 -- 172.21.15.71:0/824151171 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f016c004bd0 con 0x7f017c103cf0 2024-03-04T08:57:32.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.333+0000 7f01727fc700 1 --2- 172.21.15.71:0/824151171 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f01680a1ea0 0x7f01680a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:32.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.334+0000 7f01816f6700 1 --2- 172.21.15.71:0/824151171 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f01680a1ea0 0x7f01680a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:32.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.334+0000 7f01727fc700 1 -- 172.21.15.71:0/824151171 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f016c0ce330 con 0x7f017c103cf0 2024-03-04T08:57:32.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.334+0000 7f01727fc700 1 --2- 172.21.15.71:0/824151171 >> [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262] conn(0x7f01680ad000 0x7f01680af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:32.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.334+0000 7f01727fc700 1 -- 172.21.15.71:0/824151171 --> [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f01680afb70 con 0x7f01680ad000 2024-03-04T08:57:32.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.334+0000 7f01727fc700 1 -- 172.21.15.71:0/824151171 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f016c0d5050 con 0x7f017c103cf0 2024-03-04T08:57:32.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.334+0000 7f0181ef7700 1 --2- 172.21.15.71:0/824151171 >> [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262] conn(0x7f01680ad000 0x7f01680af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:32.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.334+0000 7f01816f6700 1 --2- 172.21.15.71:0/824151171 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f01680a1ea0 0x7f01680a4360 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0178009c60 tx=0x7f0178009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:32.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.334+0000 7f0181ef7700 1 --2- 172.21.15.71:0/824151171 >> [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262] conn(0x7f01680ad000 0x7f01680af4a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:32.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.335+0000 7f01727fc700 1 -- 172.21.15.71:0/824151171 <== osd.3 v2:172.21.15.71:6810/1311903262 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f01680afb70 con 0x7f01680ad000 2024-03-04T08:57:32.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.351+0000 7f018395a700 1 -- 172.21.15.71:0/824151171 --> [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f017c062c10 con 0x7f01680ad000 2024-03-04T08:57:32.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.352+0000 7f01727fc700 1 -- 172.21.15.71:0/824151171 <== osd.3 v2:172.21.15.71:6810/1311903262 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f017c062c10 con 0x7f01680ad000 2024-03-04T08:57:32.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.352+0000 7f018395a700 1 -- 172.21.15.71:0/824151171 >> [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262] conn(0x7f01680ad000 msgr2=0x7f01680af4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:32.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.352+0000 7f018395a700 1 --2- 172.21.15.71:0/824151171 >> [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262] conn(0x7f01680ad000 0x7f01680af4a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:32.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.352+0000 7f018395a700 1 -- 172.21.15.71:0/824151171 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f01680a1ea0 msgr2=0x7f01680a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:32.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.352+0000 7f018395a700 1 --2- 172.21.15.71:0/824151171 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f01680a1ea0 0x7f01680a4360 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0178009c60 tx=0x7f0178009260 comp rx=0 tx=0).stop 2024-03-04T08:57:32.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.352+0000 7f018395a700 1 -- 172.21.15.71:0/824151171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f017c103cf0 msgr2=0x7f017c19eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:32.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.352+0000 7f018395a700 1 --2- 172.21.15.71:0/824151171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f017c103cf0 0x7f017c19eb90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f016c00e970 tx=0x7f016c00ec80 comp rx=0 tx=0).stop 2024-03-04T08:57:32.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.352+0000 7f0181ef7700 1 -- 172.21.15.71:0/824151171 reap_dead start 2024-03-04T08:57:32.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.353+0000 7f018395a700 1 -- 172.21.15.71:0/824151171 shutdown_connections 2024-03-04T08:57:32.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.353+0000 7f018395a700 1 -- 172.21.15.71:0/824151171 >> 172.21.15.71:0/824151171 conn(0x7f017c0780f0 msgr2=0x7f017c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:32.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.353+0000 7f018395a700 1 -- 172.21.15.71:0/824151171 shutdown_connections 2024-03-04T08:57:32.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:32.353+0000 7f018395a700 1 -- 172.21.15.71:0/824151171 wait complete. 2024-03-04T08:57:32.823 INFO:teuthology.orchestra.run.smithi071.stdout:261993005102 2024-03-04T08:57:32.823 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-04T08:57:33.243 INFO:teuthology.orchestra.run.smithi071.stdout:34359738459 2024-03-04T08:57:33.243 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-04T08:57:33.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:33 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:57:33.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:33 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:57:33.779 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:33 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:57:33.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:33 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:57:33.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:33 smithi150 ceph-mon[36807]: pgmap v358: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 517 B/s rd, 1 op/s; 6.3 KiB/s, 0 objects/s recovering 2024-03-04T08:57:33.785 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:33 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:57:33.785 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:33 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:57:33.785 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:33 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:57:33.785 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:33 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:57:33.785 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:33 smithi071 ceph-mon[32789]: pgmap v358: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 517 B/s rd, 1 op/s; 6.3 KiB/s, 0 objects/s recovering 2024-03-04T08:57:33.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:33 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:57:33.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:33 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:57:33.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:33 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:57:33.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:33 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:57:33.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:33 smithi158 ceph-mon[37241]: pgmap v358: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 517 B/s rd, 1 op/s; 6.3 KiB/s, 0 objects/s recovering 2024-03-04T08:57:34.510 INFO:teuthology.orchestra.run.smithi071.stdout:111669149768 2024-03-04T08:57:34.510 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-04T08:57:34.830 INFO:teuthology.orchestra.run.smithi071.stdout:77309411406 2024-03-04T08:57:34.830 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-04T08:57:35.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.389+0000 7fa0c9014700 1 -- 172.21.15.71:0/1859327344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0c41006b0 msgr2=0x7fa0c4100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:35.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.389+0000 7fa0c9014700 1 --2- 172.21.15.71:0/1859327344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0c41006b0 0x7fa0c4100b30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009990 tx=0x7fa0b8009ca0 comp rx=0 tx=0).stop 2024-03-04T08:57:35.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.389+0000 7fa0c9014700 1 -- 172.21.15.71:0/1859327344 shutdown_connections 2024-03-04T08:57:35.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.389+0000 7fa0c9014700 1 --2- 172.21.15.71:0/1859327344 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa0c4107130 0x7fa0c4107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:35.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.389+0000 7fa0c9014700 1 --2- 172.21.15.71:0/1859327344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0c4101070 0x7fa0c410efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:35.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.389+0000 7fa0c9014700 1 --2- 172.21.15.71:0/1859327344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0c41006b0 0x7fa0c4100b30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:35.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.389+0000 7fa0c9014700 1 -- 172.21.15.71:0/1859327344 >> 172.21.15.71:0/1859327344 conn(0x7fa0c40775c0 msgr2=0x7fa0c40fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:35.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.389+0000 7fa0c9014700 1 -- 172.21.15.71:0/1859327344 shutdown_connections 2024-03-04T08:57:35.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.389+0000 7fa0c9014700 1 -- 172.21.15.71:0/1859327344 wait complete. 2024-03-04T08:57:35.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.389+0000 7fa0c9014700 1 Processor -- start 2024-03-04T08:57:35.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.390+0000 7fa0c9014700 1 -- start start 2024-03-04T08:57:35.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.390+0000 7fa0c9014700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0c41006b0 0x7fa0c419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:35.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.390+0000 7fa0c9014700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa0c4101070 0x7fa0c419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:35.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.390+0000 7fa0c9014700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0c4107130 0x7fa0c4198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:35.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.390+0000 7fa0c9014700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c4112510 con 0x7fa0c4107130 2024-03-04T08:57:35.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.390+0000 7fa0c9014700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c4112390 con 0x7fa0c41006b0 2024-03-04T08:57:35.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.390+0000 7fa0c9014700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c4112690 con 0x7fa0c4101070 2024-03-04T08:57:35.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.390+0000 7fa0c359e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0c4107130 0x7fa0c4198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:35.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.390+0000 7fa0c2d9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0c41006b0 0x7fa0c419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:35.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.390+0000 7fa0c359e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0c4107130 0x7fa0c4198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45504/0 (socket says 172.21.15.71:45504) 2024-03-04T08:57:35.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.390+0000 7fa0c359e700 1 -- 172.21.15.71:0/3258799655 learned_addr learned my addr 172.21.15.71:0/3258799655 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:57:35.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.390+0000 7fa0c259c700 1 --2- 172.21.15.71:0/3258799655 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa0c4101070 0x7fa0c419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:35.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.390+0000 7fa0c2d9d700 1 -- 172.21.15.71:0/3258799655 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa0c4101070 msgr2=0x7fa0c419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:35.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.390+0000 7fa0c2d9d700 1 --2- 172.21.15.71:0/3258799655 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa0c4101070 0x7fa0c419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:35.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.390+0000 7fa0c2d9d700 1 -- 172.21.15.71:0/3258799655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0c4107130 msgr2=0x7fa0c4198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:35.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.390+0000 7fa0c2d9d700 1 --2- 172.21.15.71:0/3258799655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0c4107130 0x7fa0c4198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:35.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.390+0000 7fa0c2d9d700 1 -- 172.21.15.71:0/3258799655 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c4199030 con 0x7fa0c41006b0 2024-03-04T08:57:35.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.390+0000 7fa0c359e700 1 --2- 172.21.15.71:0/3258799655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0c4107130 0x7fa0c4198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:57:35.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.391+0000 7fa0c2d9d700 1 --2- 172.21.15.71:0/3258799655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0c41006b0 0x7fa0c419e630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400e9f0 tx=0x7fa0b400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:35.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.391+0000 7fa0abfff700 1 -- 172.21.15.71:0/3258799655 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b400cbd0 con 0x7fa0c41006b0 2024-03-04T08:57:35.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.391+0000 7fa0abfff700 1 -- 172.21.15.71:0/3258799655 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0b4004500 con 0x7fa0c41006b0 2024-03-04T08:57:35.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.391+0000 7fa0c9014700 1 -- 172.21.15.71:0/3258799655 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c4199320 con 0x7fa0c41006b0 2024-03-04T08:57:35.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.391+0000 7fa0abfff700 1 -- 172.21.15.71:0/3258799655 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4005280 con 0x7fa0c41006b0 2024-03-04T08:57:35.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.391+0000 7fa0c9014700 1 -- 172.21.15.71:0/3258799655 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c41a5490 con 0x7fa0c41006b0 2024-03-04T08:57:35.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.392+0000 7fa0c9014700 1 -- 172.21.15.71:0/3258799655 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa0c40627d0 con 0x7fa0c41006b0 2024-03-04T08:57:35.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.393+0000 7fa0abfff700 1 -- 172.21.15.71:0/3258799655 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fa0b4005420 con 0x7fa0c41006b0 2024-03-04T08:57:35.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.393+0000 7fa0abfff700 1 --2- 172.21.15.71:0/3258799655 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa0ac0a1ea0 0x7fa0ac0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:35.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.393+0000 7fa0c259c700 1 --2- 172.21.15.71:0/3258799655 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa0ac0a1ea0 0x7fa0ac0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:35.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.393+0000 7fa0abfff700 1 -- 172.21.15.71:0/3258799655 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fa0b40ce490 con 0x7fa0c41006b0 2024-03-04T08:57:35.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.394+0000 7fa0abfff700 1 --2- 172.21.15.71:0/3258799655 >> [v2:172.21.15.150:6808/2300837573,v1:172.21.15.150:6809/2300837573] conn(0x7fa0ac0ad000 0x7fa0ac0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:35.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.394+0000 7fa0abfff700 1 -- 172.21.15.71:0/3258799655 --> [v2:172.21.15.150:6808/2300837573,v1:172.21.15.150:6809/2300837573] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0b4019e50 con 0x7fa0ac0ad000 2024-03-04T08:57:35.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.394+0000 7fa0abfff700 1 -- 172.21.15.71:0/3258799655 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0b40ce8d0 con 0x7fa0c41006b0 2024-03-04T08:57:35.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.394+0000 7fa0c259c700 1 --2- 172.21.15.71:0/3258799655 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa0ac0a1ea0 0x7fa0ac0a4360 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8006010 tx=0x7fa0b801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:35.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.394+0000 7fa0c359e700 1 --2- 172.21.15.71:0/3258799655 >> [v2:172.21.15.150:6808/2300837573,v1:172.21.15.150:6809/2300837573] conn(0x7fa0ac0ad000 0x7fa0ac0af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:35.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.394+0000 7fa0c359e700 1 --2- 172.21.15.71:0/3258799655 >> [v2:172.21.15.150:6808/2300837573,v1:172.21.15.150:6809/2300837573] conn(0x7fa0ac0ad000 0x7fa0ac0af4a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:35.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.395+0000 7fa0abfff700 1 -- 172.21.15.71:0/3258799655 <== osd.4 v2:172.21.15.150:6808/2300837573 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa0b4019e50 con 0x7fa0ac0ad000 2024-03-04T08:57:35.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.411+0000 7fa0c9014700 1 -- 172.21.15.71:0/3258799655 --> [v2:172.21.15.150:6808/2300837573,v1:172.21.15.150:6809/2300837573] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa0c4062bd0 con 0x7fa0ac0ad000 2024-03-04T08:57:35.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.411+0000 7fa0abfff700 1 -- 172.21.15.71:0/3258799655 <== osd.4 v2:172.21.15.150:6808/2300837573 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa0c4062bd0 con 0x7fa0ac0ad000 2024-03-04T08:57:35.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.412+0000 7fa0c9014700 1 -- 172.21.15.71:0/3258799655 >> [v2:172.21.15.150:6808/2300837573,v1:172.21.15.150:6809/2300837573] conn(0x7fa0ac0ad000 msgr2=0x7fa0ac0af4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:35.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.412+0000 7fa0c9014700 1 --2- 172.21.15.71:0/3258799655 >> [v2:172.21.15.150:6808/2300837573,v1:172.21.15.150:6809/2300837573] conn(0x7fa0ac0ad000 0x7fa0ac0af4a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:35.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.412+0000 7fa0c9014700 1 -- 172.21.15.71:0/3258799655 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa0ac0a1ea0 msgr2=0x7fa0ac0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:35.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.412+0000 7fa0c9014700 1 --2- 172.21.15.71:0/3258799655 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa0ac0a1ea0 0x7fa0ac0a4360 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8006010 tx=0x7fa0b801a040 comp rx=0 tx=0).stop 2024-03-04T08:57:35.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.412+0000 7fa0c9014700 1 -- 172.21.15.71:0/3258799655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0c41006b0 msgr2=0x7fa0c419e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:35.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.412+0000 7fa0c9014700 1 --2- 172.21.15.71:0/3258799655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0c41006b0 0x7fa0c419e630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400e9f0 tx=0x7fa0b400ed00 comp rx=0 tx=0).stop 2024-03-04T08:57:35.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.412+0000 7fa0c359e700 1 -- 172.21.15.71:0/3258799655 reap_dead start 2024-03-04T08:57:35.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.412+0000 7fa0c9014700 1 -- 172.21.15.71:0/3258799655 shutdown_connections 2024-03-04T08:57:35.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.412+0000 7fa0c9014700 1 -- 172.21.15.71:0/3258799655 >> 172.21.15.71:0/3258799655 conn(0x7fa0c40775c0 msgr2=0x7fa0c41051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:35.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.412+0000 7fa0c9014700 1 -- 172.21.15.71:0/3258799655 shutdown_connections 2024-03-04T08:57:35.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.412+0000 7fa0c9014700 1 -- 172.21.15.71:0/3258799655 wait complete. 2024-03-04T08:57:35.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.269+0000 7fb3d54fd700 1 -- 172.21.15.71:0/1232310729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3d0108110 msgr2=0x7fb3d01084f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:35.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.269+0000 7fb3d54fd700 1 --2- 172.21.15.71:0/1232310729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3d0108110 0x7fb3d01084f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000b0a0 tx=0x7fb3c000b3b0 comp rx=0 tx=0).stop 2024-03-04T08:57:35.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.269+0000 7fb3d54fd700 1 -- 172.21.15.71:0/1232310729 shutdown_connections 2024-03-04T08:57:35.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.269+0000 7fb3d54fd700 1 --2- 172.21.15.71:0/1232310729 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb3d006a710 0x7fb3d010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:35.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.269+0000 7fb3d54fd700 1 --2- 172.21.15.71:0/1232310729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3d0108110 0x7fb3d01084f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:35.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.269+0000 7fb3d54fd700 1 --2- 172.21.15.71:0/1232310729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3d0069d70 0x7fb3d006a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:35.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.269+0000 7fb3d54fd700 1 -- 172.21.15.71:0/1232310729 >> 172.21.15.71:0/1232310729 conn(0x7fb3d0077ef0 msgr2=0x7fb3d0078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:35.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.269+0000 7fb3d54fd700 1 -- 172.21.15.71:0/1232310729 shutdown_connections 2024-03-04T08:57:35.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.269+0000 7fb3d54fd700 1 -- 172.21.15.71:0/1232310729 wait complete. 2024-03-04T08:57:35.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.270+0000 7fb3d54fd700 1 Processor -- start 2024-03-04T08:57:35.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.270+0000 7fb3d54fd700 1 -- start start 2024-03-04T08:57:35.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.270+0000 7fb3d54fd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3d0069d70 0x7fb3d019e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:35.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.270+0000 7fb3d54fd700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb3d006a710 0x7fb3d019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:35.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.270+0000 7fb3d54fd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3d0108110 0x7fb3d01986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:35.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.270+0000 7fb3d54fd700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d0112370 con 0x7fb3d0069d70 2024-03-04T08:57:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.270+0000 7fb3d54fd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d01121f0 con 0x7fb3d0108110 2024-03-04T08:57:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.270+0000 7fb3d54fd700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d01124f0 con 0x7fb3d006a710 2024-03-04T08:57:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.270+0000 7fb3ceffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3d0069d70 0x7fb3d019e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.270+0000 7fb3ceffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3d0069d70 0x7fb3d019e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45474/0 (socket says 172.21.15.71:45474) 2024-03-04T08:57:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.270+0000 7fb3ceffd700 1 -- 172.21.15.71:0/1829584708 learned_addr learned my addr 172.21.15.71:0/1829584708 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:57:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.270+0000 7fb3cf7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3d0108110 0x7fb3d01986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.271+0000 7fb3ce7fc700 1 --2- 172.21.15.71:0/1829584708 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb3d006a710 0x7fb3d019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.271+0000 7fb3cf7fe700 1 -- 172.21.15.71:0/1829584708 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb3d006a710 msgr2=0x7fb3d019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:35.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.271+0000 7fb3cf7fe700 1 --2- 172.21.15.71:0/1829584708 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb3d006a710 0x7fb3d019ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:35.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.271+0000 7fb3cf7fe700 1 -- 172.21.15.71:0/1829584708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3d0069d70 msgr2=0x7fb3d019e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:35.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.271+0000 7fb3cf7fe700 1 --2- 172.21.15.71:0/1829584708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3d0069d70 0x7fb3d019e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:35.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.271+0000 7fb3cf7fe700 1 -- 172.21.15.71:0/1829584708 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d0198f90 con 0x7fb3d0108110 2024-03-04T08:57:35.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.271+0000 7fb3ceffd700 1 --2- 172.21.15.71:0/1829584708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3d0069d70 0x7fb3d019e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:57:35.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.271+0000 7fb3ce7fc700 1 --2- 172.21.15.71:0/1829584708 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb3d006a710 0x7fb3d019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:57:35.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.271+0000 7fb3cf7fe700 1 --2- 172.21.15.71:0/1829584708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3d0108110 0x7fb3d01986b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800d720 tx=0x7fb3c800da30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:35.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.271+0000 7fb3c7fff700 1 -- 172.21.15.71:0/1829584708 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8016070 con 0x7fb3d0108110 2024-03-04T08:57:35.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.271+0000 7fb3d54fd700 1 -- 172.21.15.71:0/1829584708 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d0199280 con 0x7fb3d0108110 2024-03-04T08:57:35.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.271+0000 7fb3c7fff700 1 -- 172.21.15.71:0/1829584708 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb3c8004d10 con 0x7fb3d0108110 2024-03-04T08:57:35.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.271+0000 7fb3c7fff700 1 -- 172.21.15.71:0/1829584708 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8005680 con 0x7fb3d0108110 2024-03-04T08:57:35.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.271+0000 7fb3d54fd700 1 -- 172.21.15.71:0/1829584708 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d01a5510 con 0x7fb3d0108110 2024-03-04T08:57:35.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.272+0000 7fb3d54fd700 1 -- 172.21.15.71:0/1829584708 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb3d0062810 con 0x7fb3d0108110 2024-03-04T08:57:35.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.273+0000 7fb3c7fff700 1 -- 172.21.15.71:0/1829584708 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fb3c8004750 con 0x7fb3d0108110 2024-03-04T08:57:35.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.274+0000 7fb3c7fff700 1 --2- 172.21.15.71:0/1829584708 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb3bc0a1ea0 0x7fb3bc0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:35.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.274+0000 7fb3ceffd700 1 --2- 172.21.15.71:0/1829584708 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb3bc0a1ea0 0x7fb3bc0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:35.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.274+0000 7fb3c7fff700 1 -- 172.21.15.71:0/1829584708 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fb3c80ce5f0 con 0x7fb3d0108110 2024-03-04T08:57:35.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.274+0000 7fb3c7fff700 1 --2- 172.21.15.71:0/1829584708 >> [v2:172.21.15.150:6824/3295420996,v1:172.21.15.150:6825/3295420996] conn(0x7fb3bc0ad000 0x7fb3bc0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:35.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.274+0000 7fb3c7fff700 1 -- 172.21.15.71:0/1829584708 --> [v2:172.21.15.150:6824/3295420996,v1:172.21.15.150:6825/3295420996] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb3c8001d30 con 0x7fb3bc0ad000 2024-03-04T08:57:35.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.274+0000 7fb3c7fff700 1 -- 172.21.15.71:0/1829584708 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3c80ce9a0 con 0x7fb3d0108110 2024-03-04T08:57:35.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.274+0000 7fb3ceffd700 1 --2- 172.21.15.71:0/1829584708 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb3bc0a1ea0 0x7fb3bc0a4360 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009e60 tx=0x7fb3c000b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:35.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.274+0000 7fb3ce7fc700 1 --2- 172.21.15.71:0/1829584708 >> [v2:172.21.15.150:6824/3295420996,v1:172.21.15.150:6825/3295420996] conn(0x7fb3bc0ad000 0x7fb3bc0af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:35.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.274+0000 7fb3ce7fc700 1 --2- 172.21.15.71:0/1829584708 >> [v2:172.21.15.150:6824/3295420996,v1:172.21.15.150:6825/3295420996] conn(0x7fb3bc0ad000 0x7fb3bc0af4a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:35.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.275+0000 7fb3c7fff700 1 -- 172.21.15.71:0/1829584708 <== osd.10 v2:172.21.15.150:6824/3295420996 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb3c8001d30 con 0x7fb3bc0ad000 2024-03-04T08:57:35.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.291+0000 7fb3d54fd700 1 -- 172.21.15.71:0/1829584708 --> [v2:172.21.15.150:6824/3295420996,v1:172.21.15.150:6825/3295420996] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb3d0062c90 con 0x7fb3bc0ad000 2024-03-04T08:57:35.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.292+0000 7fb3c7fff700 1 -- 172.21.15.71:0/1829584708 <== osd.10 v2:172.21.15.150:6824/3295420996 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb3d0062c90 con 0x7fb3bc0ad000 2024-03-04T08:57:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.292+0000 7fb3d54fd700 1 -- 172.21.15.71:0/1829584708 >> [v2:172.21.15.150:6824/3295420996,v1:172.21.15.150:6825/3295420996] conn(0x7fb3bc0ad000 msgr2=0x7fb3bc0af4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.292+0000 7fb3d54fd700 1 --2- 172.21.15.71:0/1829584708 >> [v2:172.21.15.150:6824/3295420996,v1:172.21.15.150:6825/3295420996] conn(0x7fb3bc0ad000 0x7fb3bc0af4a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.292+0000 7fb3d54fd700 1 -- 172.21.15.71:0/1829584708 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb3bc0a1ea0 msgr2=0x7fb3bc0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.292+0000 7fb3d54fd700 1 --2- 172.21.15.71:0/1829584708 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb3bc0a1ea0 0x7fb3bc0a4360 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009e60 tx=0x7fb3c000b700 comp rx=0 tx=0).stop 2024-03-04T08:57:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.292+0000 7fb3d54fd700 1 -- 172.21.15.71:0/1829584708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3d0108110 msgr2=0x7fb3d01986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.292+0000 7fb3d54fd700 1 --2- 172.21.15.71:0/1829584708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3d0108110 0x7fb3d01986b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800d720 tx=0x7fb3c800da30 comp rx=0 tx=0).stop 2024-03-04T08:57:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.292+0000 7fb3cf7fe700 1 -- 172.21.15.71:0/1829584708 reap_dead start 2024-03-04T08:57:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.292+0000 7fb3d54fd700 1 -- 172.21.15.71:0/1829584708 shutdown_connections 2024-03-04T08:57:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.292+0000 7fb3d54fd700 1 -- 172.21.15.71:0/1829584708 >> 172.21.15.71:0/1829584708 conn(0x7fb3d0077ef0 msgr2=0x7fb3d00ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.293+0000 7fb3d54fd700 1 -- 172.21.15.71:0/1829584708 shutdown_connections 2024-03-04T08:57:35.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:35.293+0000 7fb3d54fd700 1 -- 172.21.15.71:0/1829584708 wait complete. 2024-03-04T08:57:36.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:35 smithi150 ceph-mon[36807]: pgmap v359: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 470 B/s rd, 0 op/s; 5.7 KiB/s, 0 objects/s recovering 2024-03-04T08:57:36.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:35 smithi158 ceph-mon[37241]: pgmap v359: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 470 B/s rd, 0 op/s; 5.7 KiB/s, 0 objects/s recovering 2024-03-04T08:57:36.054 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:35 smithi071 ceph-mon[32789]: pgmap v359: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 470 B/s rd, 0 op/s; 5.7 KiB/s, 0 objects/s recovering 2024-03-04T08:57:36.393 INFO:teuthology.orchestra.run.smithi071.stdout:317827579936 2024-03-04T08:57:36.942 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-04T08:57:36.943 INFO:teuthology.orchestra.run.smithi071.stdout:167503724606 2024-03-04T08:57:36.943 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-04T08:57:37.277 INFO:teuthology.orchestra.run.smithi071.stdout:146028888132 2024-03-04T08:57:37.277 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-04T08:57:37.320 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:37.320 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:37.321 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:37.321 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:37.395 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:38.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:37 smithi150 ceph-mon[36807]: pgmap v360: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 5.2 KiB/s, 0 objects/s recovering 2024-03-04T08:57:38.049 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:37 smithi071 ceph-mon[32789]: pgmap v360: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 5.2 KiB/s, 0 objects/s recovering 2024-03-04T08:57:38.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:37 smithi158 ceph-mon[37241]: pgmap v360: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s; 5.2 KiB/s, 0 objects/s recovering 2024-03-04T08:57:39.053 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:39.053 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:40.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:39 smithi150 ceph-mon[36807]: pgmap v361: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:57:40.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:39 smithi158 ceph-mon[37241]: pgmap v361: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:57:40.099 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:39 smithi071 ceph-mon[32789]: pgmap v361: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:57:41.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:41 smithi071 ceph-mon[32789]: pgmap v362: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:57:42.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:41 smithi150 ceph-mon[36807]: pgmap v362: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:57:42.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:41 smithi158 ceph-mon[37241]: pgmap v362: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:57:44.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:43 smithi150 ceph-mon[36807]: pgmap v363: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:57:44.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:43 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2021406515' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-04T08:57:44.039 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:43 smithi071 ceph-mon[32789]: pgmap v363: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:57:44.039 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:43 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2021406515' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-04T08:57:44.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:43 smithi158 ceph-mon[37241]: pgmap v363: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:57:44.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:43 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2021406515' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-04T08:57:44.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.556+0000 7fada5515700 1 -- 172.21.15.71:0/3898271052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fada006a810 msgr2=0x7fada006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:44.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.556+0000 7fada5515700 1 --2- 172.21.15.71:0/3898271052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fada006a810 0x7fada006ac90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fad940099c0 tx=0x7fad94009cd0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.556+0000 7fada5515700 1 -- 172.21.15.71:0/3898271052 shutdown_connections 2024-03-04T08:57:44.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.556+0000 7fada5515700 1 --2- 172.21.15.71:0/3898271052 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fada01080f0 0x7fada01084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.556+0000 7fada5515700 1 --2- 172.21.15.71:0/3898271052 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fada006b1d0 0x7fada010ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.556+0000 7fada5515700 1 --2- 172.21.15.71:0/3898271052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fada006a810 0x7fada006ac90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.556+0000 7fada5515700 1 -- 172.21.15.71:0/3898271052 >> 172.21.15.71:0/3898271052 conn(0x7fada0077fd0 msgr2=0x7fada00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:44.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.556+0000 7fada5515700 1 -- 172.21.15.71:0/3898271052 shutdown_connections 2024-03-04T08:57:44.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.556+0000 7fada5515700 1 -- 172.21.15.71:0/3898271052 wait complete. 2024-03-04T08:57:44.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.557+0000 7fada5515700 1 Processor -- start 2024-03-04T08:57:44.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.557+0000 7fada5515700 1 -- start start 2024-03-04T08:57:44.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.557+0000 7fada5515700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fada006a810 0x7fada019e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:44.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.557+0000 7fada5515700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fada006b1d0 0x7fada019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:44.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.557+0000 7fada5515700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fada01080f0 0x7fada01985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:44.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.557+0000 7fada5515700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada0112370 con 0x7fada006a810 2024-03-04T08:57:44.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.557+0000 7fada5515700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada01121f0 con 0x7fada006b1d0 2024-03-04T08:57:44.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.557+0000 7fada5515700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada01124f0 con 0x7fada01080f0 2024-03-04T08:57:44.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.557+0000 7fad9effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fada006a810 0x7fada019e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:44.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.557+0000 7fad9effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fada006a810 0x7fada019e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59616/0 (socket says 172.21.15.71:59616) 2024-03-04T08:57:44.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.557+0000 7fad9effd700 1 -- 172.21.15.71:0/344340357 learned_addr learned my addr 172.21.15.71:0/344340357 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:57:44.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.557+0000 7fad9e7fc700 1 --2- 172.21.15.71:0/344340357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fada006b1d0 0x7fada019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:44.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.557+0000 7fad9f7fe700 1 --2- 172.21.15.71:0/344340357 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fada01080f0 0x7fada01985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:44.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.558+0000 7fad9effd700 1 -- 172.21.15.71:0/344340357 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fada01080f0 msgr2=0x7fada01985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:44.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.558+0000 7fad9effd700 1 --2- 172.21.15.71:0/344340357 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fada01080f0 0x7fada01985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.558+0000 7fad9effd700 1 -- 172.21.15.71:0/344340357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fada006b1d0 msgr2=0x7fada019e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:44.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.558+0000 7fad9effd700 1 --2- 172.21.15.71:0/344340357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fada006b1d0 0x7fada019e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.558+0000 7fad9effd700 1 -- 172.21.15.71:0/344340357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada0198d50 con 0x7fada006a810 2024-03-04T08:57:44.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.558+0000 7fad9e7fc700 1 --2- 172.21.15.71:0/344340357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fada006b1d0 0x7fada019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:57:44.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.558+0000 7fad9effd700 1 --2- 172.21.15.71:0/344340357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fada006a810 0x7fada019e430 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fad9000d950 tx=0x7fad9000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:44.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.558+0000 7fad87fff700 1 -- 172.21.15.71:0/344340357 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90016070 con 0x7fada006a810 2024-03-04T08:57:44.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.558+0000 7fad87fff700 1 -- 172.21.15.71:0/344340357 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad90009e20 con 0x7fada006a810 2024-03-04T08:57:44.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.558+0000 7fad87fff700 1 -- 172.21.15.71:0/344340357 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9000f620 con 0x7fada006a810 2024-03-04T08:57:44.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.558+0000 7fada5515700 1 -- 172.21.15.71:0/344340357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada0199040 con 0x7fada006a810 2024-03-04T08:57:44.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.558+0000 7fada5515700 1 -- 172.21.15.71:0/344340357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada0100fa0 con 0x7fada006a810 2024-03-04T08:57:44.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.559+0000 7fada5515700 1 -- 172.21.15.71:0/344340357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fada0068d80 con 0x7fada006a810 2024-03-04T08:57:44.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.560+0000 7fad87fff700 1 -- 172.21.15.71:0/344340357 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fad90019070 con 0x7fada006a810 2024-03-04T08:57:44.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.560+0000 7fad87fff700 1 --2- 172.21.15.71:0/344340357 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fad880a1f20 0x7fad880a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:44.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.561+0000 7fad9e7fc700 1 --2- 172.21.15.71:0/344340357 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fad880a1f20 0x7fad880a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:44.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.561+0000 7fad87fff700 1 -- 172.21.15.71:0/344340357 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fad900cdd70 con 0x7fada006a810 2024-03-04T08:57:44.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.561+0000 7fad9e7fc700 1 --2- 172.21.15.71:0/344340357 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fad880a1f20 0x7fad880a43e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fada00785d0 tx=0x7fad94005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:44.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.564+0000 7fad87fff700 1 -- 172.21.15.71:0/344340357 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fad9009ace0 con 0x7fada006a810 2024-03-04T08:57:44.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.769+0000 7fada5515700 1 -- 172.21.15.71:0/344340357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fada002d9b0 con 0x7fada006a810 2024-03-04T08:57:44.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.771+0000 7fad87fff700 1 -- 172.21.15.71:0/344340357 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fad9009a5e0 con 0x7fada006a810 2024-03-04T08:57:44.767 INFO:teuthology.orchestra.run.smithi071.stdout:296352743464 2024-03-04T08:57:44.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.773+0000 7fada5515700 1 -- 172.21.15.71:0/344340357 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fad880a1f20 msgr2=0x7fad880a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:44.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.773+0000 7fada5515700 1 --2- 172.21.15.71:0/344340357 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fad880a1f20 0x7fad880a43e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fada00785d0 tx=0x7fad94005fb0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.773+0000 7fada5515700 1 -- 172.21.15.71:0/344340357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fada006a810 msgr2=0x7fada019e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:44.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.773+0000 7fada5515700 1 --2- 172.21.15.71:0/344340357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fada006a810 0x7fada019e430 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fad9000d950 tx=0x7fad9000dc60 comp rx=0 tx=0).stop 2024-03-04T08:57:44.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.773+0000 7fada5515700 1 -- 172.21.15.71:0/344340357 shutdown_connections 2024-03-04T08:57:44.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.773+0000 7fada5515700 1 --2- 172.21.15.71:0/344340357 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fad880a1f20 0x7fad880a43e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.773+0000 7fada5515700 1 --2- 172.21.15.71:0/344340357 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fada01080f0 0x7fada01985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.773+0000 7fada5515700 1 --2- 172.21.15.71:0/344340357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fada006a810 0x7fada019e430 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.773+0000 7fada5515700 1 --2- 172.21.15.71:0/344340357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fada006b1d0 0x7fada019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.773+0000 7fada5515700 1 -- 172.21.15.71:0/344340357 >> 172.21.15.71:0/344340357 conn(0x7fada0077fd0 msgr2=0x7fada00ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:44.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.773+0000 7fada5515700 1 -- 172.21.15.71:0/344340357 shutdown_connections 2024-03-04T08:57:44.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.773+0000 7fada5515700 1 -- 172.21.15.71:0/344340357 wait complete. 2024-03-04T08:57:44.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.776+0000 7f20b9320700 1 -- 172.21.15.71:0/3878616122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20b4101c00 msgr2=0x7f20b4105bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:44.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.776+0000 7f20b9320700 1 --2- 172.21.15.71:0/3878616122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20b4101c00 0x7f20b4105bc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f20a80099c0 tx=0x7f20a8009cd0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.776+0000 7f20b9320700 1 -- 172.21.15.71:0/3878616122 shutdown_connections 2024-03-04T08:57:44.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.776+0000 7f20b9320700 1 --2- 172.21.15.71:0/3878616122 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f20b41062a0 0x7f20b410de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.776+0000 7f20b9320700 1 --2- 172.21.15.71:0/3878616122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20b4101250 0x7f20b4101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.776+0000 7f20b9320700 1 --2- 172.21.15.71:0/3878616122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20b4101c00 0x7f20b4105bc0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.776+0000 7f20b9320700 1 -- 172.21.15.71:0/3878616122 >> 172.21.15.71:0/3878616122 conn(0x7f20b4077010 msgr2=0x7f20b4077420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:44.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.776+0000 7f20b9320700 1 -- 172.21.15.71:0/3878616122 shutdown_connections 2024-03-04T08:57:44.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.776+0000 7f20b9320700 1 -- 172.21.15.71:0/3878616122 wait complete. 2024-03-04T08:57:44.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.777+0000 7f20b9320700 1 Processor -- start 2024-03-04T08:57:44.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.777+0000 7f20b9320700 1 -- start start 2024-03-04T08:57:44.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.777+0000 7f20b9320700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f20b4101250 0x7f20b4197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:44.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.777+0000 7f20b9320700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20b4101c00 0x7f20b4197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:44.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.778+0000 7f20b9320700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20b41062a0 0x7f20b419c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:44.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.778+0000 7f20b9320700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b4111280 con 0x7f20b41062a0 2024-03-04T08:57:44.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.778+0000 7f20b9320700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b4111100 con 0x7f20b4101c00 2024-03-04T08:57:44.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.778+0000 7f20b9320700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b4111400 con 0x7f20b4101250 2024-03-04T08:57:44.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.778+0000 7f20b27fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20b4101c00 0x7f20b4197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:44.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.778+0000 7f20b37fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20b41062a0 0x7f20b419c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:44.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.778+0000 7f20b2ffd700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f20b4101250 0x7f20b4197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:44.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.778+0000 7f20b27fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20b4101c00 0x7f20b4197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.71:39650/0 (socket says 172.21.15.71:39650) 2024-03-04T08:57:44.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.778+0000 7f20b27fc700 1 -- 172.21.15.71:0/3151209867 learned_addr learned my addr 172.21.15.71:0/3151209867 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:57:44.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.778+0000 7f20b37fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20b41062a0 0x7f20b419c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59634/0 (socket says 172.21.15.71:59634) 2024-03-04T08:57:44.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.778+0000 7f20b27fc700 1 -- 172.21.15.71:0/3151209867 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f20b4101250 msgr2=0x7f20b4197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:44.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.778+0000 7f20b27fc700 1 --2- 172.21.15.71:0/3151209867 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f20b4101250 0x7f20b4197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.778+0000 7f20b27fc700 1 -- 172.21.15.71:0/3151209867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20b41062a0 msgr2=0x7f20b419c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:44.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.778+0000 7f20b27fc700 1 --2- 172.21.15.71:0/3151209867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20b41062a0 0x7f20b419c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.778+0000 7f20b27fc700 1 -- 172.21.15.71:0/3151209867 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b419c760 con 0x7f20b4101c00 2024-03-04T08:57:44.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.778+0000 7f20b27fc700 1 --2- 172.21.15.71:0/3151209867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20b4101c00 0x7f20b4197c90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f20a800fa00 tx=0x7f20a800fa30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:44.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.778+0000 7f209bfff700 1 -- 172.21.15.71:0/3151209867 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8027070 con 0x7f20b4101c00 2024-03-04T08:57:44.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.778+0000 7f209bfff700 1 -- 172.21.15.71:0/3151209867 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20a8003680 con 0x7f20b4101c00 2024-03-04T08:57:44.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.778+0000 7f209bfff700 1 -- 172.21.15.71:0/3151209867 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a80173f0 con 0x7f20b4101c00 2024-03-04T08:57:44.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.778+0000 7f20b9320700 1 -- 172.21.15.71:0/3151209867 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b419c9f0 con 0x7f20b4101c00 2024-03-04T08:57:44.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.779+0000 7f20b9320700 1 -- 172.21.15.71:0/3151209867 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b41a42d0 con 0x7f20b4101c00 2024-03-04T08:57:44.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.780+0000 7f20b9320700 1 -- 172.21.15.71:0/3151209867 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b4068d80 con 0x7f20b4101c00 2024-03-04T08:57:44.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.780+0000 7f209bfff700 1 -- 172.21.15.71:0/3151209867 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f20a802a070 con 0x7f20b4101c00 2024-03-04T08:57:44.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.781+0000 7f209bfff700 1 --2- 172.21.15.71:0/3151209867 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f209c0a1f20 0x7f209c0a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:44.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.781+0000 7f20b2ffd700 1 --2- 172.21.15.71:0/3151209867 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f209c0a1f20 0x7f209c0a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:44.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.781+0000 7f209bfff700 1 -- 172.21.15.71:0/3151209867 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f20a80cee50 con 0x7f20b4101c00 2024-03-04T08:57:44.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.782+0000 7f20b2ffd700 1 --2- 172.21.15.71:0/3151209867 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f209c0a1f20 0x7f209c0a43e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009c60 tx=0x7f20a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:44.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.786+0000 7f209bfff700 1 -- 172.21.15.71:0/3151209867 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f20a805e280 con 0x7f20b4101c00 2024-03-04T08:57:44.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.988+0000 7f20b9320700 1 -- 172.21.15.71:0/3151209867 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f20b402d9b0 con 0x7f20b4101c00 2024-03-04T08:57:44.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.989+0000 7f209bfff700 1 -- 172.21.15.71:0/3151209867 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f20a8013070 con 0x7f20b4101c00 2024-03-04T08:57:44.985 INFO:teuthology.orchestra.run.smithi071.stdout:167503724609 2024-03-04T08:57:44.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.991+0000 7f20b9320700 1 -- 172.21.15.71:0/3151209867 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f209c0a1f20 msgr2=0x7f209c0a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:44.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.991+0000 7f20b9320700 1 --2- 172.21.15.71:0/3151209867 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f209c0a1f20 0x7f209c0a43e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009c60 tx=0x7f20a4009260 comp rx=0 tx=0).stop 2024-03-04T08:57:44.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.991+0000 7f20b9320700 1 -- 172.21.15.71:0/3151209867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20b4101c00 msgr2=0x7f20b4197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:44.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.991+0000 7f20b9320700 1 --2- 172.21.15.71:0/3151209867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20b4101c00 0x7f20b4197c90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f20a800fa00 tx=0x7f20a800fa30 comp rx=0 tx=0).stop 2024-03-04T08:57:44.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.991+0000 7f20b9320700 1 -- 172.21.15.71:0/3151209867 shutdown_connections 2024-03-04T08:57:44.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.991+0000 7f20b9320700 1 --2- 172.21.15.71:0/3151209867 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f209c0a1f20 0x7f209c0a43e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.991+0000 7f20b9320700 1 --2- 172.21.15.71:0/3151209867 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f20b4101250 0x7f20b4197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.991+0000 7f20b9320700 1 --2- 172.21.15.71:0/3151209867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20b41062a0 0x7f20b419c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.991+0000 7f20b9320700 1 --2- 172.21.15.71:0/3151209867 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20b4101c00 0x7f20b4197c90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.991+0000 7f20b9320700 1 -- 172.21.15.71:0/3151209867 >> 172.21.15.71:0/3151209867 conn(0x7f20b4077010 msgr2=0x7f20b40ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:44.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.991+0000 7f20b9320700 1 -- 172.21.15.71:0/3151209867 shutdown_connections 2024-03-04T08:57:44.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.991+0000 7f20b9320700 1 -- 172.21.15.71:0/3151209867 wait complete. 2024-03-04T08:57:44.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.987+0000 7faa06529700 1 -- 172.21.15.71:0/3515517395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa00107190 msgr2=0x7faa00107570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:44.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.987+0000 7faa06529700 1 --2- 172.21.15.71:0/3515517395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa00107190 0x7faa00107570 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa9f00099c0 tx=0x7fa9f0009cd0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.987+0000 7faa06529700 1 -- 172.21.15.71:0/3515517395 shutdown_connections 2024-03-04T08:57:44.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.987+0000 7faa06529700 1 --2- 172.21.15.71:0/3515517395 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa00101480 0x7faa0010f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.987+0000 7faa06529700 1 --2- 172.21.15.71:0/3515517395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa00107190 0x7faa00107570 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.987+0000 7faa06529700 1 --2- 172.21.15.71:0/3515517395 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faa00100ac0 0x7faa00100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.987+0000 7faa06529700 1 -- 172.21.15.71:0/3515517395 >> 172.21.15.71:0/3515517395 conn(0x7faa00076310 msgr2=0x7faa00076720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:44.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.987+0000 7faa06529700 1 -- 172.21.15.71:0/3515517395 shutdown_connections 2024-03-04T08:57:44.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.987+0000 7faa06529700 1 -- 172.21.15.71:0/3515517395 wait complete. 2024-03-04T08:57:44.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.988+0000 7faa06529700 1 Processor -- start 2024-03-04T08:57:44.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.988+0000 7faa06529700 1 -- start start 2024-03-04T08:57:44.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.988+0000 7faa06529700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faa00100ac0 0x7faa0019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:44.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.988+0000 7faa06529700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa00101480 0x7faa0019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:44.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.988+0000 7faa06529700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa00107190 0x7faa001985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:44.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.988+0000 7faa06529700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa00112420 con 0x7faa00107190 2024-03-04T08:57:44.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.988+0000 7faa06529700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa001122a0 con 0x7faa00101480 2024-03-04T08:57:44.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.988+0000 7faa06529700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa001125a0 con 0x7faa00100ac0 2024-03-04T08:57:44.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.988+0000 7faa04ac6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa00107190 0x7faa001985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:44.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.988+0000 7faa04ac6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa00107190 0x7faa001985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59646/0 (socket says 172.21.15.71:59646) 2024-03-04T08:57:44.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.988+0000 7faa04ac6700 1 -- 172.21.15.71:0/2953145805 learned_addr learned my addr 172.21.15.71:0/2953145805 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:57:44.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.988+0000 7fa9ff7fe700 1 --2- 172.21.15.71:0/2953145805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa00101480 0x7faa0019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:44.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.988+0000 7fa9fffff700 1 --2- 172.21.15.71:0/2953145805 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faa00100ac0 0x7faa0019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:44.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.988+0000 7faa04ac6700 1 -- 172.21.15.71:0/2953145805 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faa00100ac0 msgr2=0x7faa0019e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:44.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.988+0000 7faa04ac6700 1 --2- 172.21.15.71:0/2953145805 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faa00100ac0 0x7faa0019e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.988+0000 7faa04ac6700 1 -- 172.21.15.71:0/2953145805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa00101480 msgr2=0x7faa0019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:44.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.988+0000 7faa04ac6700 1 --2- 172.21.15.71:0/2953145805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa00101480 0x7faa0019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:44.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.988+0000 7faa04ac6700 1 -- 172.21.15.71:0/2953145805 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa00198d20 con 0x7faa00107190 2024-03-04T08:57:44.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.988+0000 7fa9fffff700 1 --2- 172.21.15.71:0/2953145805 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faa00100ac0 0x7faa0019e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:57:44.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.988+0000 7fa9ff7fe700 1 --2- 172.21.15.71:0/2953145805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa00101480 0x7faa0019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:57:44.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.989+0000 7faa04ac6700 1 --2- 172.21.15.71:0/2953145805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa00107190 0x7faa001985c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800ef30 tx=0x7fa9f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:44.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.989+0000 7fa9fd7fa700 1 -- 172.21.15.71:0/2953145805 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80092e0 con 0x7faa00107190 2024-03-04T08:57:44.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.989+0000 7faa06529700 1 -- 172.21.15.71:0/2953145805 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa00198fb0 con 0x7faa00107190 2024-03-04T08:57:44.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.989+0000 7fa9fd7fa700 1 -- 172.21.15.71:0/2953145805 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa9f8004bb0 con 0x7faa00107190 2024-03-04T08:57:44.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.989+0000 7fa9fd7fa700 1 -- 172.21.15.71:0/2953145805 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8004280 con 0x7faa00107190 2024-03-04T08:57:44.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.989+0000 7faa06529700 1 -- 172.21.15.71:0/2953145805 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa001a52d0 con 0x7faa00107190 2024-03-04T08:57:44.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.990+0000 7faa06529700 1 -- 172.21.15.71:0/2953145805 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa00068d80 con 0x7faa00107190 2024-03-04T08:57:45.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.991+0000 7fa9fd7fa700 1 -- 172.21.15.71:0/2953145805 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fa9f801b030 con 0x7faa00107190 2024-03-04T08:57:45.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.992+0000 7fa9fd7fa700 1 --2- 172.21.15.71:0/2953145805 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa9e80a1ea0 0x7fa9e80a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:45.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.992+0000 7fa9fffff700 1 --2- 172.21.15.71:0/2953145805 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa9e80a1ea0 0x7fa9e80a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:45.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.992+0000 7fa9fd7fa700 1 -- 172.21.15.71:0/2953145805 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fa9f80ce770 con 0x7faa00107190 2024-03-04T08:57:45.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.993+0000 7fa9fffff700 1 --2- 172.21.15.71:0/2953145805 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa9e80a1ea0 0x7fa9e80a4360 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0006010 tx=0x7fa9f0005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:45.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:44.996+0000 7fa9fd7fa700 1 -- 172.21.15.71:0/2953145805 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa9f8022020 con 0x7faa00107190 2024-03-04T08:57:45.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:45.194+0000 7faa06529700 1 -- 172.21.15.71:0/2953145805 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7faa001a54a0 con 0x7faa00107190 2024-03-04T08:57:45.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:45.195+0000 7fa9fd7fa700 1 -- 172.21.15.71:0/2953145805 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa9f8010040 con 0x7faa00107190 2024-03-04T08:57:45.191 INFO:teuthology.orchestra.run.smithi071.stdout:111669149771 2024-03-04T08:57:45.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:45.197+0000 7faa06529700 1 -- 172.21.15.71:0/2953145805 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa9e80a1ea0 msgr2=0x7fa9e80a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:45.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:45.197+0000 7faa06529700 1 --2- 172.21.15.71:0/2953145805 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa9e80a1ea0 0x7fa9e80a4360 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0006010 tx=0x7fa9f0005ab0 comp rx=0 tx=0).stop 2024-03-04T08:57:45.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:45.197+0000 7faa06529700 1 -- 172.21.15.71:0/2953145805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa00107190 msgr2=0x7faa001985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:45.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:45.197+0000 7faa06529700 1 --2- 172.21.15.71:0/2953145805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa00107190 0x7faa001985c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800ef30 tx=0x7fa9f800c3a0 comp rx=0 tx=0).stop 2024-03-04T08:57:45.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:45.197+0000 7faa06529700 1 -- 172.21.15.71:0/2953145805 shutdown_connections 2024-03-04T08:57:45.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:45.197+0000 7faa06529700 1 --2- 172.21.15.71:0/2953145805 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa9e80a1ea0 0x7fa9e80a4360 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:45.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:45.197+0000 7faa06529700 1 --2- 172.21.15.71:0/2953145805 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7faa00100ac0 0x7faa0019e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:45.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:45.197+0000 7faa06529700 1 --2- 172.21.15.71:0/2953145805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa00107190 0x7faa001985c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:45.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:45.197+0000 7faa06529700 1 --2- 172.21.15.71:0/2953145805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa00101480 0x7faa0019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:45.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:45.197+0000 7faa06529700 1 -- 172.21.15.71:0/2953145805 >> 172.21.15.71:0/2953145805 conn(0x7faa00076310 msgr2=0x7faa000fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:45.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:45.197+0000 7faa06529700 1 -- 172.21.15.71:0/2953145805 shutdown_connections 2024-03-04T08:57:45.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:45.197+0000 7faa06529700 1 -- 172.21.15.71:0/2953145805 wait complete. 2024-03-04T08:57:46.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:45 smithi150 ceph-mon[36807]: pgmap v364: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:57:46.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:45 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/344340357' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-04T08:57:46.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:45 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3151209867' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-04T08:57:46.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:45 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2953145805' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-04T08:57:46.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:45 smithi158 ceph-mon[37241]: pgmap v364: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:57:46.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:45 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/344340357' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-04T08:57:46.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:45 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3151209867' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-04T08:57:46.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:45 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2953145805' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-04T08:57:46.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:45 smithi071 ceph-mon[32789]: pgmap v364: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:57:46.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:45 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/344340357' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-04T08:57:46.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:45 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3151209867' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-04T08:57:46.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:45 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2953145805' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-04T08:57:46.281 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724606 got 167503724609 for osd.5 2024-03-04T08:57:46.281 DEBUG:teuthology.parallel:result is None 2024-03-04T08:57:47.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.814+0000 7fe2a6105700 1 -- 172.21.15.71:0/1978713907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2a0106780 msgr2=0x7fe2a0106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:47.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.814+0000 7fe2a6105700 1 --2- 172.21.15.71:0/1978713907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2a0106780 0x7fe2a0106b60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe290009990 tx=0x7fe290009ca0 comp rx=0 tx=0).stop 2024-03-04T08:57:47.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.814+0000 7fe2a6105700 1 -- 172.21.15.71:0/1978713907 shutdown_connections 2024-03-04T08:57:47.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.814+0000 7fe2a6105700 1 --2- 172.21.15.71:0/1978713907 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2a0101140 0x7fe2a010d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:47.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.814+0000 7fe2a6105700 1 --2- 172.21.15.71:0/1978713907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2a0106780 0x7fe2a0106b60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:47.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.814+0000 7fe2a6105700 1 --2- 172.21.15.71:0/1978713907 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe2a0100780 0x7fe2a0100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:47.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.814+0000 7fe2a6105700 1 -- 172.21.15.71:0/1978713907 >> 172.21.15.71:0/1978713907 conn(0x7fe2a00775c0 msgr2=0x7fe2a00fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:47.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.814+0000 7fe2a6105700 1 -- 172.21.15.71:0/1978713907 shutdown_connections 2024-03-04T08:57:47.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.814+0000 7fe2a6105700 1 -- 172.21.15.71:0/1978713907 wait complete. 2024-03-04T08:57:47.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.815+0000 7fe2a6105700 1 Processor -- start 2024-03-04T08:57:47.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.815+0000 7fe2a6105700 1 -- start start 2024-03-04T08:57:47.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.815+0000 7fe2a6105700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2a0100780 0x7fe2a019e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:47.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.815+0000 7fe2a6105700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2a0101140 0x7fe2a019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:47.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.815+0000 7fe2a6105700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe2a0106780 0x7fe2a0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:47.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.816+0000 7fe2a6105700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a0110280 con 0x7fe2a0100780 2024-03-04T08:57:47.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.816+0000 7fe2a6105700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a0110100 con 0x7fe2a0101140 2024-03-04T08:57:47.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.816+0000 7fe2a6105700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a0110400 con 0x7fe2a0106780 2024-03-04T08:57:47.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.816+0000 7fe29f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2a0100780 0x7fe2a019e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:47.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.816+0000 7fe29f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2a0100780 0x7fe2a019e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59602/0 (socket says 172.21.15.71:59602) 2024-03-04T08:57:47.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.816+0000 7fe29f7fe700 1 -- 172.21.15.71:0/2021406515 learned_addr learned my addr 172.21.15.71:0/2021406515 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:57:47.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.816+0000 7fe29effd700 1 --2- 172.21.15.71:0/2021406515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2a0101140 0x7fe2a019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:47.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.816+0000 7fe29ffff700 1 --2- 172.21.15.71:0/2021406515 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe2a0106780 0x7fe2a0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:47.541 INFO:teuthology.orchestra.run.smithi071.stdout:34359738462 2024-03-04T08:57:47.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.816+0000 7fe29f7fe700 1 -- 172.21.15.71:0/2021406515 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe2a0106780 msgr2=0x7fe2a0198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:47.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.816+0000 7fe29f7fe700 1 --2- 172.21.15.71:0/2021406515 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe2a0106780 0x7fe2a0198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:47.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.816+0000 7fe29f7fe700 1 -- 172.21.15.71:0/2021406515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2a0101140 msgr2=0x7fe2a019eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:47.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.816+0000 7fe29f7fe700 1 --2- 172.21.15.71:0/2021406515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2a0101140 0x7fe2a019eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:47.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.816+0000 7fe29f7fe700 1 -- 172.21.15.71:0/2021406515 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a0199030 con 0x7fe2a0100780 2024-03-04T08:57:47.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.816+0000 7fe29f7fe700 1 --2- 172.21.15.71:0/2021406515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2a0100780 0x7fe2a019e630 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe290005910 tx=0x7fe29000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:47.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.816+0000 7fe29cff9700 1 -- 172.21.15.71:0/2021406515 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe290027070 con 0x7fe2a0100780 2024-03-04T08:57:47.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.816+0000 7fe2a6105700 1 -- 172.21.15.71:0/2021406515 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a01992c0 con 0x7fe2a0100780 2024-03-04T08:57:47.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.816+0000 7fe29cff9700 1 -- 172.21.15.71:0/2021406515 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe290003d70 con 0x7fe2a0100780 2024-03-04T08:57:47.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.816+0000 7fe29cff9700 1 -- 172.21.15.71:0/2021406515 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29000f460 con 0x7fe2a0100780 2024-03-04T08:57:47.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.816+0000 7fe2a6105700 1 -- 172.21.15.71:0/2021406515 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a0196640 con 0x7fe2a0100780 2024-03-04T08:57:47.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.817+0000 7fe2a6105700 1 -- 172.21.15.71:0/2021406515 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2a0199620 con 0x7fe2a0100780 2024-03-04T08:57:47.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.818+0000 7fe29cff9700 1 -- 172.21.15.71:0/2021406515 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fe29002a070 con 0x7fe2a0100780 2024-03-04T08:57:47.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.819+0000 7fe29cff9700 1 --2- 172.21.15.71:0/2021406515 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe2880a1e80 0x7fe2880a4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:47.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.819+0000 7fe29effd700 1 --2- 172.21.15.71:0/2021406515 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe2880a1e80 0x7fe2880a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:47.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.819+0000 7fe29cff9700 1 -- 172.21.15.71:0/2021406515 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fe2900cf100 con 0x7fe2a0100780 2024-03-04T08:57:47.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.819+0000 7fe29effd700 1 --2- 172.21.15.71:0/2021406515 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe2880a1e80 0x7fe2880a4340 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0199ec0 tx=0x7fe29400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:47.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:42.823+0000 7fe29cff9700 1 -- 172.21.15.71:0/2021406515 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe29005e280 con 0x7fe2a0100780 2024-03-04T08:57:47.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:43.024+0000 7fe2a6105700 1 -- 172.21.15.71:0/2021406515 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe2a0068d80 con 0x7fe2a0100780 2024-03-04T08:57:47.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:43.024+0000 7fe29cff9700 1 -- 172.21.15.71:0/2021406515 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe290013070 con 0x7fe2a0100780 2024-03-04T08:57:47.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:43.026+0000 7fe2a6105700 1 -- 172.21.15.71:0/2021406515 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe2880a1e80 msgr2=0x7fe2880a4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:47.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:43.026+0000 7fe2a6105700 1 --2- 172.21.15.71:0/2021406515 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe2880a1e80 0x7fe2880a4340 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0199ec0 tx=0x7fe29400b410 comp rx=0 tx=0).stop 2024-03-04T08:57:47.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:43.026+0000 7fe2a6105700 1 -- 172.21.15.71:0/2021406515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2a0100780 msgr2=0x7fe2a019e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:47.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:43.026+0000 7fe2a6105700 1 --2- 172.21.15.71:0/2021406515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2a0100780 0x7fe2a019e630 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe290005910 tx=0x7fe29000bab0 comp rx=0 tx=0).stop 2024-03-04T08:57:47.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:43.026+0000 7fe2a6105700 1 -- 172.21.15.71:0/2021406515 shutdown_connections 2024-03-04T08:57:47.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:43.026+0000 7fe2a6105700 1 --2- 172.21.15.71:0/2021406515 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fe2880a1e80 0x7fe2880a4340 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:47.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:43.026+0000 7fe2a6105700 1 --2- 172.21.15.71:0/2021406515 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fe2a0106780 0x7fe2a0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:47.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:43.026+0000 7fe2a6105700 1 --2- 172.21.15.71:0/2021406515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2a0100780 0x7fe2a019e630 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:47.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:43.026+0000 7fe2a6105700 1 --2- 172.21.15.71:0/2021406515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe2a0101140 0x7fe2a019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:47.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:43.026+0000 7fe2a6105700 1 -- 172.21.15.71:0/2021406515 >> 172.21.15.71:0/2021406515 conn(0x7fe2a00775c0 msgr2=0x7fe2a0103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:47.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:43.027+0000 7fe2a6105700 1 -- 172.21.15.71:0/2021406515 shutdown_connections 2024-03-04T08:57:47.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:43.027+0000 7fe2a6105700 1 -- 172.21.15.71:0/2021406515 wait complete. 2024-03-04T08:57:48.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:47 smithi150 ceph-mon[36807]: pgmap v365: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:57:48.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:47 smithi158 ceph-mon[37241]: pgmap v365: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:57:48.106 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:47 smithi071 ceph-mon[32789]: pgmap v365: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:57:48.232 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738459 got 34359738462 for osd.0 2024-03-04T08:57:48.232 DEBUG:teuthology.parallel:result is None 2024-03-04T08:57:48.983 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:48.983 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:49.018 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743461 got 296352743464 for osd.9 2024-03-04T08:57:49.019 DEBUG:teuthology.parallel:result is None 2024-03-04T08:57:50.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:49 smithi150 ceph-mon[36807]: pgmap v366: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:57:50.045 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:49 smithi071 ceph-mon[32789]: pgmap v366: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:57:50.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:49 smithi158 ceph-mon[37241]: pgmap v366: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:57:50.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.060+0000 7f941d6e9700 1 -- 172.21.15.71:0/2013102124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9418101c00 msgr2=0x7f9418105bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:50.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.060+0000 7f941d6e9700 1 --2- 172.21.15.71:0/2013102124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9418101c00 0x7f9418105bc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f940c0099c0 tx=0x7f940c009cd0 comp rx=0 tx=0).stop 2024-03-04T08:57:50.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.060+0000 7f941d6e9700 1 -- 172.21.15.71:0/2013102124 shutdown_connections 2024-03-04T08:57:50.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.060+0000 7f941d6e9700 1 --2- 172.21.15.71:0/2013102124 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f94181062a0 0x7f941810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:50.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.060+0000 7f941d6e9700 1 --2- 172.21.15.71:0/2013102124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9418101250 0x7f9418101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:50.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.060+0000 7f941d6e9700 1 --2- 172.21.15.71:0/2013102124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9418101c00 0x7f9418105bc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:50.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.060+0000 7f941d6e9700 1 -- 172.21.15.71:0/2013102124 >> 172.21.15.71:0/2013102124 conn(0x7f9418077010 msgr2=0x7f9418077420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:50.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.060+0000 7f941d6e9700 1 -- 172.21.15.71:0/2013102124 shutdown_connections 2024-03-04T08:57:50.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.060+0000 7f941d6e9700 1 -- 172.21.15.71:0/2013102124 wait complete. 2024-03-04T08:57:50.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.060+0000 7f941d6e9700 1 Processor -- start 2024-03-04T08:57:50.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.061+0000 7f941d6e9700 1 -- start start 2024-03-04T08:57:50.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.061+0000 7f941d6e9700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9418101250 0x7f941819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:50.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.061+0000 7f941d6e9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9418101c00 0x7f941819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:50.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.061+0000 7f941d6e9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94181062a0 0x7f9418198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:50.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.061+0000 7f941d6e9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9418111280 con 0x7f94181062a0 2024-03-04T08:57:50.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.061+0000 7f941d6e9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9418111100 con 0x7f9418101c00 2024-03-04T08:57:50.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.061+0000 7f941d6e9700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9418111400 con 0x7f9418101250 2024-03-04T08:57:50.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.061+0000 7f94177fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94181062a0 0x7f9418198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:50.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.061+0000 7f94177fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94181062a0 0x7f9418198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41940/0 (socket says 172.21.15.71:41940) 2024-03-04T08:57:50.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.061+0000 7f94177fe700 1 -- 172.21.15.71:0/3757473114 learned_addr learned my addr 172.21.15.71:0/3757473114 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:57:50.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.061+0000 7f94167fc700 1 --2- 172.21.15.71:0/3757473114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9418101c00 0x7f941819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:50.059 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.061+0000 7f9416ffd700 1 --2- 172.21.15.71:0/3757473114 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9418101250 0x7f941819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:50.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.061+0000 7f94177fe700 1 -- 172.21.15.71:0/3757473114 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9418101250 msgr2=0x7f941819e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:50.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.061+0000 7f94177fe700 1 --2- 172.21.15.71:0/3757473114 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9418101250 0x7f941819e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:50.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.061+0000 7f94177fe700 1 -- 172.21.15.71:0/3757473114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9418101c00 msgr2=0x7f941819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:50.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.061+0000 7f94177fe700 1 --2- 172.21.15.71:0/3757473114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9418101c00 0x7f941819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:50.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.061+0000 7f94177fe700 1 -- 172.21.15.71:0/3757473114 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9418198f70 con 0x7f94181062a0 2024-03-04T08:57:50.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.061+0000 7f9416ffd700 1 --2- 172.21.15.71:0/3757473114 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9418101250 0x7f941819e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:57:50.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.061+0000 7f94177fe700 1 --2- 172.21.15.71:0/3757473114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94181062a0 0x7f9418198690 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f941000ef30 tx=0x7f941000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:50.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.062+0000 7f93fffff700 1 -- 172.21.15.71:0/3757473114 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94100092e0 con 0x7f94181062a0 2024-03-04T08:57:50.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.062+0000 7f93fffff700 1 -- 172.21.15.71:0/3757473114 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9410004bb0 con 0x7f94181062a0 2024-03-04T08:57:50.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.062+0000 7f93fffff700 1 -- 172.21.15.71:0/3757473114 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9410004280 con 0x7f94181062a0 2024-03-04T08:57:50.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.062+0000 7f941d6e9700 1 -- 172.21.15.71:0/3757473114 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9418199200 con 0x7f94181062a0 2024-03-04T08:57:50.060 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.062+0000 7f941d6e9700 1 -- 172.21.15.71:0/3757473114 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94181a5540 con 0x7f94181062a0 2024-03-04T08:57:50.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.063+0000 7f941d6e9700 1 -- 172.21.15.71:0/3757473114 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94181026a0 con 0x7f94181062a0 2024-03-04T08:57:50.061 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.065+0000 7f93fffff700 1 -- 172.21.15.71:0/3757473114 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f941001b030 con 0x7f94181062a0 2024-03-04T08:57:50.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.065+0000 7f93fffff700 1 --2- 172.21.15.71:0/3757473114 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f94000a1f20 0x7f94000a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:50.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.065+0000 7f9416ffd700 1 --2- 172.21.15.71:0/3757473114 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f94000a1f20 0x7f94000a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:50.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.065+0000 7f93fffff700 1 -- 172.21.15.71:0/3757473114 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f94100ce6c0 con 0x7f94181062a0 2024-03-04T08:57:50.062 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.066+0000 7f9416ffd700 1 --2- 172.21.15.71:0/3757473114 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f94000a1f20 0x7f94000a43e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9408009cb0 tx=0x7f9408009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:50.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.070+0000 7f93fffff700 1 -- 172.21.15.71:0/3757473114 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9410022020 con 0x7f94181062a0 2024-03-04T08:57:50.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.264+0000 7f941d6e9700 1 -- 172.21.15.71:0/3757473114 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f941802d9b0 con 0x7f94181062a0 2024-03-04T08:57:50.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.265+0000 7f93fffff700 1 -- 172.21.15.71:0/3757473114 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9410010040 con 0x7f94181062a0 2024-03-04T08:57:50.261 INFO:teuthology.orchestra.run.smithi071.stdout:77309411411 2024-03-04T08:57:50.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.267+0000 7f941d6e9700 1 -- 172.21.15.71:0/3757473114 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f94000a1f20 msgr2=0x7f94000a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:50.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.267+0000 7f941d6e9700 1 --2- 172.21.15.71:0/3757473114 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f94000a1f20 0x7f94000a43e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9408009cb0 tx=0x7f9408009260 comp rx=0 tx=0).stop 2024-03-04T08:57:50.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.267+0000 7f941d6e9700 1 -- 172.21.15.71:0/3757473114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94181062a0 msgr2=0x7f9418198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:50.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.267+0000 7f941d6e9700 1 --2- 172.21.15.71:0/3757473114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94181062a0 0x7f9418198690 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f941000ef30 tx=0x7f941000c3a0 comp rx=0 tx=0).stop 2024-03-04T08:57:50.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.267+0000 7f941d6e9700 1 -- 172.21.15.71:0/3757473114 shutdown_connections 2024-03-04T08:57:50.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.267+0000 7f941d6e9700 1 --2- 172.21.15.71:0/3757473114 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f94000a1f20 0x7f94000a43e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:50.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.267+0000 7f941d6e9700 1 --2- 172.21.15.71:0/3757473114 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9418101250 0x7f941819e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:50.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.267+0000 7f941d6e9700 1 --2- 172.21.15.71:0/3757473114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94181062a0 0x7f9418198690 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:50.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.267+0000 7f941d6e9700 1 --2- 172.21.15.71:0/3757473114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9418101c00 0x7f941819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:50.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.267+0000 7f941d6e9700 1 -- 172.21.15.71:0/3757473114 >> 172.21.15.71:0/3757473114 conn(0x7f9418077010 msgr2=0x7f941806a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:50.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.267+0000 7f941d6e9700 1 -- 172.21.15.71:0/3757473114 shutdown_connections 2024-03-04T08:57:50.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.267+0000 7f941d6e9700 1 -- 172.21.15.71:0/3757473114 wait complete. 2024-03-04T08:57:50.551 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149768 got 111669149771 for osd.3 2024-03-04T08:57:50.552 DEBUG:teuthology.parallel:result is None 2024-03-04T08:57:50.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.865+0000 7fa78a622700 1 -- 172.21.15.71:0/2606582841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa784102ad0 msgr2=0x7fa78410ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:50.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.865+0000 7fa78a622700 1 --2- 172.21.15.71:0/2606582841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa784102ad0 0x7fa78410ef10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00b0d0 tx=0x7fa77c00b3e0 comp rx=0 tx=0).stop 2024-03-04T08:57:50.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.865+0000 7fa78a622700 1 -- 172.21.15.71:0/2606582841 shutdown_connections 2024-03-04T08:57:50.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.865+0000 7fa78a622700 1 --2- 172.21.15.71:0/2606582841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa784102ad0 0x7fa78410ef10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:50.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.865+0000 7fa78a622700 1 --2- 172.21.15.71:0/2606582841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7841080f0 0x7fa7841084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:50.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.865+0000 7fa78a622700 1 --2- 172.21.15.71:0/2606582841 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa784102110 0x7fa784102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:50.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.865+0000 7fa78a622700 1 -- 172.21.15.71:0/2606582841 >> 172.21.15.71:0/2606582841 conn(0x7fa7840791f0 msgr2=0x7fa784079600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:50.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.866+0000 7fa78a622700 1 -- 172.21.15.71:0/2606582841 shutdown_connections 2024-03-04T08:57:50.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.866+0000 7fa78a622700 1 -- 172.21.15.71:0/2606582841 wait complete. 2024-03-04T08:57:50.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.866+0000 7fa78a622700 1 Processor -- start 2024-03-04T08:57:50.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.866+0000 7fa78a622700 1 -- start start 2024-03-04T08:57:50.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.866+0000 7fa78a622700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa784102110 0x7fa7841986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:50.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.866+0000 7fa78a622700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa784102ad0 0x7fa784198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:50.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.866+0000 7fa78a622700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7841080f0 0x7fa78419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:50.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.866+0000 7fa78a622700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa784112370 con 0x7fa7841080f0 2024-03-04T08:57:50.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.866+0000 7fa78a622700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7841121f0 con 0x7fa784102ad0 2024-03-04T08:57:50.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.867+0000 7fa78a622700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7841124f0 con 0x7fa784102110 2024-03-04T08:57:50.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.867+0000 7fa788bbf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7841080f0 0x7fa78419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:50.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.867+0000 7fa788bbf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7841080f0 0x7fa78419cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41952/0 (socket says 172.21.15.71:41952) 2024-03-04T08:57:50.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.867+0000 7fa788bbf700 1 -- 172.21.15.71:0/222999462 learned_addr learned my addr 172.21.15.71:0/222999462 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:57:50.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.867+0000 7fa7837fe700 1 --2- 172.21.15.71:0/222999462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa784102ad0 0x7fa784198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:50.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.867+0000 7fa783fff700 1 --2- 172.21.15.71:0/222999462 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa784102110 0x7fa7841986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:50.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.867+0000 7fa788bbf700 1 -- 172.21.15.71:0/222999462 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa784102110 msgr2=0x7fa7841986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:50.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.867+0000 7fa788bbf700 1 --2- 172.21.15.71:0/222999462 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa784102110 0x7fa7841986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:50.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.867+0000 7fa788bbf700 1 -- 172.21.15.71:0/222999462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa784102ad0 msgr2=0x7fa784198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:50.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.867+0000 7fa788bbf700 1 --2- 172.21.15.71:0/222999462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa784102ad0 0x7fa784198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:50.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.867+0000 7fa788bbf700 1 -- 172.21.15.71:0/222999462 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78419d6b0 con 0x7fa7841080f0 2024-03-04T08:57:50.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.867+0000 7fa7837fe700 1 --2- 172.21.15.71:0/222999462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa784102ad0 0x7fa784198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:57:50.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.867+0000 7fa788bbf700 1 --2- 172.21.15.71:0/222999462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7841080f0 0x7fa78419cf70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa77c006070 tx=0x7fa77c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:50.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.867+0000 7fa7817fa700 1 -- 172.21.15.71:0/222999462 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c003c00 con 0x7fa7841080f0 2024-03-04T08:57:50.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.867+0000 7fa7817fa700 1 -- 172.21.15.71:0/222999462 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa77c003da0 con 0x7fa7841080f0 2024-03-04T08:57:50.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.867+0000 7fa78a622700 1 -- 172.21.15.71:0/222999462 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa78419d940 con 0x7fa7841080f0 2024-03-04T08:57:50.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.867+0000 7fa7817fa700 1 -- 172.21.15.71:0/222999462 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c01b880 con 0x7fa7841080f0 2024-03-04T08:57:50.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.867+0000 7fa78a622700 1 -- 172.21.15.71:0/222999462 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7840742f0 con 0x7fa7841080f0 2024-03-04T08:57:50.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.868+0000 7fa78a622700 1 -- 172.21.15.71:0/222999462 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7841037a0 con 0x7fa7841080f0 2024-03-04T08:57:50.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.869+0000 7fa7817fa700 1 -- 172.21.15.71:0/222999462 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fa77c01ba20 con 0x7fa7841080f0 2024-03-04T08:57:50.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.870+0000 7fa7817fa700 1 --2- 172.21.15.71:0/222999462 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa76c0a1ea0 0x7fa76c0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:50.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.870+0000 7fa783fff700 1 --2- 172.21.15.71:0/222999462 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa76c0a1ea0 0x7fa76c0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:50.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.870+0000 7fa7817fa700 1 -- 172.21.15.71:0/222999462 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fa77c00f020 con 0x7fa7841080f0 2024-03-04T08:57:50.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.871+0000 7fa783fff700 1 --2- 172.21.15.71:0/222999462 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa76c0a1ea0 0x7fa76c0a4360 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa774010d80 tx=0x7fa774012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:50.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:50.874+0000 7fa7817fa700 1 -- 172.21.15.71:0/222999462 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa77c09c3e0 con 0x7fa7841080f0 2024-03-04T08:57:51.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:50 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3757473114' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-04T08:57:51.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:50 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3757473114' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-04T08:57:51.069 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:50 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3757473114' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-04T08:57:51.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:51.073+0000 7fa78a622700 1 -- 172.21.15.71:0/222999462 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fa784199a00 con 0x7fa7841080f0 2024-03-04T08:57:51.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:51.073+0000 7fa7817fa700 1 -- 172.21.15.71:0/222999462 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fa77c09bce0 con 0x7fa7841080f0 2024-03-04T08:57:51.070 INFO:teuthology.orchestra.run.smithi071.stdout:317827579939 2024-03-04T08:57:51.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:51.075+0000 7fa78a622700 1 -- 172.21.15.71:0/222999462 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa76c0a1ea0 msgr2=0x7fa76c0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:51.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:51.075+0000 7fa78a622700 1 --2- 172.21.15.71:0/222999462 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa76c0a1ea0 0x7fa76c0a4360 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa774010d80 tx=0x7fa774012040 comp rx=0 tx=0).stop 2024-03-04T08:57:51.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:51.075+0000 7fa78a622700 1 -- 172.21.15.71:0/222999462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7841080f0 msgr2=0x7fa78419cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:51.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:51.075+0000 7fa78a622700 1 --2- 172.21.15.71:0/222999462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7841080f0 0x7fa78419cf70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa77c006070 tx=0x7fa77c007a20 comp rx=0 tx=0).stop 2024-03-04T08:57:51.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:51.075+0000 7fa78a622700 1 -- 172.21.15.71:0/222999462 shutdown_connections 2024-03-04T08:57:51.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:51.075+0000 7fa78a622700 1 --2- 172.21.15.71:0/222999462 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa76c0a1ea0 0x7fa76c0a4360 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:51.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:51.075+0000 7fa78a622700 1 --2- 172.21.15.71:0/222999462 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa784102110 0x7fa7841986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:51.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:51.075+0000 7fa78a622700 1 --2- 172.21.15.71:0/222999462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7841080f0 0x7fa78419cf70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:51.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:51.075+0000 7fa78a622700 1 --2- 172.21.15.71:0/222999462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa784102ad0 0x7fa784198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:51.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:51.075+0000 7fa78a622700 1 -- 172.21.15.71:0/222999462 >> 172.21.15.71:0/222999462 conn(0x7fa7840791f0 msgr2=0x7fa784100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:51.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:51.075+0000 7fa78a622700 1 -- 172.21.15.71:0/222999462 shutdown_connections 2024-03-04T08:57:51.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:51.075+0000 7fa78a622700 1 -- 172.21.15.71:0/222999462 wait complete. 2024-03-04T08:57:52.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:51 smithi150 ceph-mon[36807]: pgmap v367: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:57:52.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:51 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/222999462' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-04T08:57:52.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:51 smithi158 ceph-mon[37241]: pgmap v367: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:57:52.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:51 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/222999462' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-04T08:57:52.121 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:51 smithi071 ceph-mon[32789]: pgmap v367: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:57:52.121 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:51 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/222999462' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-04T08:57:52.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.891+0000 7fafb2e82700 1 -- 172.21.15.71:0/516071932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fafac102b00 msgr2=0x7fafac10ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:52.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.891+0000 7fafb2e82700 1 --2- 172.21.15.71:0/516071932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fafac102b00 0x7fafac10ef60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faf94009990 tx=0x7faf94009ca0 comp rx=0 tx=0).stop 2024-03-04T08:57:52.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.891+0000 7fafb2e82700 1 -- 172.21.15.71:0/516071932 shutdown_connections 2024-03-04T08:57:52.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.891+0000 7fafb2e82700 1 --2- 172.21.15.71:0/516071932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fafac102b00 0x7fafac10ef60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:52.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.891+0000 7fafb2e82700 1 --2- 172.21.15.71:0/516071932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafac108220 0x7fafac108600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:52.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.891+0000 7fafb2e82700 1 --2- 172.21.15.71:0/516071932 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafac102140 0x7fafac1025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:52.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.891+0000 7fafb2e82700 1 -- 172.21.15.71:0/516071932 >> 172.21.15.71:0/516071932 conn(0x7fafac079180 msgr2=0x7fafac079590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:52.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.891+0000 7fafb2e82700 1 -- 172.21.15.71:0/516071932 shutdown_connections 2024-03-04T08:57:52.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.891+0000 7fafb2e82700 1 -- 172.21.15.71:0/516071932 wait complete. 2024-03-04T08:57:52.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.891+0000 7fafb2e82700 1 Processor -- start 2024-03-04T08:57:52.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.892+0000 7fafb2e82700 1 -- start start 2024-03-04T08:57:52.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.892+0000 7fafb2e82700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fafac102140 0x7fafac1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:52.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.892+0000 7fafb2e82700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafac102b00 0x7fafac198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:52.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.892+0000 7fafb2e82700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafac108220 0x7fafac19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:52.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.892+0000 7fafb2e82700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac112380 con 0x7fafac102140 2024-03-04T08:57:52.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.892+0000 7fafb2e82700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac112200 con 0x7fafac108220 2024-03-04T08:57:52.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.892+0000 7fafb2e82700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac112500 con 0x7fafac102b00 2024-03-04T08:57:52.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.892+0000 7fafb0c1e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fafac102140 0x7fafac1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:52.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.892+0000 7fafb0c1e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fafac102140 0x7fafac1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41960/0 (socket says 172.21.15.71:41960) 2024-03-04T08:57:52.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.892+0000 7fafb0c1e700 1 -- 172.21.15.71:0/1881622540 learned_addr learned my addr 172.21.15.71:0/1881622540 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:57:52.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.892+0000 7fafb141f700 1 --2- 172.21.15.71:0/1881622540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafac108220 0x7fafac19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:52.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.892+0000 7fafabfff700 1 --2- 172.21.15.71:0/1881622540 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafac102b00 0x7fafac198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:52.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.892+0000 7fafb0c1e700 1 -- 172.21.15.71:0/1881622540 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafac102b00 msgr2=0x7fafac198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:52.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.892+0000 7fafb0c1e700 1 --2- 172.21.15.71:0/1881622540 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafac102b00 0x7fafac198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:52.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.892+0000 7fafb0c1e700 1 -- 172.21.15.71:0/1881622540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafac108220 msgr2=0x7fafac19cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:52.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.892+0000 7fafb0c1e700 1 --2- 172.21.15.71:0/1881622540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafac108220 0x7fafac19cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:52.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.892+0000 7fafb0c1e700 1 -- 172.21.15.71:0/1881622540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafac19d700 con 0x7fafac102140 2024-03-04T08:57:52.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.892+0000 7fafabfff700 1 --2- 172.21.15.71:0/1881622540 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafac102b00 0x7fafac198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:57:52.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.892+0000 7fafb0c1e700 1 --2- 172.21.15.71:0/1881622540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fafac102140 0x7fafac1986f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fafa400ef10 tx=0x7fafa400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:52.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.893+0000 7fafa9ffb700 1 -- 172.21.15.71:0/1881622540 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa400ca70 con 0x7fafac102140 2024-03-04T08:57:52.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.893+0000 7fafa9ffb700 1 -- 172.21.15.71:0/1881622540 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fafa4019070 con 0x7fafac102140 2024-03-04T08:57:52.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.893+0000 7fafb2e82700 1 -- 172.21.15.71:0/1881622540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafac19d990 con 0x7fafac102140 2024-03-04T08:57:52.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.893+0000 7fafa9ffb700 1 -- 172.21.15.71:0/1881622540 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa40146e0 con 0x7fafac102140 2024-03-04T08:57:52.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.893+0000 7fafb2e82700 1 -- 172.21.15.71:0/1881622540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafac1a5270 con 0x7fafac102140 2024-03-04T08:57:52.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.894+0000 7fafb2e82700 1 -- 172.21.15.71:0/1881622540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafac068d80 con 0x7fafac102140 2024-03-04T08:57:52.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.895+0000 7fafa9ffb700 1 -- 172.21.15.71:0/1881622540 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fafa4012050 con 0x7fafac102140 2024-03-04T08:57:52.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.895+0000 7fafa9ffb700 1 --2- 172.21.15.71:0/1881622540 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7faf980a1f20 0x7faf980a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:52.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.895+0000 7fafa9ffb700 1 -- 172.21.15.71:0/1881622540 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fafa40ce240 con 0x7fafac102140 2024-03-04T08:57:52.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.895+0000 7fafabfff700 1 --2- 172.21.15.71:0/1881622540 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7faf980a1f20 0x7faf980a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:52.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.896+0000 7fafabfff700 1 --2- 172.21.15.71:0/1881622540 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7faf980a1f20 0x7faf980a43e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faf9c005fd0 tx=0x7faf9c005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:52.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:52.900+0000 7fafa9ffb700 1 -- 172.21.15.71:0/1881622540 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fafa4010040 con 0x7fafac102140 2024-03-04T08:57:53.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:53.097+0000 7fafb2e82700 1 -- 172.21.15.71:0/1881622540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fafac1a5440 con 0x7fafac102140 2024-03-04T08:57:53.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:53.097+0000 7fafa9ffb700 1 -- 172.21.15.71:0/1881622540 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fafa409b1b0 con 0x7fafac102140 2024-03-04T08:57:53.094 INFO:teuthology.orchestra.run.smithi071.stdout:223338299445 2024-03-04T08:57:53.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:53.100+0000 7fafb2e82700 1 -- 172.21.15.71:0/1881622540 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7faf980a1f20 msgr2=0x7faf980a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:53.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:53.100+0000 7fafb2e82700 1 --2- 172.21.15.71:0/1881622540 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7faf980a1f20 0x7faf980a43e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faf9c005fd0 tx=0x7faf9c005e20 comp rx=0 tx=0).stop 2024-03-04T08:57:53.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:53.100+0000 7fafb2e82700 1 -- 172.21.15.71:0/1881622540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fafac102140 msgr2=0x7fafac1986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:53.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:53.100+0000 7fafb2e82700 1 --2- 172.21.15.71:0/1881622540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fafac102140 0x7fafac1986f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fafa400ef10 tx=0x7fafa400c3a0 comp rx=0 tx=0).stop 2024-03-04T08:57:53.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:53.100+0000 7fafb2e82700 1 -- 172.21.15.71:0/1881622540 shutdown_connections 2024-03-04T08:57:53.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:53.100+0000 7fafb2e82700 1 --2- 172.21.15.71:0/1881622540 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7faf980a1f20 0x7faf980a43e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:53.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:53.100+0000 7fafb2e82700 1 --2- 172.21.15.71:0/1881622540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafac108220 0x7fafac19cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:53.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:53.100+0000 7fafb2e82700 1 --2- 172.21.15.71:0/1881622540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fafac102140 0x7fafac1986f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:53.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:53.100+0000 7fafb2e82700 1 --2- 172.21.15.71:0/1881622540 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fafac102b00 0x7fafac198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:53.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:53.100+0000 7fafb2e82700 1 -- 172.21.15.71:0/1881622540 >> 172.21.15.71:0/1881622540 conn(0x7fafac079180 msgr2=0x7fafac100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:53.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:53.100+0000 7fafb2e82700 1 -- 172.21.15.71:0/1881622540 shutdown_connections 2024-03-04T08:57:53.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:53.100+0000 7fafb2e82700 1 -- 172.21.15.71:0/1881622540 wait complete. 2024-03-04T08:57:53.510 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:53.512 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579936 got 317827579939 for osd.10 2024-03-04T08:57:53.512 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:53.512 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:57:53.512 DEBUG:teuthology.parallel:result is None 2024-03-04T08:57:53.945 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411406 got 77309411411 for osd.2 2024-03-04T08:57:53.945 DEBUG:teuthology.parallel:result is None 2024-03-04T08:57:54.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:53 smithi158 ceph-mon[37241]: pgmap v368: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:57:54.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:53 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1881622540' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-04T08:57:54.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:53 smithi071 ceph-mon[32789]: pgmap v368: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:57:54.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:53 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1881622540' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-04T08:57:54.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:53 smithi150 ceph-mon[36807]: pgmap v368: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:57:54.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:53 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1881622540' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-04T08:57:55.001 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299440 got 223338299445 for osd.7 2024-03-04T08:57:55.001 DEBUG:teuthology.parallel:result is None 2024-03-04T08:57:56.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:55 smithi158 ceph-mon[37241]: pgmap v369: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:57:56.068 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:55 smithi071 ceph-mon[32789]: pgmap v369: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:57:56.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:55 smithi150 ceph-mon[36807]: pgmap v369: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:57:56.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.603+0000 7f0897baf700 1 -- 172.21.15.71:0/2256882300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0890100c50 msgr2=0x7f0890101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:56.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.603+0000 7f0897baf700 1 --2- 172.21.15.71:0/2256882300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0890100c50 0x7f0890101030 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f088c0099c0 tx=0x7f088c009cd0 comp rx=0 tx=0).stop 2024-03-04T08:57:56.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.603+0000 7f0897baf700 1 -- 172.21.15.71:0/2256882300 shutdown_connections 2024-03-04T08:57:56.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.603+0000 7f0897baf700 1 --2- 172.21.15.71:0/2256882300 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0890107470 0x7f089010eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:56.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.603+0000 7f0897baf700 1 --2- 172.21.15.71:0/2256882300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0890100c50 0x7f0890101030 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:56.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.603+0000 7f0897baf700 1 --2- 172.21.15.71:0/2256882300 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0890101600 0x7f0890106d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:56.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.603+0000 7f0897baf700 1 -- 172.21.15.71:0/2256882300 >> 172.21.15.71:0/2256882300 conn(0x7f08900762f0 msgr2=0x7f0890076700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:56.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.604+0000 7f0897baf700 1 -- 172.21.15.71:0/2256882300 shutdown_connections 2024-03-04T08:57:56.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.604+0000 7f0897baf700 1 -- 172.21.15.71:0/2256882300 wait complete. 2024-03-04T08:57:56.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.604+0000 7f0897baf700 1 Processor -- start 2024-03-04T08:57:56.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.604+0000 7f0897baf700 1 -- start start 2024-03-04T08:57:56.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.604+0000 7f0897baf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0890100c50 0x7f0890101bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:56.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.604+0000 7f0897baf700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0890101600 0x7f0890102110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:56.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.604+0000 7f0897baf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0890107470 0x7f0890102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:56.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.604+0000 7f0897baf700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0890112480 con 0x7f0890100c50 2024-03-04T08:57:56.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.604+0000 7f0897baf700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0890112300 con 0x7f0890107470 2024-03-04T08:57:56.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.604+0000 7f0897baf700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0890112600 con 0x7f0890101600 2024-03-04T08:57:56.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.604+0000 7f089594b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0890100c50 0x7f0890101bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:56.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.604+0000 7f089594b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0890100c50 0x7f0890101bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37296/0 (socket says 172.21.15.71:37296) 2024-03-04T08:57:56.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.604+0000 7f089594b700 1 -- 172.21.15.71:0/677433227 learned_addr learned my addr 172.21.15.71:0/677433227 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:57:56.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.604+0000 7f089614c700 1 --2- 172.21.15.71:0/677433227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0890107470 0x7f0890102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:56.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.605+0000 7f089514a700 1 --2- 172.21.15.71:0/677433227 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0890101600 0x7f0890102110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:56.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.605+0000 7f089594b700 1 -- 172.21.15.71:0/677433227 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0890101600 msgr2=0x7f0890102110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:56.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.605+0000 7f089594b700 1 --2- 172.21.15.71:0/677433227 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0890101600 0x7f0890102110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:56.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.605+0000 7f089594b700 1 -- 172.21.15.71:0/677433227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0890107470 msgr2=0x7f0890102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:56.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.605+0000 7f089594b700 1 --2- 172.21.15.71:0/677433227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0890107470 0x7f0890102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:56.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.605+0000 7f089594b700 1 -- 172.21.15.71:0/677433227 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08901a6e10 con 0x7f0890100c50 2024-03-04T08:57:56.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.605+0000 7f089594b700 1 --2- 172.21.15.71:0/677433227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0890100c50 0x7f0890101bd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f088c006010 tx=0x7f088c003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:56.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.605+0000 7f0886ffd700 1 -- 172.21.15.71:0/677433227 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088c009e60 con 0x7f0890100c50 2024-03-04T08:57:56.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.605+0000 7f0886ffd700 1 -- 172.21.15.71:0/677433227 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f088c003680 con 0x7f0890100c50 2024-03-04T08:57:56.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.605+0000 7f0886ffd700 1 -- 172.21.15.71:0/677433227 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088c0173f0 con 0x7f0890100c50 2024-03-04T08:57:56.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.605+0000 7f0897baf700 1 -- 172.21.15.71:0/677433227 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08901a70a0 con 0x7f0890100c50 2024-03-04T08:57:56.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.605+0000 7f0897baf700 1 -- 172.21.15.71:0/677433227 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08901a74f0 con 0x7f0890100c50 2024-03-04T08:57:56.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.606+0000 7f0897baf700 1 -- 172.21.15.71:0/677433227 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f089010a6b0 con 0x7f0890100c50 2024-03-04T08:57:56.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.607+0000 7f0886ffd700 1 -- 172.21.15.71:0/677433227 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f088c00f5d0 con 0x7f0890100c50 2024-03-04T08:57:56.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.608+0000 7f0886ffd700 1 --2- 172.21.15.71:0/677433227 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f087c0a1ea0 0x7f087c0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:57:56.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.608+0000 7f089514a700 1 --2- 172.21.15.71:0/677433227 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f087c0a1ea0 0x7f087c0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:57:56.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.608+0000 7f0886ffd700 1 -- 172.21.15.71:0/677433227 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f088c0cf0c0 con 0x7f0890100c50 2024-03-04T08:57:56.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.608+0000 7f089514a700 1 --2- 172.21.15.71:0/677433227 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f087c0a1ea0 0x7f087c0a4360 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0890076cf0 tx=0x7f0880009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:57:56.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.611+0000 7f0886ffd700 1 -- 172.21.15.71:0/677433227 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f088c013070 con 0x7f0890100c50 2024-03-04T08:57:56.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.809+0000 7f0897baf700 1 -- 172.21.15.71:0/677433227 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f089010a090 con 0x7f0890100c50 2024-03-04T08:57:56.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.810+0000 7f0886ffd700 1 -- 172.21.15.71:0/677433227 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f088c09c0c0 con 0x7f0890100c50 2024-03-04T08:57:56.806 INFO:teuthology.orchestra.run.smithi071.stdout:261993005108 2024-03-04T08:57:56.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.812+0000 7f0897baf700 1 -- 172.21.15.71:0/677433227 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f087c0a1ea0 msgr2=0x7f087c0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:56.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.812+0000 7f0897baf700 1 --2- 172.21.15.71:0/677433227 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f087c0a1ea0 0x7f087c0a4360 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0890076cf0 tx=0x7f0880009260 comp rx=0 tx=0).stop 2024-03-04T08:57:56.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.812+0000 7f0897baf700 1 -- 172.21.15.71:0/677433227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0890100c50 msgr2=0x7f0890101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:57:56.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.812+0000 7f0897baf700 1 --2- 172.21.15.71:0/677433227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0890100c50 0x7f0890101bd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f088c006010 tx=0x7f088c003e00 comp rx=0 tx=0).stop 2024-03-04T08:57:56.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.812+0000 7f0897baf700 1 -- 172.21.15.71:0/677433227 shutdown_connections 2024-03-04T08:57:56.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.812+0000 7f0897baf700 1 --2- 172.21.15.71:0/677433227 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f087c0a1ea0 0x7f087c0a4360 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:56.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.812+0000 7f0897baf700 1 --2- 172.21.15.71:0/677433227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0890107470 0x7f0890102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:56.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.812+0000 7f0897baf700 1 --2- 172.21.15.71:0/677433227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0890100c50 0x7f0890101bd0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:56.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.812+0000 7f0897baf700 1 --2- 172.21.15.71:0/677433227 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0890101600 0x7f0890102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:57:56.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.812+0000 7f0897baf700 1 -- 172.21.15.71:0/677433227 >> 172.21.15.71:0/677433227 conn(0x7f08900762f0 msgr2=0x7f08900fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:57:56.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.812+0000 7f0897baf700 1 -- 172.21.15.71:0/677433227 shutdown_connections 2024-03-04T08:57:56.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:57:56.812+0000 7f0897baf700 1 -- 172.21.15.71:0/677433227 wait complete. 2024-03-04T08:57:57.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:57 smithi071 ceph-mon[32789]: pgmap v370: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:57:57.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:57 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/677433227' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-04T08:57:58.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:57 smithi158 ceph-mon[37241]: pgmap v370: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:57:58.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:57 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/677433227' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-04T08:57:58.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:57 smithi150 ceph-mon[36807]: pgmap v370: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:57:58.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:57 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/677433227' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-04T08:57:58.786 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005102 got 261993005108 for osd.8 2024-03-04T08:57:58.786 DEBUG:teuthology.parallel:result is None 2024-03-04T08:58:00.049 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:57:59 smithi071 ceph-mon[32789]: pgmap v371: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:00.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:57:59 smithi158 ceph-mon[37241]: pgmap v371: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:00.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:57:59 smithi150 ceph-mon[36807]: pgmap v371: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:00.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.588+0000 7f1440529700 1 -- 172.21.15.71:0/4251350069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1438107450 msgr2=0x7f143810efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:00.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.588+0000 7f1440529700 1 --2- 172.21.15.71:0/4251350069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1438107450 0x7f143810efd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f14280099c0 tx=0x7f1428009cd0 comp rx=0 tx=0).stop 2024-03-04T08:58:00.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.588+0000 7f1440529700 1 -- 172.21.15.71:0/4251350069 shutdown_connections 2024-03-04T08:58:00.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.588+0000 7f1440529700 1 --2- 172.21.15.71:0/4251350069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1438107450 0x7f143810efd0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:00.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.588+0000 7f1440529700 1 --2- 172.21.15.71:0/4251350069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14381023a0 0x7f1438102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:00.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.588+0000 7f1440529700 1 --2- 172.21.15.71:0/4251350069 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f1438102d50 0x7f1438106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:00.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.588+0000 7f1440529700 1 -- 172.21.15.71:0/4251350069 >> 172.21.15.71:0/4251350069 conn(0x7f14380791e0 msgr2=0x7f14380795f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:00.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.588+0000 7f1440529700 1 -- 172.21.15.71:0/4251350069 shutdown_connections 2024-03-04T08:58:00.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.588+0000 7f1440529700 1 -- 172.21.15.71:0/4251350069 wait complete. 2024-03-04T08:58:00.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.589+0000 7f1440529700 1 Processor -- start 2024-03-04T08:58:00.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.589+0000 7f1440529700 1 -- start start 2024-03-04T08:58:00.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.589+0000 7f1440529700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f14381023a0 0x7f143819e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:00.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.589+0000 7f1440529700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1438102d50 0x7f143819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:00.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.589+0000 7f1440529700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1438107450 0x7f1438198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:00.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.589+0000 7f1440529700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1438112410 con 0x7f1438102d50 2024-03-04T08:58:00.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.589+0000 7f1440529700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1438112290 con 0x7f1438107450 2024-03-04T08:58:00.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.589+0000 7f1440529700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1438112590 con 0x7f14381023a0 2024-03-04T08:58:00.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.589+0000 7f143dac4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1438102d50 0x7f143819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:00.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.589+0000 7f143dac4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1438102d50 0x7f143819eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37324/0 (socket says 172.21.15.71:37324) 2024-03-04T08:58:00.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.589+0000 7f143eac6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1438107450 0x7f1438198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:00.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.589+0000 7f143dac4700 1 -- 172.21.15.71:0/824216628 learned_addr learned my addr 172.21.15.71:0/824216628 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:00.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.589+0000 7f143e2c5700 1 --2- 172.21.15.71:0/824216628 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f14381023a0 0x7f143819e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:00.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.589+0000 7f143eac6700 1 -- 172.21.15.71:0/824216628 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f14381023a0 msgr2=0x7f143819e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:00.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.589+0000 7f143eac6700 1 --2- 172.21.15.71:0/824216628 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f14381023a0 0x7f143819e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:00.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.589+0000 7f143eac6700 1 -- 172.21.15.71:0/824216628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1438102d50 msgr2=0x7f143819eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:00.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.589+0000 7f143eac6700 1 --2- 172.21.15.71:0/824216628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1438102d50 0x7f143819eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:00.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.589+0000 7f143eac6700 1 -- 172.21.15.71:0/824216628 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1438198f30 con 0x7f1438107450 2024-03-04T08:58:00.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.589+0000 7f143dac4700 1 --2- 172.21.15.71:0/824216628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1438102d50 0x7f143819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:58:00.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.590+0000 7f143eac6700 1 --2- 172.21.15.71:0/824216628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1438107450 0x7f1438198650 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f14280098e0 tx=0x7f142800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:00.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.590+0000 7f14377fe700 1 -- 172.21.15.71:0/824216628 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1428027070 con 0x7f1438107450 2024-03-04T08:58:00.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.590+0000 7f14377fe700 1 -- 172.21.15.71:0/824216628 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1428003680 con 0x7f1438107450 2024-03-04T08:58:00.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.590+0000 7f14377fe700 1 -- 172.21.15.71:0/824216628 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14280173f0 con 0x7f1438107450 2024-03-04T08:58:00.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.590+0000 7f1440529700 1 -- 172.21.15.71:0/824216628 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14381991c0 con 0x7f1438107450 2024-03-04T08:58:00.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.590+0000 7f1440529700 1 -- 172.21.15.71:0/824216628 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14381a5330 con 0x7f1438107450 2024-03-04T08:58:00.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.591+0000 7f1440529700 1 -- 172.21.15.71:0/824216628 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1438103800 con 0x7f1438107450 2024-03-04T08:58:00.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.592+0000 7f14377fe700 1 -- 172.21.15.71:0/824216628 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f142802a070 con 0x7f1438107450 2024-03-04T08:58:00.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.592+0000 7f14377fe700 1 --2- 172.21.15.71:0/824216628 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f14200a1ea0 0x7f14200a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:00.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.592+0000 7f143e2c5700 1 --2- 172.21.15.71:0/824216628 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f14200a1ea0 0x7f14200a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:00.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.593+0000 7f14377fe700 1 -- 172.21.15.71:0/824216628 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f14280ceeb0 con 0x7f1438107450 2024-03-04T08:58:00.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.593+0000 7f143e2c5700 1 --2- 172.21.15.71:0/824216628 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f14200a1ea0 0x7f14200a4360 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f143000f540 tx=0x7f143000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:00.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.597+0000 7f14377fe700 1 -- 172.21.15.71:0/824216628 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f142805e280 con 0x7f1438107450 2024-03-04T08:58:00.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.800+0000 7f1440529700 1 -- 172.21.15.71:0/824216628 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f143802d9b0 con 0x7f1438107450 2024-03-04T08:58:00.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.800+0000 7f14377fe700 1 -- 172.21.15.71:0/824216628 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1428013070 con 0x7f1438107450 2024-03-04T08:58:00.796 INFO:teuthology.orchestra.run.smithi071.stdout:55834574941 2024-03-04T08:58:00.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.802+0000 7f1440529700 1 -- 172.21.15.71:0/824216628 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f14200a1ea0 msgr2=0x7f14200a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:00.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.802+0000 7f1440529700 1 --2- 172.21.15.71:0/824216628 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f14200a1ea0 0x7f14200a4360 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f143000f540 tx=0x7f143000b040 comp rx=0 tx=0).stop 2024-03-04T08:58:00.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.802+0000 7f1440529700 1 -- 172.21.15.71:0/824216628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1438107450 msgr2=0x7f1438198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:00.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.802+0000 7f1440529700 1 --2- 172.21.15.71:0/824216628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1438107450 0x7f1438198650 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f14280098e0 tx=0x7f142800fa00 comp rx=0 tx=0).stop 2024-03-04T08:58:00.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.802+0000 7f1440529700 1 -- 172.21.15.71:0/824216628 shutdown_connections 2024-03-04T08:58:00.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.802+0000 7f1440529700 1 --2- 172.21.15.71:0/824216628 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f14200a1ea0 0x7f14200a4360 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:00.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.802+0000 7f1440529700 1 --2- 172.21.15.71:0/824216628 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f14381023a0 0x7f143819e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:00.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.802+0000 7f1440529700 1 --2- 172.21.15.71:0/824216628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1438107450 0x7f1438198650 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:00.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.802+0000 7f1440529700 1 --2- 172.21.15.71:0/824216628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1438102d50 0x7f143819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:00.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.802+0000 7f1440529700 1 -- 172.21.15.71:0/824216628 >> 172.21.15.71:0/824216628 conn(0x7f14380791e0 msgr2=0x7f1438100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:00.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.802+0000 7f1440529700 1 -- 172.21.15.71:0/824216628 shutdown_connections 2024-03-04T08:58:00.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:00.802+0000 7f1440529700 1 -- 172.21.15.71:0/824216628 wait complete. 2024-03-04T08:58:02.039 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:01 smithi071 ceph-mon[32789]: pgmap v372: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:02.039 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:01 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/824216628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-04T08:58:02.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:01 smithi158 ceph-mon[37241]: pgmap v372: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:02.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:01 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/824216628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-04T08:58:02.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:01 smithi150 ceph-mon[36807]: pgmap v372: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:02.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:01 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/824216628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-04T08:58:02.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.355+0000 7f8219a27700 1 -- 172.21.15.71:0/3330313081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8214100fd0 msgr2=0x7f821410efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:02.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.355+0000 7f8219a27700 1 --2- 172.21.15.71:0/3330313081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8214100fd0 0x7f821410efe0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f820c00b0d0 tx=0x7f820c00b3e0 comp rx=0 tx=0).stop 2024-03-04T08:58:02.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.355+0000 7f8219a27700 1 -- 172.21.15.71:0/3330313081 shutdown_connections 2024-03-04T08:58:02.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.355+0000 7f8219a27700 1 --2- 172.21.15.71:0/3330313081 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8214108210 0x7f82141085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:02.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.355+0000 7f8219a27700 1 --2- 172.21.15.71:0/3330313081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8214100fd0 0x7f821410efe0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:02.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.355+0000 7f8219a27700 1 --2- 172.21.15.71:0/3330313081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8214100610 0x7f8214100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:02.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.355+0000 7f8219a27700 1 -- 172.21.15.71:0/3330313081 >> 172.21.15.71:0/3330313081 conn(0x7f8214077590 msgr2=0x7f82140fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:02.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.355+0000 7f8219a27700 1 -- 172.21.15.71:0/3330313081 shutdown_connections 2024-03-04T08:58:02.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.355+0000 7f8219a27700 1 -- 172.21.15.71:0/3330313081 wait complete. 2024-03-04T08:58:02.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.356+0000 7f8219a27700 1 Processor -- start 2024-03-04T08:58:02.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.356+0000 7f8219a27700 1 -- start start 2024-03-04T08:58:02.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.356+0000 7f8219a27700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8214100610 0x7f821419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:02.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.356+0000 7f8219a27700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8214100fd0 0x7f821419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:02.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.356+0000 7f8219a27700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8214108210 0x7f8214198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:02.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.356+0000 7f8219a27700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82141124a0 con 0x7f8214108210 2024-03-04T08:58:02.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.356+0000 7f8219a27700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8214112320 con 0x7f8214100610 2024-03-04T08:58:02.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.356+0000 7f8219a27700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8214112620 con 0x7f8214100fd0 2024-03-04T08:58:02.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.357+0000 7f8212ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8214100610 0x7f821419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:02.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.357+0000 7f82137fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8214108210 0x7f8214198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:02.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.357+0000 7f8212ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8214100610 0x7f821419e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.71:60696/0 (socket says 172.21.15.71:60696) 2024-03-04T08:58:02.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.357+0000 7f8212ffd700 1 -- 172.21.15.71:0/671137661 learned_addr learned my addr 172.21.15.71:0/671137661 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:02.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.357+0000 7f82127fc700 1 --2- 172.21.15.71:0/671137661 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8214100fd0 0x7f821419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:02.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.357+0000 7f8212ffd700 1 -- 172.21.15.71:0/671137661 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8214100fd0 msgr2=0x7f821419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:02.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.357+0000 7f8212ffd700 1 --2- 172.21.15.71:0/671137661 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8214100fd0 0x7f821419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:02.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.357+0000 7f8212ffd700 1 -- 172.21.15.71:0/671137661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8214108210 msgr2=0x7f8214198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:02.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.357+0000 7f8212ffd700 1 --2- 172.21.15.71:0/671137661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8214108210 0x7f8214198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:02.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.357+0000 7f8212ffd700 1 -- 172.21.15.71:0/671137661 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8214199020 con 0x7f8214100610 2024-03-04T08:58:02.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.357+0000 7f8212ffd700 1 --2- 172.21.15.71:0/671137661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8214100610 0x7f821419e620 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f820400c970 tx=0x7f820400cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:02.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.357+0000 7f8218a25700 1 -- 172.21.15.71:0/671137661 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82040078d0 con 0x7f8214100610 2024-03-04T08:58:02.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.357+0000 7f8218a25700 1 -- 172.21.15.71:0/671137661 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8204018480 con 0x7f8214100610 2024-03-04T08:58:02.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.357+0000 7f8219a27700 1 -- 172.21.15.71:0/671137661 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8214199310 con 0x7f8214100610 2024-03-04T08:58:02.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.357+0000 7f8218a25700 1 -- 172.21.15.71:0/671137661 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820400f6b0 con 0x7f8214100610 2024-03-04T08:58:02.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.357+0000 7f8219a27700 1 -- 172.21.15.71:0/671137661 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82141a55a0 con 0x7f8214100610 2024-03-04T08:58:02.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.358+0000 7f8219a27700 1 -- 172.21.15.71:0/671137661 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8214103930 con 0x7f8214100610 2024-03-04T08:58:02.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.359+0000 7f8218a25700 1 -- 172.21.15.71:0/671137661 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f8204018630 con 0x7f8214100610 2024-03-04T08:58:02.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.360+0000 7f8218a25700 1 --2- 172.21.15.71:0/671137661 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f81fc0a1ea0 0x7f81fc0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:02.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.360+0000 7f82127fc700 1 --2- 172.21.15.71:0/671137661 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f81fc0a1ea0 0x7f81fc0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:02.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.360+0000 7f8218a25700 1 -- 172.21.15.71:0/671137661 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f82040ce450 con 0x7f8214100610 2024-03-04T08:58:02.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.360+0000 7f82127fc700 1 --2- 172.21.15.71:0/671137661 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f81fc0a1ea0 0x7f81fc0a4360 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8214199eb0 tx=0x7f8208009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:02.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.363+0000 7f8218a25700 1 -- 172.21.15.71:0/671137661 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f82040173e0 con 0x7f8214100610 2024-03-04T08:58:02.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.567+0000 7f8219a27700 1 -- 172.21.15.71:0/671137661 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f8214199c20 con 0x7f8214100610 2024-03-04T08:58:02.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.567+0000 7f8218a25700 1 -- 172.21.15.71:0/671137661 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f820409b450 con 0x7f8214100610 2024-03-04T08:58:02.565 INFO:teuthology.orchestra.run.smithi071.stdout:352187318305 2024-03-04T08:58:02.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.569+0000 7f8219a27700 1 -- 172.21.15.71:0/671137661 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f81fc0a1ea0 msgr2=0x7f81fc0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:02.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.569+0000 7f8219a27700 1 --2- 172.21.15.71:0/671137661 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f81fc0a1ea0 0x7f81fc0a4360 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8214199eb0 tx=0x7f8208009260 comp rx=0 tx=0).stop 2024-03-04T08:58:02.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.569+0000 7f8219a27700 1 -- 172.21.15.71:0/671137661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8214100610 msgr2=0x7f821419e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:02.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.569+0000 7f8219a27700 1 --2- 172.21.15.71:0/671137661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8214100610 0x7f821419e620 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f820400c970 tx=0x7f820400cc80 comp rx=0 tx=0).stop 2024-03-04T08:58:02.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.569+0000 7f8219a27700 1 -- 172.21.15.71:0/671137661 shutdown_connections 2024-03-04T08:58:02.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.569+0000 7f8219a27700 1 --2- 172.21.15.71:0/671137661 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f81fc0a1ea0 0x7f81fc0a4360 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:02.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.569+0000 7f8219a27700 1 --2- 172.21.15.71:0/671137661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8214108210 0x7f8214198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:02.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.569+0000 7f8219a27700 1 --2- 172.21.15.71:0/671137661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8214100610 0x7f821419e620 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:02.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.569+0000 7f8219a27700 1 --2- 172.21.15.71:0/671137661 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8214100fd0 0x7f821419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:02.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.569+0000 7f8219a27700 1 -- 172.21.15.71:0/671137661 >> 172.21.15.71:0/671137661 conn(0x7f8214077590 msgr2=0x7f8214104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:02.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.570+0000 7f8219a27700 1 -- 172.21.15.71:0/671137661 shutdown_connections 2024-03-04T08:58:02.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.570+0000 7f8219a27700 1 -- 172.21.15.71:0/671137661 wait complete. 2024-03-04T08:58:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.604+0000 7f3cb3831700 1 -- 172.21.15.71:0/3693750701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cac06a2b0 msgr2=0x7f3cac06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.604+0000 7f3cb3831700 1 --2- 172.21.15.71:0/3693750701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cac06a2b0 0x7f3cac06a710 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009990 tx=0x7f3c9c009ca0 comp rx=0 tx=0).stop 2024-03-04T08:58:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.604+0000 7f3cb3831700 1 -- 172.21.15.71:0/3693750701 shutdown_connections 2024-03-04T08:58:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.604+0000 7f3cb3831700 1 --2- 172.21.15.71:0/3693750701 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3cac106fb0 0x7f3cac069d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.604+0000 7f3cb3831700 1 --2- 172.21.15.71:0/3693750701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cac104ac0 0x7f3cac104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.604+0000 7f3cb3831700 1 --2- 172.21.15.71:0/3693750701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cac06a2b0 0x7f3cac06a710 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.604+0000 7f3cb3831700 1 -- 172.21.15.71:0/3693750701 >> 172.21.15.71:0/3693750701 conn(0x7f3cac077ef0 msgr2=0x7f3cac078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.604+0000 7f3cb3831700 1 -- 172.21.15.71:0/3693750701 shutdown_connections 2024-03-04T08:58:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.604+0000 7f3cb3831700 1 -- 172.21.15.71:0/3693750701 wait complete. 2024-03-04T08:58:02.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.604+0000 7f3cb3831700 1 Processor -- start 2024-03-04T08:58:02.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.604+0000 7f3cb3831700 1 -- start start 2024-03-04T08:58:02.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.605+0000 7f3cb3831700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3cac06a2b0 0x7f3cac19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:02.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.605+0000 7f3cb3831700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cac104ac0 0x7f3cac19f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:02.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.605+0000 7f3cb3831700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cac106fb0 0x7f3cac1a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:02.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.605+0000 7f3cb0dcc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cac104ac0 0x7f3cac19f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:02.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.605+0000 7f3cb3831700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac118950 con 0x7f3cac104ac0 2024-03-04T08:58:02.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.605+0000 7f3cb3831700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac1187d0 con 0x7f3cac106fb0 2024-03-04T08:58:02.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.605+0000 7f3cb3831700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac118ad0 con 0x7f3cac06a2b0 2024-03-04T08:58:02.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.605+0000 7f3cb0dcc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cac104ac0 0x7f3cac19f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37356/0 (socket says 172.21.15.71:37356) 2024-03-04T08:58:02.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.605+0000 7f3cb0dcc700 1 -- 172.21.15.71:0/436272073 learned_addr learned my addr 172.21.15.71:0/436272073 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:02.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.605+0000 7f3cb15cd700 1 --2- 172.21.15.71:0/436272073 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3cac06a2b0 0x7f3cac19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:02.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.605+0000 7f3cb1dce700 1 --2- 172.21.15.71:0/436272073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cac106fb0 0x7f3cac1a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:02.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.605+0000 7f3cb0dcc700 1 -- 172.21.15.71:0/436272073 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3cac06a2b0 msgr2=0x7f3cac19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:02.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.605+0000 7f3cb0dcc700 1 --2- 172.21.15.71:0/436272073 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3cac06a2b0 0x7f3cac19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:02.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.605+0000 7f3cb0dcc700 1 -- 172.21.15.71:0/436272073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cac106fb0 msgr2=0x7f3cac1a3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:02.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.605+0000 7f3cb0dcc700 1 --2- 172.21.15.71:0/436272073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cac106fb0 0x7f3cac1a3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:02.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.605+0000 7f3cb0dcc700 1 -- 172.21.15.71:0/436272073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cac1a3eb0 con 0x7f3cac104ac0 2024-03-04T08:58:02.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.605+0000 7f3cb1dce700 1 --2- 172.21.15.71:0/436272073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cac106fb0 0x7f3cac1a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:58:02.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.606+0000 7f3cb0dcc700 1 --2- 172.21.15.71:0/436272073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cac104ac0 0x7f3cac19f3e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c000c00 tx=0x7f3c9c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:02.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.606+0000 7f3ca27fc700 1 -- 172.21.15.71:0/436272073 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c027070 con 0x7f3cac104ac0 2024-03-04T08:58:02.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.606+0000 7f3ca27fc700 1 -- 172.21.15.71:0/436272073 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c9c003d70 con 0x7f3cac104ac0 2024-03-04T08:58:02.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.606+0000 7f3cb3831700 1 -- 172.21.15.71:0/436272073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cac1a4140 con 0x7f3cac104ac0 2024-03-04T08:58:02.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.606+0000 7f3ca27fc700 1 -- 172.21.15.71:0/436272073 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c00f6c0 con 0x7f3cac104ac0 2024-03-04T08:58:02.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.606+0000 7f3cb3831700 1 -- 172.21.15.71:0/436272073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cac1aba20 con 0x7f3cac104ac0 2024-03-04T08:58:02.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.607+0000 7f3cb3831700 1 -- 172.21.15.71:0/436272073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cac068d80 con 0x7f3cac104ac0 2024-03-04T08:58:02.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.608+0000 7f3ca27fc700 1 -- 172.21.15.71:0/436272073 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f3c9c02a070 con 0x7f3cac104ac0 2024-03-04T08:58:02.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.609+0000 7f3ca27fc700 1 --2- 172.21.15.71:0/436272073 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3c980a1f20 0x7f3c980a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:02.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.609+0000 7f3ca27fc700 1 -- 172.21.15.71:0/436272073 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f3c9c0cf790 con 0x7f3cac104ac0 2024-03-04T08:58:02.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.609+0000 7f3cb15cd700 1 --2- 172.21.15.71:0/436272073 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3c980a1f20 0x7f3c980a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:02.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.609+0000 7f3cb15cd700 1 --2- 172.21.15.71:0/436272073 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3c980a1f20 0x7f3c980a43e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8005950 tx=0x7f3ca800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:02.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.612+0000 7f3ca27fc700 1 -- 172.21.15.71:0/436272073 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3c9c031070 con 0x7f3cac104ac0 2024-03-04T08:58:02.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.817+0000 7f3cb3831700 1 -- 172.21.15.71:0/436272073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f3cac1abbf0 con 0x7f3cac104ac0 2024-03-04T08:58:02.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.817+0000 7f3ca27fc700 1 -- 172.21.15.71:0/436272073 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3c9c013070 con 0x7f3cac104ac0 2024-03-04T08:58:02.824 INFO:teuthology.orchestra.run.smithi071.stdout:146028888137 2024-03-04T08:58:02.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.820+0000 7f3cb3831700 1 -- 172.21.15.71:0/436272073 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3c980a1f20 msgr2=0x7f3c980a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:02.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.820+0000 7f3cb3831700 1 --2- 172.21.15.71:0/436272073 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3c980a1f20 0x7f3c980a43e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8005950 tx=0x7f3ca800b410 comp rx=0 tx=0).stop 2024-03-04T08:58:02.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.820+0000 7f3cb3831700 1 -- 172.21.15.71:0/436272073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cac104ac0 msgr2=0x7f3cac19f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:02.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.820+0000 7f3cb3831700 1 --2- 172.21.15.71:0/436272073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cac104ac0 0x7f3cac19f3e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c000c00 tx=0x7f3c9c00bab0 comp rx=0 tx=0).stop 2024-03-04T08:58:02.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.821+0000 7f3cb3831700 1 -- 172.21.15.71:0/436272073 shutdown_connections 2024-03-04T08:58:02.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.821+0000 7f3cb3831700 1 --2- 172.21.15.71:0/436272073 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3c980a1f20 0x7f3c980a43e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:02.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.821+0000 7f3cb3831700 1 --2- 172.21.15.71:0/436272073 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3cac06a2b0 0x7f3cac19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:02.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.821+0000 7f3cb3831700 1 --2- 172.21.15.71:0/436272073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cac106fb0 0x7f3cac1a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:02.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.821+0000 7f3cb3831700 1 --2- 172.21.15.71:0/436272073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cac104ac0 0x7f3cac19f3e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:02.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.821+0000 7f3cb3831700 1 -- 172.21.15.71:0/436272073 >> 172.21.15.71:0/436272073 conn(0x7f3cac077ef0 msgr2=0x7f3cac101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:02.828 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:02 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/671137661' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-04T08:58:02.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.821+0000 7f3cb3831700 1 -- 172.21.15.71:0/436272073 shutdown_connections 2024-03-04T08:58:02.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:02.821+0000 7f3cb3831700 1 -- 172.21.15.71:0/436272073 wait complete. 2024-03-04T08:58:03.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.031+0000 7fd1d5ef7700 1 -- 172.21.15.71:0/2024694174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1d01073e0 msgr2=0x7fd1d010efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:03.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.031+0000 7fd1d5ef7700 1 --2- 172.21.15.71:0/2024694174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1d01073e0 0x7fd1d010efb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800b0d0 tx=0x7fd1c800b3e0 comp rx=0 tx=0).stop 2024-03-04T08:58:03.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.031+0000 7fd1d5ef7700 1 -- 172.21.15.71:0/2024694174 shutdown_connections 2024-03-04T08:58:03.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.031+0000 7fd1d5ef7700 1 --2- 172.21.15.71:0/2024694174 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd1d0102300 0x7fd1d01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:03.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.031+0000 7fd1d5ef7700 1 --2- 172.21.15.71:0/2024694174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1d01073e0 0x7fd1d010efb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:03.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.031+0000 7fd1d5ef7700 1 --2- 172.21.15.71:0/2024694174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1d0102cb0 0x7fd1d0106d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:03.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.031+0000 7fd1d5ef7700 1 -- 172.21.15.71:0/2024694174 >> 172.21.15.71:0/2024694174 conn(0x7fd1d00fdb70 msgr2=0x7fd1d00fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:03.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.031+0000 7fd1d5ef7700 1 -- 172.21.15.71:0/2024694174 shutdown_connections 2024-03-04T08:58:03.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.031+0000 7fd1d5ef7700 1 -- 172.21.15.71:0/2024694174 wait complete. 2024-03-04T08:58:03.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.032+0000 7fd1d5ef7700 1 Processor -- start 2024-03-04T08:58:03.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.032+0000 7fd1d5ef7700 1 -- start start 2024-03-04T08:58:03.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.032+0000 7fd1d5ef7700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd1d0102300 0x7fd1d00743c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:03.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.032+0000 7fd1d5ef7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1d0102cb0 0x7fd1d0074900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:03.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.032+0000 7fd1d5ef7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1d01073e0 0x7fd1d0074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:03.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.032+0000 7fd1d5ef7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d0112440 con 0x7fd1d0102cb0 2024-03-04T08:58:03.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.032+0000 7fd1d5ef7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d01122c0 con 0x7fd1d01073e0 2024-03-04T08:58:03.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.032+0000 7fd1d5ef7700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d01125c0 con 0x7fd1d0102300 2024-03-04T08:58:03.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.032+0000 7fd1ceffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1d0102cb0 0x7fd1d0074900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:03.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.032+0000 7fd1ceffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1d0102cb0 0x7fd1d0074900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37372/0 (socket says 172.21.15.71:37372) 2024-03-04T08:58:03.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.032+0000 7fd1ceffd700 1 -- 172.21.15.71:0/767933355 learned_addr learned my addr 172.21.15.71:0/767933355 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:03.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.032+0000 7fd1cffff700 1 --2- 172.21.15.71:0/767933355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1d01073e0 0x7fd1d0074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:03.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.032+0000 7fd1cf7fe700 1 --2- 172.21.15.71:0/767933355 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd1d0102300 0x7fd1d00743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:03.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.032+0000 7fd1ceffd700 1 -- 172.21.15.71:0/767933355 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd1d0102300 msgr2=0x7fd1d00743c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:03.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.032+0000 7fd1ceffd700 1 --2- 172.21.15.71:0/767933355 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd1d0102300 0x7fd1d00743c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:03.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.032+0000 7fd1ceffd700 1 -- 172.21.15.71:0/767933355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1d01073e0 msgr2=0x7fd1d0074e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:03.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.032+0000 7fd1ceffd700 1 --2- 172.21.15.71:0/767933355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1d01073e0 0x7fd1d0074e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:03.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.033+0000 7fd1ceffd700 1 -- 172.21.15.71:0/767933355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d01a6da0 con 0x7fd1d0102cb0 2024-03-04T08:58:03.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.033+0000 7fd1cffff700 1 --2- 172.21.15.71:0/767933355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1d01073e0 0x7fd1d0074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:58:03.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.033+0000 7fd1ceffd700 1 --2- 172.21.15.71:0/767933355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1d0102cb0 0x7fd1d0074900 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000ea30 tx=0x7fd1c000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:03.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.033+0000 7fd1ccff9700 1 -- 172.21.15.71:0/767933355 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c000cbd0 con 0x7fd1d0102cb0 2024-03-04T08:58:03.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.033+0000 7fd1ccff9700 1 -- 172.21.15.71:0/767933355 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1c0004500 con 0x7fd1d0102cb0 2024-03-04T08:58:03.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.033+0000 7fd1d5ef7700 1 -- 172.21.15.71:0/767933355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d01a7090 con 0x7fd1d0102cb0 2024-03-04T08:58:03.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.034+0000 7fd1ccff9700 1 -- 172.21.15.71:0/767933355 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0003e30 con 0x7fd1d0102cb0 2024-03-04T08:58:03.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.034+0000 7fd1d5ef7700 1 -- 172.21.15.71:0/767933355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d01a75d0 con 0x7fd1d0102cb0 2024-03-04T08:58:03.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.034+0000 7fd1d5ef7700 1 -- 172.21.15.71:0/767933355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d0068d80 con 0x7fd1d0102cb0 2024-03-04T08:58:03.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.035+0000 7fd1ccff9700 1 -- 172.21.15.71:0/767933355 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fd1c0004000 con 0x7fd1d0102cb0 2024-03-04T08:58:03.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.036+0000 7fd1ccff9700 1 --2- 172.21.15.71:0/767933355 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd1b80a1ea0 0x7fd1b80a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:03.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.036+0000 7fd1cf7fe700 1 --2- 172.21.15.71:0/767933355 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd1b80a1ea0 0x7fd1b80a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:03.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.036+0000 7fd1ccff9700 1 -- 172.21.15.71:0/767933355 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fd1c00ce2f0 con 0x7fd1d0102cb0 2024-03-04T08:58:03.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.037+0000 7fd1cf7fe700 1 --2- 172.21.15.71:0/767933355 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd1b80a1ea0 0x7fd1b80a4360 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4004630 tx=0x7fd1c400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:03.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.041+0000 7fd1ccff9700 1 -- 172.21.15.71:0/767933355 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd1c009b260 con 0x7fd1d0102cb0 2024-03-04T08:58:03.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:02 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/671137661' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-04T08:58:03.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.244+0000 7fd1d5ef7700 1 -- 172.21.15.71:0/767933355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fd1d00758f0 con 0x7fd1d0102cb0 2024-03-04T08:58:03.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.245+0000 7fd1ccff9700 1 -- 172.21.15.71:0/767933355 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd1c009ab60 con 0x7fd1d0102cb0 2024-03-04T08:58:03.258 INFO:teuthology.orchestra.run.smithi071.stdout:201863462973 2024-03-04T08:58:03.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.247+0000 7fd1d5ef7700 1 -- 172.21.15.71:0/767933355 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd1b80a1ea0 msgr2=0x7fd1b80a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:03.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.247+0000 7fd1d5ef7700 1 --2- 172.21.15.71:0/767933355 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd1b80a1ea0 0x7fd1b80a4360 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4004630 tx=0x7fd1c400b3c0 comp rx=0 tx=0).stop 2024-03-04T08:58:03.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.247+0000 7fd1d5ef7700 1 -- 172.21.15.71:0/767933355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1d0102cb0 msgr2=0x7fd1d0074900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:03.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.247+0000 7fd1d5ef7700 1 --2- 172.21.15.71:0/767933355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1d0102cb0 0x7fd1d0074900 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000ea30 tx=0x7fd1c000ed40 comp rx=0 tx=0).stop 2024-03-04T08:58:03.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.247+0000 7fd1d5ef7700 1 -- 172.21.15.71:0/767933355 shutdown_connections 2024-03-04T08:58:03.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.247+0000 7fd1d5ef7700 1 --2- 172.21.15.71:0/767933355 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd1b80a1ea0 0x7fd1b80a4360 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:03.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.247+0000 7fd1d5ef7700 1 --2- 172.21.15.71:0/767933355 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd1d0102300 0x7fd1d00743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:03.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.247+0000 7fd1d5ef7700 1 --2- 172.21.15.71:0/767933355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1d01073e0 0x7fd1d0074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:03.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.247+0000 7fd1d5ef7700 1 --2- 172.21.15.71:0/767933355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1d0102cb0 0x7fd1d0074900 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:03.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.247+0000 7fd1d5ef7700 1 -- 172.21.15.71:0/767933355 >> 172.21.15.71:0/767933355 conn(0x7fd1d00fdb70 msgr2=0x7fd1d00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:03.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.247+0000 7fd1d5ef7700 1 -- 172.21.15.71:0/767933355 shutdown_connections 2024-03-04T08:58:03.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:03.247+0000 7fd1d5ef7700 1 -- 172.21.15.71:0/767933355 wait complete. 2024-03-04T08:58:03.289 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:02 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/671137661' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-04T08:58:03.824 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888132 got 146028888137 for osd.4 2024-03-04T08:58:03.824 DEBUG:teuthology.parallel:result is None 2024-03-04T08:58:04.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:03 smithi158 ceph-mon[37241]: pgmap v373: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:58:04.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:03 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/436272073' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-04T08:58:04.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:03 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/767933355' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-04T08:58:04.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:03 smithi071 ceph-mon[32789]: pgmap v373: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:58:04.091 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:03 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/436272073' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-04T08:58:04.091 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:03 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/767933355' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-04T08:58:04.117 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574932 got 55834574941 for osd.1 2024-03-04T08:58:04.117 DEBUG:teuthology.parallel:result is None 2024-03-04T08:58:04.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:03 smithi150 ceph-mon[36807]: pgmap v373: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:58:04.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:03 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/436272073' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-04T08:58:04.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:03 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/767933355' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-04T08:58:04.831 INFO:tasks.cephadm.ceph_manager.ceph:need seq 352187318298 got 352187318305 for osd.11 2024-03-04T08:58:04.831 DEBUG:teuthology.parallel:result is None 2024-03-04T08:58:05.439 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462966 got 201863462973 for osd.6 2024-03-04T08:58:05.439 DEBUG:teuthology.parallel:result is None 2024-03-04T08:58:05.439 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-04T08:58:05.439 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph pg dump --format=json 2024-03-04T08:58:05.714 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:58:06.081 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:05 smithi150 ceph-mon[36807]: pgmap v374: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:06.081 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:05 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:58:06.081 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:05 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:58:06.086 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:05 smithi071 ceph-mon[32789]: pgmap v374: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:06.086 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:05 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:58:06.086 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:05 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:58:06.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:05 smithi158 ceph-mon[37241]: pgmap v374: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:06.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:05 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:58:06.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:05 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:58:07.130 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:06 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:58:07.130 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:06 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:58:07.130 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:06 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:58:07.204 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:06 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:58:07.205 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:06 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:58:07.205 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:06 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:58:07.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:06 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:58:07.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:06 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:58:07.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:06 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' 2024-03-04T08:58:07.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.517+0000 7f013ad46700 1 -- 172.21.15.71:0/2236425479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0134100fc0 msgr2=0x7f0134101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:07.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.517+0000 7f013ad46700 1 --2- 172.21.15.71:0/2236425479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0134100fc0 0x7f0134101420 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0120009990 tx=0x7f0120009ca0 comp rx=0 tx=0).stop 2024-03-04T08:58:07.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.517+0000 7f013ad46700 1 -- 172.21.15.71:0/2236425479 shutdown_connections 2024-03-04T08:58:07.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.517+0000 7f013ad46700 1 --2- 172.21.15.71:0/2236425479 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0134101960 0x7f013410de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:07.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.517+0000 7f013ad46700 1 --2- 172.21.15.71:0/2236425479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0134107000 0x7f01341073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:07.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.517+0000 7f013ad46700 1 --2- 172.21.15.71:0/2236425479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0134100fc0 0x7f0134101420 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:07.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.517+0000 7f013ad46700 1 -- 172.21.15.71:0/2236425479 >> 172.21.15.71:0/2236425479 conn(0x7f0134077ef0 msgr2=0x7f0134078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:07.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.517+0000 7f013ad46700 1 -- 172.21.15.71:0/2236425479 shutdown_connections 2024-03-04T08:58:07.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.517+0000 7f013ad46700 1 -- 172.21.15.71:0/2236425479 wait complete. 2024-03-04T08:58:07.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.517+0000 7f013ad46700 1 Processor -- start 2024-03-04T08:58:07.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.518+0000 7f013ad46700 1 -- start start 2024-03-04T08:58:07.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.518+0000 7f013ad46700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0134100fc0 0x7f013410e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:07.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.518+0000 7f013ad46700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0134101960 0x7f013410ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:07.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.518+0000 7f013ad46700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0134107000 0x7f013410f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:07.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.518+0000 7f013ad46700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01341123e0 con 0x7f0134101960 2024-03-04T08:58:07.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.518+0000 7f013ad46700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0134112260 con 0x7f0134100fc0 2024-03-04T08:58:07.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.518+0000 7f013ad46700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0134112560 con 0x7f0134107000 2024-03-04T08:58:07.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.518+0000 7f0133fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0134101960 0x7f013410ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:07.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.518+0000 7f0133fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0134101960 0x7f013410ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42982/0 (socket says 172.21.15.71:42982) 2024-03-04T08:58:07.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.518+0000 7f0133fff700 1 -- 172.21.15.71:0/2820328879 learned_addr learned my addr 172.21.15.71:0/2820328879 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:07.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.518+0000 7f0138ae2700 1 --2- 172.21.15.71:0/2820328879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0134100fc0 0x7f013410e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:07.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.518+0000 7f01392e3700 1 --2- 172.21.15.71:0/2820328879 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0134107000 0x7f013410f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:07.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.518+0000 7f0133fff700 1 -- 172.21.15.71:0/2820328879 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0134107000 msgr2=0x7f013410f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:07.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.518+0000 7f0133fff700 1 --2- 172.21.15.71:0/2820328879 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0134107000 0x7f013410f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:07.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.518+0000 7f0133fff700 1 -- 172.21.15.71:0/2820328879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0134100fc0 msgr2=0x7f013410e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:07.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.518+0000 7f0133fff700 1 --2- 172.21.15.71:0/2820328879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0134100fc0 0x7f013410e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:07.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.518+0000 7f0133fff700 1 -- 172.21.15.71:0/2820328879 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01341a6d40 con 0x7f0134101960 2024-03-04T08:58:07.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.518+0000 7f01392e3700 1 --2- 172.21.15.71:0/2820328879 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0134107000 0x7f013410f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:58:07.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.518+0000 7f0133fff700 1 --2- 172.21.15.71:0/2820328879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0134101960 0x7f013410ec90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0120000c00 tx=0x7f012000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:07.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.518+0000 7f0131ffb700 1 -- 172.21.15.71:0/2820328879 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0120027070 con 0x7f0134101960 2024-03-04T08:58:07.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.519+0000 7f0131ffb700 1 -- 172.21.15.71:0/2820328879 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0120003d70 con 0x7f0134101960 2024-03-04T08:58:07.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.519+0000 7f013ad46700 1 -- 172.21.15.71:0/2820328879 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01341a6fd0 con 0x7f0134101960 2024-03-04T08:58:07.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.519+0000 7f0131ffb700 1 -- 172.21.15.71:0/2820328879 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012000f6c0 con 0x7f0134101960 2024-03-04T08:58:07.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.519+0000 7f013ad46700 1 -- 172.21.15.71:0/2820328879 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01341a7420 con 0x7f0134101960 2024-03-04T08:58:07.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.520+0000 7f013ad46700 1 -- 172.21.15.71:0/2820328879 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0134068d80 con 0x7f0134101960 2024-03-04T08:58:07.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.521+0000 7f0131ffb700 1 -- 172.21.15.71:0/2820328879 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f012002a070 con 0x7f0134101960 2024-03-04T08:58:07.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.521+0000 7f0131ffb700 1 --2- 172.21.15.71:0/2820328879 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f01240a1f20 0x7f01240a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:07.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.521+0000 7f0138ae2700 1 --2- 172.21.15.71:0/2820328879 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f01240a1f20 0x7f01240a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:07.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.521+0000 7f0131ffb700 1 -- 172.21.15.71:0/2820328879 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f01200cf6d0 con 0x7f0134101960 2024-03-04T08:58:07.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.522+0000 7f0138ae2700 1 --2- 172.21.15.71:0/2820328879 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f01240a1f20 0x7f01240a43e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0128009be0 tx=0x7f0128009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:07.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.525+0000 7f0131ffb700 1 -- 172.21.15.71:0/2820328879 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0120031070 con 0x7f0134101960 2024-03-04T08:58:07.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.718+0000 7f013ad46700 1 -- 172.21.15.71:0/2820328879 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0134062500 con 0x7f01240a1f20 2024-03-04T08:58:07.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.727+0000 7f0131ffb700 1 -- 172.21.15.71:0/2820328879 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340295 (secure 0 0 0) 0x7f0134062500 con 0x7f01240a1f20 2024-03-04T08:58:07.728 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:58:07.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.730+0000 7f013ad46700 1 -- 172.21.15.71:0/2820328879 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f01240a1f20 msgr2=0x7f01240a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:07.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.730+0000 7f013ad46700 1 --2- 172.21.15.71:0/2820328879 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f01240a1f20 0x7f01240a43e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0128009be0 tx=0x7f0128009260 comp rx=0 tx=0).stop 2024-03-04T08:58:07.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.730+0000 7f013ad46700 1 -- 172.21.15.71:0/2820328879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0134101960 msgr2=0x7f013410ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:07.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.730+0000 7f013ad46700 1 --2- 172.21.15.71:0/2820328879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0134101960 0x7f013410ec90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0120000c00 tx=0x7f012000bab0 comp rx=0 tx=0).stop 2024-03-04T08:58:07.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.730+0000 7f013ad46700 1 -- 172.21.15.71:0/2820328879 shutdown_connections 2024-03-04T08:58:07.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.730+0000 7f013ad46700 1 --2- 172.21.15.71:0/2820328879 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f01240a1f20 0x7f01240a43e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:07.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.730+0000 7f013ad46700 1 --2- 172.21.15.71:0/2820328879 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f0134107000 0x7f013410f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:07.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.730+0000 7f013ad46700 1 --2- 172.21.15.71:0/2820328879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0134100fc0 0x7f013410e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:07.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.730+0000 7f013ad46700 1 --2- 172.21.15.71:0/2820328879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0134101960 0x7f013410ec90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:07.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.730+0000 7f013ad46700 1 -- 172.21.15.71:0/2820328879 >> 172.21.15.71:0/2820328879 conn(0x7f0134077ef0 msgr2=0x7f01340ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:07.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.730+0000 7f013ad46700 1 -- 172.21.15.71:0/2820328879 shutdown_connections 2024-03-04T08:58:07.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:07.730+0000 7f013ad46700 1 -- 172.21.15.71:0/2820328879 wait complete. 2024-03-04T08:58:07.732 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-03-04T08:58:08.129 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:07 smithi071 ceph-mon[32789]: pgmap v375: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:58:08.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:07 smithi150 ceph-mon[36807]: pgmap v375: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:58:08.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:07 smithi158 ceph-mon[37241]: pgmap v375: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:58:08.495 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2024-03-04T08:58:06.714550+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654756,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":283,"num_read_kb":257,"num_write":262,"num_write_kb":3030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1244092,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359264,"kb_used_data":14192,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582560,"statfs":{"total":1151940427776,"available":1151572541440,"internally_reserved":0,"allocated":14532608,"data_stored":8056166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003525"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813847+0000","last_change":"2024-03-04T08:56:30.650344+0000","last_active":"2024-03-04T08:57:23.813847+0000","last_peered":"2024-03-04T08:57:23.813847+0000","last_clean":"2024-03-04T08:57:23.813847+0000","last_became_active":"2024-03-04T08:56:30.649842+0000","last_became_peered":"2024-03-04T08:56:30.649842+0000","last_unstale":"2024-03-04T08:57:23.813847+0000","last_undegraded":"2024-03-04T08:57:23.813847+0000","last_fullsized":"2024-03-04T08:57:23.813847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:04:57.162321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816882+0000","last_change":"2024-03-04T08:56:30.651616+0000","last_active":"2024-03-04T08:57:23.816882+0000","last_peered":"2024-03-04T08:57:23.816882+0000","last_clean":"2024-03-04T08:57:23.816882+0000","last_became_active":"2024-03-04T08:56:30.651283+0000","last_became_peered":"2024-03-04T08:56:30.651283+0000","last_unstale":"2024-03-04T08:57:23.816882+0000","last_undegraded":"2024-03-04T08:57:23.816882+0000","last_fullsized":"2024-03-04T08:57:23.816882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:18:48.018257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816784+0000","last_change":"2024-03-04T08:56:30.657459+0000","last_active":"2024-03-04T08:57:23.816784+0000","last_peered":"2024-03-04T08:57:23.816784+0000","last_clean":"2024-03-04T08:57:23.816784+0000","last_became_active":"2024-03-04T08:56:30.656993+0000","last_became_peered":"2024-03-04T08:56:30.656993+0000","last_unstale":"2024-03-04T08:57:23.816784+0000","last_undegraded":"2024-03-04T08:57:23.816784+0000","last_fullsized":"2024-03-04T08:57:23.816784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:07:34.898068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.818684+0000","last_change":"2024-03-04T08:57:23.818097+0000","last_active":"2024-03-04T08:57:23.818684+0000","last_peered":"2024-03-04T08:57:23.818684+0000","last_clean":"2024-03-04T08:57:23.818684+0000","last_became_active":"2024-03-04T08:57:23.817619+0000","last_became_peered":"2024-03-04T08:57:23.817619+0000","last_unstale":"2024-03-04T08:57:23.818684+0000","last_undegraded":"2024-03-04T08:57:23.818684+0000","last_fullsized":"2024-03-04T08:57:23.818684+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:48:15.249967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.815939+0000","last_change":"2024-03-04T08:56:30.654640+0000","last_active":"2024-03-04T08:57:23.815939+0000","last_peered":"2024-03-04T08:57:23.815939+0000","last_clean":"2024-03-04T08:57:23.815939+0000","last_became_active":"2024-03-04T08:56:30.654312+0000","last_became_peered":"2024-03-04T08:56:30.654312+0000","last_unstale":"2024-03-04T08:57:23.815939+0000","last_undegraded":"2024-03-04T08:57:23.815939+0000","last_fullsized":"2024-03-04T08:57:23.815939+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:09:43.930713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.810547+0000","last_change":"2024-03-04T08:56:30.647445+0000","last_active":"2024-03-04T08:57:23.810547+0000","last_peered":"2024-03-04T08:57:23.810547+0000","last_clean":"2024-03-04T08:57:23.810547+0000","last_became_active":"2024-03-04T08:56:30.647130+0000","last_became_peered":"2024-03-04T08:56:30.647130+0000","last_unstale":"2024-03-04T08:57:23.810547+0000","last_undegraded":"2024-03-04T08:57:23.810547+0000","last_fullsized":"2024-03-04T08:57:23.810547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:40:18.906604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816002+0000","last_change":"2024-03-04T08:56:30.645776+0000","last_active":"2024-03-04T08:57:23.816002+0000","last_peered":"2024-03-04T08:57:23.816002+0000","last_clean":"2024-03-04T08:57:23.816002+0000","last_became_active":"2024-03-04T08:56:30.645107+0000","last_became_peered":"2024-03-04T08:56:30.645107+0000","last_unstale":"2024-03-04T08:57:23.816002+0000","last_undegraded":"2024-03-04T08:57:23.816002+0000","last_fullsized":"2024-03-04T08:57:23.816002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:44:35.230491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.160395+0000","last_change":"2024-03-04T08:56:30.646424+0000","last_active":"2024-03-04T08:57:24.160395+0000","last_peered":"2024-03-04T08:57:24.160395+0000","last_clean":"2024-03-04T08:57:24.160395+0000","last_became_active":"2024-03-04T08:56:30.646115+0000","last_became_peered":"2024-03-04T08:56:30.646115+0000","last_unstale":"2024-03-04T08:57:24.160395+0000","last_undegraded":"2024-03-04T08:57:24.160395+0000","last_fullsized":"2024-03-04T08:57:24.160395+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:42:36.201984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813891+0000","last_change":"2024-03-04T08:56:30.655027+0000","last_active":"2024-03-04T08:57:23.813891+0000","last_peered":"2024-03-04T08:57:23.813891+0000","last_clean":"2024-03-04T08:57:23.813891+0000","last_became_active":"2024-03-04T08:56:30.654766+0000","last_became_peered":"2024-03-04T08:56:30.654766+0000","last_unstale":"2024-03-04T08:57:23.813891+0000","last_undegraded":"2024-03-04T08:57:23.813891+0000","last_fullsized":"2024-03-04T08:57:23.813891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:00:34.168740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.811005+0000","last_change":"2024-03-04T08:56:30.648268+0000","last_active":"2024-03-04T08:57:23.811005+0000","last_peered":"2024-03-04T08:57:23.811005+0000","last_clean":"2024-03-04T08:57:23.811005+0000","last_became_active":"2024-03-04T08:56:30.647905+0000","last_became_peered":"2024-03-04T08:56:30.647905+0000","last_unstale":"2024-03-04T08:57:23.811005+0000","last_undegraded":"2024-03-04T08:57:23.811005+0000","last_fullsized":"2024-03-04T08:57:23.811005+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:07:32.387510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816537+0000","last_change":"2024-03-04T08:56:30.648061+0000","last_active":"2024-03-04T08:57:23.816537+0000","last_peered":"2024-03-04T08:57:23.816537+0000","last_clean":"2024-03-04T08:57:23.816537+0000","last_became_active":"2024-03-04T08:56:30.647740+0000","last_became_peered":"2024-03-04T08:56:30.647740+0000","last_unstale":"2024-03-04T08:57:23.816537+0000","last_undegraded":"2024-03-04T08:57:23.816537+0000","last_fullsized":"2024-03-04T08:57:23.816537+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:35:23.065902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813785+0000","last_change":"2024-03-04T08:56:30.645802+0000","last_active":"2024-03-04T08:57:23.813785+0000","last_peered":"2024-03-04T08:57:23.813785+0000","last_clean":"2024-03-04T08:57:23.813785+0000","last_became_active":"2024-03-04T08:56:30.645474+0000","last_became_peered":"2024-03-04T08:56:30.645474+0000","last_unstale":"2024-03-04T08:57:23.813785+0000","last_undegraded":"2024-03-04T08:57:23.813785+0000","last_fullsized":"2024-03-04T08:57:23.813785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:35:40.207215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.812978+0000","last_change":"2024-03-04T08:56:30.656149+0000","last_active":"2024-03-04T08:57:23.812978+0000","last_peered":"2024-03-04T08:57:23.812978+0000","last_clean":"2024-03-04T08:57:23.812978+0000","last_became_active":"2024-03-04T08:56:30.655493+0000","last_became_peered":"2024-03-04T08:56:30.655493+0000","last_unstale":"2024-03-04T08:57:23.812978+0000","last_undegraded":"2024-03-04T08:57:23.812978+0000","last_fullsized":"2024-03-04T08:57:23.812978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:23:28.320328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816645+0000","last_change":"2024-03-04T08:56:30.658309+0000","last_active":"2024-03-04T08:57:23.816645+0000","last_peered":"2024-03-04T08:57:23.816645+0000","last_clean":"2024-03-04T08:57:23.816645+0000","last_became_active":"2024-03-04T08:56:30.656773+0000","last_became_peered":"2024-03-04T08:56:30.656773+0000","last_unstale":"2024-03-04T08:57:23.816645+0000","last_undegraded":"2024-03-04T08:57:23.816645+0000","last_fullsized":"2024-03-04T08:57:23.816645+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:26:57.021442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816806+0000","last_change":"2024-03-04T08:56:30.650336+0000","last_active":"2024-03-04T08:57:23.816806+0000","last_peered":"2024-03-04T08:57:23.816806+0000","last_clean":"2024-03-04T08:57:23.816806+0000","last_became_active":"2024-03-04T08:56:30.648377+0000","last_became_peered":"2024-03-04T08:56:30.648377+0000","last_unstale":"2024-03-04T08:57:23.816806+0000","last_undegraded":"2024-03-04T08:57:23.816806+0000","last_fullsized":"2024-03-04T08:57:23.816806+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:56:41.358895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813248+0000","last_change":"2024-03-04T08:56:30.653327+0000","last_active":"2024-03-04T08:57:23.813248+0000","last_peered":"2024-03-04T08:57:23.813248+0000","last_clean":"2024-03-04T08:57:23.813248+0000","last_became_active":"2024-03-04T08:56:30.652523+0000","last_became_peered":"2024-03-04T08:56:30.652523+0000","last_unstale":"2024-03-04T08:57:23.813248+0000","last_undegraded":"2024-03-04T08:57:23.813248+0000","last_fullsized":"2024-03-04T08:57:23.813248+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:18:41.726921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813425+0000","last_change":"2024-03-04T08:56:30.647210+0000","last_active":"2024-03-04T08:57:23.813425+0000","last_peered":"2024-03-04T08:57:23.813425+0000","last_clean":"2024-03-04T08:57:23.813425+0000","last_became_active":"2024-03-04T08:56:30.646897+0000","last_became_peered":"2024-03-04T08:56:30.646897+0000","last_unstale":"2024-03-04T08:57:23.813425+0000","last_undegraded":"2024-03-04T08:57:23.813425+0000","last_fullsized":"2024-03-04T08:57:23.813425+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:40:35.021061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.814452+0000","last_change":"2024-03-04T08:56:30.648751+0000","last_active":"2024-03-04T08:57:23.814452+0000","last_peered":"2024-03-04T08:57:23.814452+0000","last_clean":"2024-03-04T08:57:23.814452+0000","last_became_active":"2024-03-04T08:56:30.648457+0000","last_became_peered":"2024-03-04T08:56:30.648457+0000","last_unstale":"2024-03-04T08:57:23.814452+0000","last_undegraded":"2024-03-04T08:57:23.814452+0000","last_fullsized":"2024-03-04T08:57:23.814452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:20:20.435769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.811779+0000","last_change":"2024-03-04T08:56:30.644523+0000","last_active":"2024-03-04T08:57:23.811779+0000","last_peered":"2024-03-04T08:57:23.811779+0000","last_clean":"2024-03-04T08:57:23.811779+0000","last_became_active":"2024-03-04T08:56:30.644183+0000","last_became_peered":"2024-03-04T08:56:30.644183+0000","last_unstale":"2024-03-04T08:57:23.811779+0000","last_undegraded":"2024-03-04T08:57:23.811779+0000","last_fullsized":"2024-03-04T08:57:23.811779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:58:16.065571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.811048+0000","last_change":"2024-03-04T08:56:34.681350+0000","last_active":"2024-03-04T08:57:23.811048+0000","last_peered":"2024-03-04T08:57:23.811048+0000","last_clean":"2024-03-04T08:57:23.811048+0000","last_became_active":"2024-03-04T08:56:34.680940+0000","last_became_peered":"2024-03-04T08:56:34.680940+0000","last_unstale":"2024-03-04T08:57:23.811048+0000","last_undegraded":"2024-03-04T08:57:23.811048+0000","last_fullsized":"2024-03-04T08:57:23.811048+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:41:00.690541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.814319+0000","last_change":"2024-03-04T08:56:29.989827+0000","last_active":"2024-03-04T08:57:23.814319+0000","last_peered":"2024-03-04T08:57:23.814319+0000","last_clean":"2024-03-04T08:57:23.814319+0000","last_became_active":"2024-03-04T08:56:29.989509+0000","last_became_peered":"2024-03-04T08:56:29.989509+0000","last_unstale":"2024-03-04T08:57:23.814319+0000","last_undegraded":"2024-03-04T08:57:23.814319+0000","last_fullsized":"2024-03-04T08:57:23.814319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:27:37.771477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.160609+0000","last_change":"2024-03-04T08:56:30.638354+0000","last_active":"2024-03-04T08:57:24.160609+0000","last_peered":"2024-03-04T08:57:24.160609+0000","last_clean":"2024-03-04T08:57:24.160609+0000","last_became_active":"2024-03-04T08:56:30.638058+0000","last_became_peered":"2024-03-04T08:56:30.638058+0000","last_unstale":"2024-03-04T08:57:24.160609+0000","last_undegraded":"2024-03-04T08:57:24.160609+0000","last_fullsized":"2024-03-04T08:57:24.160609+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:43:51.920411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813316+0000","last_change":"2024-03-04T08:56:34.680206+0000","last_active":"2024-03-04T08:57:23.813316+0000","last_peered":"2024-03-04T08:57:23.813316+0000","last_clean":"2024-03-04T08:57:23.813316+0000","last_became_active":"2024-03-04T08:56:34.679821+0000","last_became_peered":"2024-03-04T08:56:34.679821+0000","last_unstale":"2024-03-04T08:57:23.813316+0000","last_undegraded":"2024-03-04T08:57:23.813316+0000","last_fullsized":"2024-03-04T08:57:23.813316+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:18:39.945962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'1","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.161028+0000","last_change":"2024-03-04T08:56:29.994017+0000","last_active":"2024-03-04T08:57:24.161028+0000","last_peered":"2024-03-04T08:57:24.161028+0000","last_clean":"2024-03-04T08:57:24.161028+0000","last_became_active":"2024-03-04T08:56:29.993687+0000","last_became_peered":"2024-03-04T08:56:29.993687+0000","last_unstale":"2024-03-04T08:57:24.161028+0000","last_undegraded":"2024-03-04T08:57:24.161028+0000","last_fullsized":"2024-03-04T08:57:24.161028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:32:51.542130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816984+0000","last_change":"2024-03-04T08:56:30.654446+0000","last_active":"2024-03-04T08:57:23.816984+0000","last_peered":"2024-03-04T08:57:23.816984+0000","last_clean":"2024-03-04T08:57:23.816984+0000","last_became_active":"2024-03-04T08:56:30.653920+0000","last_became_peered":"2024-03-04T08:56:30.653920+0000","last_unstale":"2024-03-04T08:57:23.816984+0000","last_undegraded":"2024-03-04T08:57:23.816984+0000","last_fullsized":"2024-03-04T08:57:23.816984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:48:43.177893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813859+0000","last_change":"2024-03-04T08:56:34.680888+0000","last_active":"2024-03-04T08:57:23.813859+0000","last_peered":"2024-03-04T08:57:23.813859+0000","last_clean":"2024-03-04T08:57:23.813859+0000","last_became_active":"2024-03-04T08:56:34.679950+0000","last_became_peered":"2024-03-04T08:56:34.679950+0000","last_unstale":"2024-03-04T08:57:23.813859+0000","last_undegraded":"2024-03-04T08:57:23.813859+0000","last_fullsized":"2024-03-04T08:57:23.813859+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:08:26.660124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816585+0000","last_change":"2024-03-04T08:56:29.986476+0000","last_active":"2024-03-04T08:57:23.816585+0000","last_peered":"2024-03-04T08:57:23.816585+0000","last_clean":"2024-03-04T08:57:23.816585+0000","last_became_active":"2024-03-04T08:56:29.986106+0000","last_became_peered":"2024-03-04T08:56:29.986106+0000","last_unstale":"2024-03-04T08:57:23.816585+0000","last_undegraded":"2024-03-04T08:57:23.816585+0000","last_fullsized":"2024-03-04T08:57:23.816585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:45:15.775106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.814028+0000","last_change":"2024-03-04T08:56:30.645841+0000","last_active":"2024-03-04T08:57:23.814028+0000","last_peered":"2024-03-04T08:57:23.814028+0000","last_clean":"2024-03-04T08:57:23.814028+0000","last_became_active":"2024-03-04T08:56:30.645459+0000","last_became_peered":"2024-03-04T08:56:30.645459+0000","last_unstale":"2024-03-04T08:57:23.814028+0000","last_undegraded":"2024-03-04T08:57:23.814028+0000","last_fullsized":"2024-03-04T08:57:23.814028+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:37:51.204517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813447+0000","last_change":"2024-03-04T08:56:34.674562+0000","last_active":"2024-03-04T08:57:23.813447+0000","last_peered":"2024-03-04T08:57:23.813447+0000","last_clean":"2024-03-04T08:57:23.813447+0000","last_became_active":"2024-03-04T08:56:34.674239+0000","last_became_peered":"2024-03-04T08:56:34.674239+0000","last_unstale":"2024-03-04T08:57:23.813447+0000","last_undegraded":"2024-03-04T08:57:23.813447+0000","last_fullsized":"2024-03-04T08:57:23.813447+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:47:34.819487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816862+0000","last_change":"2024-03-04T08:56:29.984025+0000","last_active":"2024-03-04T08:57:23.816862+0000","last_peered":"2024-03-04T08:57:23.816862+0000","last_clean":"2024-03-04T08:57:23.816862+0000","last_became_active":"2024-03-04T08:56:29.983676+0000","last_became_peered":"2024-03-04T08:56:29.983676+0000","last_unstale":"2024-03-04T08:57:23.816862+0000","last_undegraded":"2024-03-04T08:57:23.816862+0000","last_fullsized":"2024-03-04T08:57:23.816862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:48:24.920513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.820557+0000","last_change":"2024-03-04T08:56:30.649301+0000","last_active":"2024-03-04T08:57:23.820557+0000","last_peered":"2024-03-04T08:57:23.820557+0000","last_clean":"2024-03-04T08:57:23.820557+0000","last_became_active":"2024-03-04T08:56:30.643987+0000","last_became_peered":"2024-03-04T08:56:30.643987+0000","last_unstale":"2024-03-04T08:57:23.820557+0000","last_undegraded":"2024-03-04T08:57:23.820557+0000","last_fullsized":"2024-03-04T08:57:23.820557+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:05:21.171953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.811629+0000","last_change":"2024-03-04T08:56:34.672893+0000","last_active":"2024-03-04T08:57:23.811629+0000","last_peered":"2024-03-04T08:57:23.811629+0000","last_clean":"2024-03-04T08:57:23.811629+0000","last_became_active":"2024-03-04T08:56:34.672364+0000","last_became_peered":"2024-03-04T08:56:34.672364+0000","last_unstale":"2024-03-04T08:57:23.811629+0000","last_undegraded":"2024-03-04T08:57:23.811629+0000","last_fullsized":"2024-03-04T08:57:23.811629+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:33:29.287772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'1","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.819456+0000","last_change":"2024-03-04T08:56:29.986822+0000","last_active":"2024-03-04T08:57:23.819456+0000","last_peered":"2024-03-04T08:57:23.819456+0000","last_clean":"2024-03-04T08:57:23.819456+0000","last_became_active":"2024-03-04T08:56:29.986498+0000","last_became_peered":"2024-03-04T08:56:29.986498+0000","last_unstale":"2024-03-04T08:57:23.819456+0000","last_undegraded":"2024-03-04T08:57:23.819456+0000","last_fullsized":"2024-03-04T08:57:23.819456+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:15:01.585997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.815645+0000","last_change":"2024-03-04T08:56:30.646164+0000","last_active":"2024-03-04T08:57:23.815645+0000","last_peered":"2024-03-04T08:57:23.815645+0000","last_clean":"2024-03-04T08:57:23.815645+0000","last_became_active":"2024-03-04T08:56:30.645843+0000","last_became_peered":"2024-03-04T08:56:30.645843+0000","last_unstale":"2024-03-04T08:57:23.815645+0000","last_undegraded":"2024-03-04T08:57:23.815645+0000","last_fullsized":"2024-03-04T08:57:23.815645+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:11:48.082226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.811857+0000","last_change":"2024-03-04T08:56:34.680387+0000","last_active":"2024-03-04T08:57:23.811857+0000","last_peered":"2024-03-04T08:57:23.811857+0000","last_clean":"2024-03-04T08:57:23.811857+0000","last_became_active":"2024-03-04T08:56:34.680005+0000","last_became_peered":"2024-03-04T08:56:34.680005+0000","last_unstale":"2024-03-04T08:57:23.811857+0000","last_undegraded":"2024-03-04T08:57:23.811857+0000","last_fullsized":"2024-03-04T08:57:23.811857+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:04:40.403187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.815686+0000","last_change":"2024-03-04T08:57:23.815686+0000","last_active":"2024-03-04T08:57:23.815686+0000","last_peered":"2024-03-04T08:57:23.815686+0000","last_clean":"2024-03-04T08:57:23.815686+0000","last_became_active":"2024-03-04T08:57:23.815243+0000","last_became_peered":"2024-03-04T08:57:23.815243+0000","last_unstale":"2024-03-04T08:57:23.815686+0000","last_undegraded":"2024-03-04T08:57:23.815686+0000","last_fullsized":"2024-03-04T08:57:23.815686+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:56:49.407722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.819641+0000","last_change":"2024-03-04T08:56:30.649321+0000","last_active":"2024-03-04T08:57:23.819641+0000","last_peered":"2024-03-04T08:57:23.819641+0000","last_clean":"2024-03-04T08:57:23.819641+0000","last_became_active":"2024-03-04T08:56:30.648705+0000","last_became_peered":"2024-03-04T08:56:30.648705+0000","last_unstale":"2024-03-04T08:57:23.819641+0000","last_undegraded":"2024-03-04T08:57:23.819641+0000","last_fullsized":"2024-03-04T08:57:23.819641+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:26:12.549471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813388+0000","last_change":"2024-03-04T08:56:34.682560+0000","last_active":"2024-03-04T08:57:23.813388+0000","last_peered":"2024-03-04T08:57:23.813388+0000","last_clean":"2024-03-04T08:57:23.813388+0000","last_became_active":"2024-03-04T08:56:34.682156+0000","last_became_peered":"2024-03-04T08:56:34.682156+0000","last_unstale":"2024-03-04T08:57:23.813388+0000","last_undegraded":"2024-03-04T08:57:23.813388+0000","last_fullsized":"2024-03-04T08:57:23.813388+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:57:49.132436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.812575+0000","last_change":"2024-03-04T08:56:29.984947+0000","last_active":"2024-03-04T08:57:23.812575+0000","last_peered":"2024-03-04T08:57:23.812575+0000","last_clean":"2024-03-04T08:57:23.812575+0000","last_became_active":"2024-03-04T08:56:29.984575+0000","last_became_peered":"2024-03-04T08:56:29.984575+0000","last_unstale":"2024-03-04T08:57:23.812575+0000","last_undegraded":"2024-03-04T08:57:23.812575+0000","last_fullsized":"2024-03-04T08:57:23.812575+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:01:16.698417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.160555+0000","last_change":"2024-03-04T08:56:30.649000+0000","last_active":"2024-03-04T08:57:24.160555+0000","last_peered":"2024-03-04T08:57:24.160555+0000","last_clean":"2024-03-04T08:57:24.160555+0000","last_became_active":"2024-03-04T08:56:30.648700+0000","last_became_peered":"2024-03-04T08:56:30.648700+0000","last_unstale":"2024-03-04T08:57:24.160555+0000","last_undegraded":"2024-03-04T08:57:24.160555+0000","last_fullsized":"2024-03-04T08:57:24.160555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:46:09.384040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816872+0000","last_change":"2024-03-04T08:56:34.671042+0000","last_active":"2024-03-04T08:57:23.816872+0000","last_peered":"2024-03-04T08:57:23.816872+0000","last_clean":"2024-03-04T08:57:23.816872+0000","last_became_active":"2024-03-04T08:56:34.670683+0000","last_became_peered":"2024-03-04T08:56:34.670683+0000","last_unstale":"2024-03-04T08:57:23.816872+0000","last_undegraded":"2024-03-04T08:57:23.816872+0000","last_fullsized":"2024-03-04T08:57:23.816872+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:13:09.166986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.819316+0000","last_change":"2024-03-04T08:56:29.983359+0000","last_active":"2024-03-04T08:57:23.819316+0000","last_peered":"2024-03-04T08:57:23.819316+0000","last_clean":"2024-03-04T08:57:23.819316+0000","last_became_active":"2024-03-04T08:56:29.983009+0000","last_became_peered":"2024-03-04T08:56:29.983009+0000","last_unstale":"2024-03-04T08:57:23.819316+0000","last_undegraded":"2024-03-04T08:57:23.819316+0000","last_fullsized":"2024-03-04T08:57:23.819316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:03:16.549109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813656+0000","last_change":"2024-03-04T08:56:30.649612+0000","last_active":"2024-03-04T08:57:23.813656+0000","last_peered":"2024-03-04T08:57:23.813656+0000","last_clean":"2024-03-04T08:57:23.813656+0000","last_became_active":"2024-03-04T08:56:30.649299+0000","last_became_peered":"2024-03-04T08:56:30.649299+0000","last_unstale":"2024-03-04T08:57:23.813656+0000","last_undegraded":"2024-03-04T08:57:23.813656+0000","last_fullsized":"2024-03-04T08:57:23.813656+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:57:50.571696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.160731+0000","last_change":"2024-03-04T08:56:34.677974+0000","last_active":"2024-03-04T08:57:24.160731+0000","last_peered":"2024-03-04T08:57:24.160731+0000","last_clean":"2024-03-04T08:57:24.160731+0000","last_became_active":"2024-03-04T08:56:34.676628+0000","last_became_peered":"2024-03-04T08:56:34.676628+0000","last_unstale":"2024-03-04T08:57:24.160731+0000","last_undegraded":"2024-03-04T08:57:24.160731+0000","last_fullsized":"2024-03-04T08:57:24.160731+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:18:40.301336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.810975+0000","last_change":"2024-03-04T08:56:29.976760+0000","last_active":"2024-03-04T08:57:23.810975+0000","last_peered":"2024-03-04T08:57:23.810975+0000","last_clean":"2024-03-04T08:57:23.810975+0000","last_became_active":"2024-03-04T08:56:29.976446+0000","last_became_peered":"2024-03-04T08:56:29.976446+0000","last_unstale":"2024-03-04T08:57:23.810975+0000","last_undegraded":"2024-03-04T08:57:23.810975+0000","last_fullsized":"2024-03-04T08:57:23.810975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:46:35.038622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813548+0000","last_change":"2024-03-04T08:56:30.644326+0000","last_active":"2024-03-04T08:57:23.813548+0000","last_peered":"2024-03-04T08:57:23.813548+0000","last_clean":"2024-03-04T08:57:23.813548+0000","last_became_active":"2024-03-04T08:56:30.644017+0000","last_became_peered":"2024-03-04T08:56:30.644017+0000","last_unstale":"2024-03-04T08:57:23.813548+0000","last_undegraded":"2024-03-04T08:57:23.813548+0000","last_fullsized":"2024-03-04T08:57:23.813548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:01:40.843915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813546+0000","last_change":"2024-03-04T08:56:34.675757+0000","last_active":"2024-03-04T08:57:23.813546+0000","last_peered":"2024-03-04T08:57:23.813546+0000","last_clean":"2024-03-04T08:57:23.813546+0000","last_became_active":"2024-03-04T08:56:34.675375+0000","last_became_peered":"2024-03-04T08:56:34.675375+0000","last_unstale":"2024-03-04T08:57:23.813546+0000","last_undegraded":"2024-03-04T08:57:23.813546+0000","last_fullsized":"2024-03-04T08:57:23.813546+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:16:12.762024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816521+0000","last_change":"2024-03-04T08:56:29.987614+0000","last_active":"2024-03-04T08:57:23.816521+0000","last_peered":"2024-03-04T08:57:23.816521+0000","last_clean":"2024-03-04T08:57:23.816521+0000","last_became_active":"2024-03-04T08:56:29.986639+0000","last_became_peered":"2024-03-04T08:56:29.986639+0000","last_unstale":"2024-03-04T08:57:23.816521+0000","last_undegraded":"2024-03-04T08:57:23.816521+0000","last_fullsized":"2024-03-04T08:57:23.816521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:18:50.320069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816761+0000","last_change":"2024-03-04T08:56:30.655145+0000","last_active":"2024-03-04T08:57:23.816761+0000","last_peered":"2024-03-04T08:57:23.816761+0000","last_clean":"2024-03-04T08:57:23.816761+0000","last_became_active":"2024-03-04T08:56:30.654834+0000","last_became_peered":"2024-03-04T08:56:30.654834+0000","last_unstale":"2024-03-04T08:57:23.816761+0000","last_undegraded":"2024-03-04T08:57:23.816761+0000","last_fullsized":"2024-03-04T08:57:23.816761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:45:19.567325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.814077+0000","last_change":"2024-03-04T08:56:34.680530+0000","last_active":"2024-03-04T08:57:23.814077+0000","last_peered":"2024-03-04T08:57:23.814077+0000","last_clean":"2024-03-04T08:57:23.814077+0000","last_became_active":"2024-03-04T08:56:34.679860+0000","last_became_peered":"2024-03-04T08:56:34.679860+0000","last_unstale":"2024-03-04T08:57:23.814077+0000","last_undegraded":"2024-03-04T08:57:23.814077+0000","last_fullsized":"2024-03-04T08:57:23.814077+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:25:09.036145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'3","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.820442+0000","last_change":"2024-03-04T08:56:29.982516+0000","last_active":"2024-03-04T08:57:23.820442+0000","last_peered":"2024-03-04T08:57:23.820442+0000","last_clean":"2024-03-04T08:57:23.820442+0000","last_became_active":"2024-03-04T08:56:29.982166+0000","last_became_peered":"2024-03-04T08:56:29.982166+0000","last_unstale":"2024-03-04T08:57:23.820442+0000","last_undegraded":"2024-03-04T08:57:23.820442+0000","last_fullsized":"2024-03-04T08:57:23.820442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:48:57.463584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.814193+0000","last_change":"2024-03-04T08:56:30.650030+0000","last_active":"2024-03-04T08:57:23.814193+0000","last_peered":"2024-03-04T08:57:23.814193+0000","last_clean":"2024-03-04T08:57:23.814193+0000","last_became_active":"2024-03-04T08:56:30.647826+0000","last_became_peered":"2024-03-04T08:56:30.647826+0000","last_unstale":"2024-03-04T08:57:23.814193+0000","last_undegraded":"2024-03-04T08:57:23.814193+0000","last_fullsized":"2024-03-04T08:57:23.814193+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:07:45.189935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813823+0000","last_change":"2024-03-04T08:56:34.679189+0000","last_active":"2024-03-04T08:57:23.813823+0000","last_peered":"2024-03-04T08:57:23.813823+0000","last_clean":"2024-03-04T08:57:23.813823+0000","last_became_active":"2024-03-04T08:56:34.678744+0000","last_became_peered":"2024-03-04T08:56:34.678744+0000","last_unstale":"2024-03-04T08:57:23.813823+0000","last_undegraded":"2024-03-04T08:57:23.813823+0000","last_fullsized":"2024-03-04T08:57:23.813823+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:05:15.128703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'30","reported_seq":223,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:58:05.879369+0000","last_change":"2024-03-04T08:57:23.825562+0000","last_active":"2024-03-04T08:58:05.879369+0000","last_peered":"2024-03-04T08:58:05.879369+0000","last_clean":"2024-03-04T08:58:05.879369+0000","last_became_active":"2024-03-04T08:57:23.814977+0000","last_became_peered":"2024-03-04T08:57:23.814977+0000","last_unstale":"2024-03-04T08:58:05.879369+0000","last_undegraded":"2024-03-04T08:58:05.879369+0000","last_fullsized":"2024-03-04T08:58:05.879369+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:29:26.493310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63356,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":141,"num_read_kb":132,"num_write":30,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":63356,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816374+0000","last_change":"2024-03-04T08:56:30.647764+0000","last_active":"2024-03-04T08:57:23.816374+0000","last_peered":"2024-03-04T08:57:23.816374+0000","last_clean":"2024-03-04T08:57:23.816374+0000","last_became_active":"2024-03-04T08:56:30.647445+0000","last_became_peered":"2024-03-04T08:56:30.647445+0000","last_unstale":"2024-03-04T08:57:23.816374+0000","last_undegraded":"2024-03-04T08:57:23.816374+0000","last_fullsized":"2024-03-04T08:57:23.816374+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:13:25.135993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.820947+0000","last_change":"2024-03-04T08:56:34.674674+0000","last_active":"2024-03-04T08:57:23.820947+0000","last_peered":"2024-03-04T08:57:23.820947+0000","last_clean":"2024-03-04T08:57:23.820947+0000","last_became_active":"2024-03-04T08:56:34.674138+0000","last_became_peered":"2024-03-04T08:56:34.674138+0000","last_unstale":"2024-03-04T08:57:23.820947+0000","last_undegraded":"2024-03-04T08:57:23.820947+0000","last_fullsized":"2024-03-04T08:57:23.820947+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:20:36.123111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.811491+0000","last_change":"2024-03-04T08:56:29.980997+0000","last_active":"2024-03-04T08:57:23.811491+0000","last_peered":"2024-03-04T08:57:23.811491+0000","last_clean":"2024-03-04T08:57:23.811491+0000","last_became_active":"2024-03-04T08:56:29.980651+0000","last_became_peered":"2024-03-04T08:56:29.980651+0000","last_unstale":"2024-03-04T08:57:23.811491+0000","last_undegraded":"2024-03-04T08:57:23.811491+0000","last_fullsized":"2024-03-04T08:57:23.811491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:23:33.434703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.812705+0000","last_change":"2024-03-04T08:56:30.657229+0000","last_active":"2024-03-04T08:57:23.812705+0000","last_peered":"2024-03-04T08:57:23.812705+0000","last_clean":"2024-03-04T08:57:23.812705+0000","last_became_active":"2024-03-04T08:56:30.656877+0000","last_became_peered":"2024-03-04T08:56:30.656877+0000","last_unstale":"2024-03-04T08:57:23.812705+0000","last_undegraded":"2024-03-04T08:57:23.812705+0000","last_fullsized":"2024-03-04T08:57:23.812705+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:17:55.063418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.818314+0000","last_change":"2024-03-04T08:57:23.818314+0000","last_active":"2024-03-04T08:57:23.818314+0000","last_peered":"2024-03-04T08:57:23.818314+0000","last_clean":"2024-03-04T08:57:23.818314+0000","last_became_active":"2024-03-04T08:57:23.817831+0000","last_became_peered":"2024-03-04T08:57:23.817831+0000","last_unstale":"2024-03-04T08:57:23.818314+0000","last_undegraded":"2024-03-04T08:57:23.818314+0000","last_fullsized":"2024-03-04T08:57:23.818314+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:06:54.889154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.819691+0000","last_change":"2024-03-04T08:56:29.987759+0000","last_active":"2024-03-04T08:57:23.819691+0000","last_peered":"2024-03-04T08:57:23.819691+0000","last_clean":"2024-03-04T08:57:23.819691+0000","last_became_active":"2024-03-04T08:56:29.987458+0000","last_became_peered":"2024-03-04T08:56:29.987458+0000","last_unstale":"2024-03-04T08:57:23.819691+0000","last_undegraded":"2024-03-04T08:57:23.819691+0000","last_fullsized":"2024-03-04T08:57:23.819691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:36:18.878556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.818888+0000","last_change":"2024-03-04T08:57:23.818888+0000","last_active":"2024-03-04T08:57:23.818888+0000","last_peered":"2024-03-04T08:57:23.818888+0000","last_clean":"2024-03-04T08:57:23.818888+0000","last_became_active":"2024-03-04T08:57:23.818510+0000","last_became_peered":"2024-03-04T08:57:23.818510+0000","last_unstale":"2024-03-04T08:57:23.818888+0000","last_undegraded":"2024-03-04T08:57:23.818888+0000","last_fullsized":"2024-03-04T08:57:23.818888+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:50:21.276520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.814191+0000","last_change":"2024-03-04T08:56:34.683727+0000","last_active":"2024-03-04T08:57:23.814191+0000","last_peered":"2024-03-04T08:57:23.814191+0000","last_clean":"2024-03-04T08:57:23.814191+0000","last_became_active":"2024-03-04T08:56:34.681865+0000","last_became_peered":"2024-03-04T08:56:34.681865+0000","last_unstale":"2024-03-04T08:57:23.814191+0000","last_undegraded":"2024-03-04T08:57:23.814191+0000","last_fullsized":"2024-03-04T08:57:23.814191+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:44:46.586791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813619+0000","last_change":"2024-03-04T08:56:29.983385+0000","last_active":"2024-03-04T08:57:23.813619+0000","last_peered":"2024-03-04T08:57:23.813619+0000","last_clean":"2024-03-04T08:57:23.813619+0000","last_became_active":"2024-03-04T08:56:29.983079+0000","last_became_peered":"2024-03-04T08:56:29.983079+0000","last_unstale":"2024-03-04T08:57:23.813619+0000","last_undegraded":"2024-03-04T08:57:23.813619+0000","last_fullsized":"2024-03-04T08:57:23.813619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:23:15.417389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.815799+0000","last_change":"2024-03-04T08:56:30.641187+0000","last_active":"2024-03-04T08:57:23.815799+0000","last_peered":"2024-03-04T08:57:23.815799+0000","last_clean":"2024-03-04T08:57:23.815799+0000","last_became_active":"2024-03-04T08:56:30.640863+0000","last_became_peered":"2024-03-04T08:56:30.640863+0000","last_unstale":"2024-03-04T08:57:23.815799+0000","last_undegraded":"2024-03-04T08:57:23.815799+0000","last_fullsized":"2024-03-04T08:57:23.815799+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:08:30.257806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.162018+0000","last_change":"2024-03-04T08:57:24.162018+0000","last_active":"2024-03-04T08:57:24.162018+0000","last_peered":"2024-03-04T08:57:24.162018+0000","last_clean":"2024-03-04T08:57:24.162018+0000","last_became_active":"2024-03-04T08:57:24.161551+0000","last_became_peered":"2024-03-04T08:57:24.161551+0000","last_unstale":"2024-03-04T08:57:24.162018+0000","last_undegraded":"2024-03-04T08:57:24.162018+0000","last_fullsized":"2024-03-04T08:57:24.162018+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:21:35.698806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'8","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.815252+0000","last_change":"2024-03-04T08:56:29.994016+0000","last_active":"2024-03-04T08:57:23.815252+0000","last_peered":"2024-03-04T08:57:23.815252+0000","last_clean":"2024-03-04T08:57:23.815252+0000","last_became_active":"2024-03-04T08:56:29.993683+0000","last_became_peered":"2024-03-04T08:56:29.993683+0000","last_unstale":"2024-03-04T08:57:23.815252+0000","last_undegraded":"2024-03-04T08:57:23.815252+0000","last_fullsized":"2024-03-04T08:57:23.815252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:25:37.931248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.819930+0000","last_change":"2024-03-04T08:56:30.647390+0000","last_active":"2024-03-04T08:57:23.819930+0000","last_peered":"2024-03-04T08:57:23.819930+0000","last_clean":"2024-03-04T08:57:23.819930+0000","last_became_active":"2024-03-04T08:56:30.643879+0000","last_became_peered":"2024-03-04T08:56:30.643879+0000","last_unstale":"2024-03-04T08:57:23.819930+0000","last_undegraded":"2024-03-04T08:57:23.819930+0000","last_fullsized":"2024-03-04T08:57:23.819930+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:41:13.030295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.162126+0000","last_change":"2024-03-04T08:57:24.162126+0000","last_active":"2024-03-04T08:57:24.162126+0000","last_peered":"2024-03-04T08:57:24.162126+0000","last_clean":"2024-03-04T08:57:24.162126+0000","last_became_active":"2024-03-04T08:57:24.161597+0000","last_became_peered":"2024-03-04T08:57:24.161597+0000","last_unstale":"2024-03-04T08:57:24.162126+0000","last_undegraded":"2024-03-04T08:57:24.162126+0000","last_fullsized":"2024-03-04T08:57:24.162126+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:56:01.936796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"86'1","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.815985+0000","last_change":"2024-03-04T08:56:29.983195+0000","last_active":"2024-03-04T08:57:23.815985+0000","last_peered":"2024-03-04T08:57:23.815985+0000","last_clean":"2024-03-04T08:57:23.815985+0000","last_became_active":"2024-03-04T08:56:29.982881+0000","last_became_peered":"2024-03-04T08:56:29.982881+0000","last_unstale":"2024-03-04T08:57:23.815985+0000","last_undegraded":"2024-03-04T08:57:23.815985+0000","last_fullsized":"2024-03-04T08:57:23.815985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:59:12.802430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813582+0000","last_change":"2024-03-04T08:56:30.650452+0000","last_active":"2024-03-04T08:57:23.813582+0000","last_peered":"2024-03-04T08:57:23.813582+0000","last_clean":"2024-03-04T08:57:23.813582+0000","last_became_active":"2024-03-04T08:56:30.650148+0000","last_became_peered":"2024-03-04T08:56:30.650148+0000","last_unstale":"2024-03-04T08:57:23.813582+0000","last_undegraded":"2024-03-04T08:57:23.813582+0000","last_fullsized":"2024-03-04T08:57:23.813582+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:22:55.741505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.815542+0000","last_change":"2024-03-04T08:56:34.860608+0000","last_active":"2024-03-04T08:57:23.815542+0000","last_peered":"2024-03-04T08:57:23.815542+0000","last_clean":"2024-03-04T08:57:23.815542+0000","last_became_active":"2024-03-04T08:56:34.860087+0000","last_became_peered":"2024-03-04T08:56:34.860087+0000","last_unstale":"2024-03-04T08:57:23.815542+0000","last_undegraded":"2024-03-04T08:57:23.815542+0000","last_fullsized":"2024-03-04T08:57:23.815542+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:33:35.675886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.814013+0000","last_change":"2024-03-04T08:56:29.980947+0000","last_active":"2024-03-04T08:57:23.814013+0000","last_peered":"2024-03-04T08:57:23.814013+0000","last_clean":"2024-03-04T08:57:23.814013+0000","last_became_active":"2024-03-04T08:56:29.980625+0000","last_became_peered":"2024-03-04T08:56:29.980625+0000","last_unstale":"2024-03-04T08:57:23.814013+0000","last_undegraded":"2024-03-04T08:57:23.814013+0000","last_fullsized":"2024-03-04T08:57:23.814013+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:53:53.944112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.810713+0000","last_change":"2024-03-04T08:56:30.642047+0000","last_active":"2024-03-04T08:57:23.810713+0000","last_peered":"2024-03-04T08:57:23.810713+0000","last_clean":"2024-03-04T08:57:23.810713+0000","last_became_active":"2024-03-04T08:56:30.641700+0000","last_became_peered":"2024-03-04T08:56:30.641700+0000","last_unstale":"2024-03-04T08:57:23.810713+0000","last_undegraded":"2024-03-04T08:57:23.810713+0000","last_fullsized":"2024-03-04T08:57:23.810713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:43:53.641512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813031+0000","last_change":"2024-03-04T08:56:34.674944+0000","last_active":"2024-03-04T08:57:23.813031+0000","last_peered":"2024-03-04T08:57:23.813031+0000","last_clean":"2024-03-04T08:57:23.813031+0000","last_became_active":"2024-03-04T08:56:34.674000+0000","last_became_peered":"2024-03-04T08:56:34.674000+0000","last_unstale":"2024-03-04T08:57:23.813031+0000","last_undegraded":"2024-03-04T08:57:23.813031+0000","last_fullsized":"2024-03-04T08:57:23.813031+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:26:19.386329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816135+0000","last_change":"2024-03-04T08:56:29.989020+0000","last_active":"2024-03-04T08:57:23.816135+0000","last_peered":"2024-03-04T08:57:23.816135+0000","last_clean":"2024-03-04T08:57:23.816135+0000","last_became_active":"2024-03-04T08:56:29.988751+0000","last_became_peered":"2024-03-04T08:56:29.988751+0000","last_unstale":"2024-03-04T08:57:23.816135+0000","last_undegraded":"2024-03-04T08:57:23.816135+0000","last_fullsized":"2024-03-04T08:57:23.816135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:38:45.003221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.810596+0000","last_change":"2024-03-04T08:56:30.649976+0000","last_active":"2024-03-04T08:57:23.810596+0000","last_peered":"2024-03-04T08:57:23.810596+0000","last_clean":"2024-03-04T08:57:23.810596+0000","last_became_active":"2024-03-04T08:56:30.649680+0000","last_became_peered":"2024-03-04T08:56:30.649680+0000","last_unstale":"2024-03-04T08:57:23.810596+0000","last_undegraded":"2024-03-04T08:57:23.810596+0000","last_fullsized":"2024-03-04T08:57:23.810596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:50:02.197087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813389+0000","last_change":"2024-03-04T08:56:34.860654+0000","last_active":"2024-03-04T08:57:23.813389+0000","last_peered":"2024-03-04T08:57:23.813389+0000","last_clean":"2024-03-04T08:57:23.813389+0000","last_became_active":"2024-03-04T08:56:34.860170+0000","last_became_peered":"2024-03-04T08:56:34.860170+0000","last_unstale":"2024-03-04T08:57:23.813389+0000","last_undegraded":"2024-03-04T08:57:23.813389+0000","last_fullsized":"2024-03-04T08:57:23.813389+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:08:49.703317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.819410+0000","last_change":"2024-03-04T08:56:30.643510+0000","last_active":"2024-03-04T08:57:23.819410+0000","last_peered":"2024-03-04T08:57:23.819410+0000","last_clean":"2024-03-04T08:57:23.819410+0000","last_became_active":"2024-03-04T08:56:30.643141+0000","last_became_peered":"2024-03-04T08:56:30.643141+0000","last_unstale":"2024-03-04T08:57:23.819410+0000","last_undegraded":"2024-03-04T08:57:23.819410+0000","last_fullsized":"2024-03-04T08:57:23.819410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:36:03.619735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.812466+0000","last_change":"2024-03-04T08:56:29.994030+0000","last_active":"2024-03-04T08:57:23.812466+0000","last_peered":"2024-03-04T08:57:23.812466+0000","last_clean":"2024-03-04T08:57:23.812466+0000","last_became_active":"2024-03-04T08:56:29.993698+0000","last_became_peered":"2024-03-04T08:56:29.993698+0000","last_unstale":"2024-03-04T08:57:23.812466+0000","last_undegraded":"2024-03-04T08:57:23.812466+0000","last_fullsized":"2024-03-04T08:57:23.812466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:28:38.609330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816388+0000","last_change":"2024-03-04T08:56:34.678337+0000","last_active":"2024-03-04T08:57:23.816388+0000","last_peered":"2024-03-04T08:57:23.816388+0000","last_clean":"2024-03-04T08:57:23.816388+0000","last_became_active":"2024-03-04T08:56:34.678091+0000","last_became_peered":"2024-03-04T08:56:34.678091+0000","last_unstale":"2024-03-04T08:57:23.816388+0000","last_undegraded":"2024-03-04T08:57:23.816388+0000","last_fullsized":"2024-03-04T08:57:23.816388+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:05:54.075257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.160651+0000","last_change":"2024-03-04T08:56:30.641609+0000","last_active":"2024-03-04T08:57:24.160651+0000","last_peered":"2024-03-04T08:57:24.160651+0000","last_clean":"2024-03-04T08:57:24.160651+0000","last_became_active":"2024-03-04T08:56:30.641323+0000","last_became_peered":"2024-03-04T08:56:30.641323+0000","last_unstale":"2024-03-04T08:57:24.160651+0000","last_undegraded":"2024-03-04T08:57:24.160651+0000","last_fullsized":"2024-03-04T08:57:24.160651+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:39:03.935424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813966+0000","last_change":"2024-03-04T08:56:29.976581+0000","last_active":"2024-03-04T08:57:23.813966+0000","last_peered":"2024-03-04T08:57:23.813966+0000","last_clean":"2024-03-04T08:57:23.813966+0000","last_became_active":"2024-03-04T08:56:29.976215+0000","last_became_peered":"2024-03-04T08:56:29.976215+0000","last_unstale":"2024-03-04T08:57:23.813966+0000","last_undegraded":"2024-03-04T08:57:23.813966+0000","last_fullsized":"2024-03-04T08:57:23.813966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:53:29.040404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":361,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.815720+0000","last_change":"2024-03-04T08:55:10.344909+0000","last_active":"2024-03-04T08:57:23.815720+0000","last_peered":"2024-03-04T08:57:23.815720+0000","last_clean":"2024-03-04T08:57:23.815720+0000","last_became_active":"2024-03-04T08:55:10.344424+0000","last_became_peered":"2024-03-04T08:55:10.344424+0000","last_unstale":"2024-03-04T08:57:23.815720+0000","last_undegraded":"2024-03-04T08:57:23.815720+0000","last_fullsized":"2024-03-04T08:57:23.815720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:51:15.334055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:51:15.334055+0000","last_clean_scrub_stamp":"2024-03-04T08:51:15.334055+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:36:29.766101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.820502+0000","last_change":"2024-03-04T08:56:34.861141+0000","last_active":"2024-03-04T08:57:23.820502+0000","last_peered":"2024-03-04T08:57:23.820502+0000","last_clean":"2024-03-04T08:57:23.820502+0000","last_became_active":"2024-03-04T08:56:34.860618+0000","last_became_peered":"2024-03-04T08:56:34.860618+0000","last_unstale":"2024-03-04T08:57:23.820502+0000","last_undegraded":"2024-03-04T08:57:23.820502+0000","last_fullsized":"2024-03-04T08:57:23.820502+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:53:20.918385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.812408+0000","last_change":"2024-03-04T08:56:30.644248+0000","last_active":"2024-03-04T08:57:23.812408+0000","last_peered":"2024-03-04T08:57:23.812408+0000","last_clean":"2024-03-04T08:57:23.812408+0000","last_became_active":"2024-03-04T08:56:30.643956+0000","last_became_peered":"2024-03-04T08:56:30.643956+0000","last_unstale":"2024-03-04T08:57:23.812408+0000","last_undegraded":"2024-03-04T08:57:23.812408+0000","last_fullsized":"2024-03-04T08:57:23.812408+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:19:24.298145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816701+0000","last_change":"2024-03-04T08:56:29.987584+0000","last_active":"2024-03-04T08:57:23.816701+0000","last_peered":"2024-03-04T08:57:23.816701+0000","last_clean":"2024-03-04T08:57:23.816701+0000","last_became_active":"2024-03-04T08:56:29.986741+0000","last_became_peered":"2024-03-04T08:56:29.986741+0000","last_unstale":"2024-03-04T08:57:23.816701+0000","last_undegraded":"2024-03-04T08:57:23.816701+0000","last_fullsized":"2024-03-04T08:57:23.816701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:46:19.672027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813250+0000","last_change":"2024-03-04T08:56:34.675485+0000","last_active":"2024-03-04T08:57:23.813250+0000","last_peered":"2024-03-04T08:57:23.813250+0000","last_clean":"2024-03-04T08:57:23.813250+0000","last_became_active":"2024-03-04T08:56:34.674961+0000","last_became_peered":"2024-03-04T08:56:34.674961+0000","last_unstale":"2024-03-04T08:57:23.813250+0000","last_undegraded":"2024-03-04T08:57:23.813250+0000","last_fullsized":"2024-03-04T08:57:23.813250+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:39:43.387849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816504+0000","last_change":"2024-03-04T08:56:29.992287+0000","last_active":"2024-03-04T08:57:23.816504+0000","last_peered":"2024-03-04T08:57:23.816504+0000","last_clean":"2024-03-04T08:57:23.816504+0000","last_became_active":"2024-03-04T08:56:29.991998+0000","last_became_peered":"2024-03-04T08:56:29.991998+0000","last_unstale":"2024-03-04T08:57:23.816504+0000","last_undegraded":"2024-03-04T08:57:23.816504+0000","last_fullsized":"2024-03-04T08:57:23.816504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:33:10.557684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813150+0000","last_change":"2024-03-04T08:56:30.644190+0000","last_active":"2024-03-04T08:57:23.813150+0000","last_peered":"2024-03-04T08:57:23.813150+0000","last_clean":"2024-03-04T08:57:23.813150+0000","last_became_active":"2024-03-04T08:56:30.643783+0000","last_became_peered":"2024-03-04T08:56:30.643783+0000","last_unstale":"2024-03-04T08:57:23.813150+0000","last_undegraded":"2024-03-04T08:57:23.813150+0000","last_fullsized":"2024-03-04T08:57:23.813150+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:25:12.886305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.160397+0000","last_change":"2024-03-04T08:56:34.860415+0000","last_active":"2024-03-04T08:57:24.160397+0000","last_peered":"2024-03-04T08:57:24.160397+0000","last_clean":"2024-03-04T08:57:24.160397+0000","last_became_active":"2024-03-04T08:56:34.860008+0000","last_became_peered":"2024-03-04T08:56:34.860008+0000","last_unstale":"2024-03-04T08:57:24.160397+0000","last_undegraded":"2024-03-04T08:57:24.160397+0000","last_fullsized":"2024-03-04T08:57:24.160397+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:55:45.673567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.814550+0000","last_change":"2024-03-04T08:56:29.989599+0000","last_active":"2024-03-04T08:57:23.814550+0000","last_peered":"2024-03-04T08:57:23.814550+0000","last_clean":"2024-03-04T08:57:23.814550+0000","last_became_active":"2024-03-04T08:56:29.989281+0000","last_became_peered":"2024-03-04T08:56:29.989281+0000","last_unstale":"2024-03-04T08:57:23.814550+0000","last_undegraded":"2024-03-04T08:57:23.814550+0000","last_fullsized":"2024-03-04T08:57:23.814550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:59:04.006117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.160732+0000","last_change":"2024-03-04T08:56:30.640315+0000","last_active":"2024-03-04T08:57:24.160732+0000","last_peered":"2024-03-04T08:57:24.160732+0000","last_clean":"2024-03-04T08:57:24.160732+0000","last_became_active":"2024-03-04T08:56:30.640001+0000","last_became_peered":"2024-03-04T08:56:30.640001+0000","last_unstale":"2024-03-04T08:57:24.160732+0000","last_undegraded":"2024-03-04T08:57:24.160732+0000","last_fullsized":"2024-03-04T08:57:24.160732+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:43:57.049948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.811032+0000","last_change":"2024-03-04T08:56:34.667973+0000","last_active":"2024-03-04T08:57:23.811032+0000","last_peered":"2024-03-04T08:57:23.811032+0000","last_clean":"2024-03-04T08:57:23.811032+0000","last_became_active":"2024-03-04T08:56:34.667527+0000","last_became_peered":"2024-03-04T08:56:34.667527+0000","last_unstale":"2024-03-04T08:57:23.811032+0000","last_undegraded":"2024-03-04T08:57:23.811032+0000","last_fullsized":"2024-03-04T08:57:23.811032+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:07:23.991933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813883+0000","last_change":"2024-03-04T08:56:29.982263+0000","last_active":"2024-03-04T08:57:23.813883+0000","last_peered":"2024-03-04T08:57:23.813883+0000","last_clean":"2024-03-04T08:57:23.813883+0000","last_became_active":"2024-03-04T08:56:29.981802+0000","last_became_peered":"2024-03-04T08:56:29.981802+0000","last_unstale":"2024-03-04T08:57:23.813883+0000","last_undegraded":"2024-03-04T08:57:23.813883+0000","last_fullsized":"2024-03-04T08:57:23.813883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:24:33.351428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816995+0000","last_change":"2024-03-04T08:56:30.657847+0000","last_active":"2024-03-04T08:57:23.816995+0000","last_peered":"2024-03-04T08:57:23.816995+0000","last_clean":"2024-03-04T08:57:23.816995+0000","last_became_active":"2024-03-04T08:56:30.657533+0000","last_became_peered":"2024-03-04T08:56:30.657533+0000","last_unstale":"2024-03-04T08:57:23.816995+0000","last_undegraded":"2024-03-04T08:57:23.816995+0000","last_fullsized":"2024-03-04T08:57:23.816995+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:11:01.914455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.814164+0000","last_change":"2024-03-04T08:56:34.861093+0000","last_active":"2024-03-04T08:57:23.814164+0000","last_peered":"2024-03-04T08:57:23.814164+0000","last_clean":"2024-03-04T08:57:23.814164+0000","last_became_active":"2024-03-04T08:56:34.860660+0000","last_became_peered":"2024-03-04T08:56:34.860660+0000","last_unstale":"2024-03-04T08:57:23.814164+0000","last_undegraded":"2024-03-04T08:57:23.814164+0000","last_fullsized":"2024-03-04T08:57:23.814164+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:37:38.424914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813376+0000","last_change":"2024-03-04T08:56:29.990546+0000","last_active":"2024-03-04T08:57:23.813376+0000","last_peered":"2024-03-04T08:57:23.813376+0000","last_clean":"2024-03-04T08:57:23.813376+0000","last_became_active":"2024-03-04T08:56:29.990342+0000","last_became_peered":"2024-03-04T08:56:29.990342+0000","last_unstale":"2024-03-04T08:57:23.813376+0000","last_undegraded":"2024-03-04T08:57:23.813376+0000","last_fullsized":"2024-03-04T08:57:23.813376+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:01:06.929112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.815687+0000","last_change":"2024-03-04T08:56:30.653228+0000","last_active":"2024-03-04T08:57:23.815687+0000","last_peered":"2024-03-04T08:57:23.815687+0000","last_clean":"2024-03-04T08:57:23.815687+0000","last_became_active":"2024-03-04T08:56:30.652926+0000","last_became_peered":"2024-03-04T08:56:30.652926+0000","last_unstale":"2024-03-04T08:57:23.815687+0000","last_undegraded":"2024-03-04T08:57:23.815687+0000","last_fullsized":"2024-03-04T08:57:23.815687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:06:27.728135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.814329+0000","last_change":"2024-03-04T08:56:34.681096+0000","last_active":"2024-03-04T08:57:23.814329+0000","last_peered":"2024-03-04T08:57:23.814329+0000","last_clean":"2024-03-04T08:57:23.814329+0000","last_became_active":"2024-03-04T08:56:34.680718+0000","last_became_peered":"2024-03-04T08:56:34.680718+0000","last_unstale":"2024-03-04T08:57:23.814329+0000","last_undegraded":"2024-03-04T08:57:23.814329+0000","last_fullsized":"2024-03-04T08:57:23.814329+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:04:58.908904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.160895+0000","last_change":"2024-03-04T08:56:29.988039+0000","last_active":"2024-03-04T08:57:24.160895+0000","last_peered":"2024-03-04T08:57:24.160895+0000","last_clean":"2024-03-04T08:57:24.160895+0000","last_became_active":"2024-03-04T08:56:29.987310+0000","last_became_peered":"2024-03-04T08:56:29.987310+0000","last_unstale":"2024-03-04T08:57:24.160895+0000","last_undegraded":"2024-03-04T08:57:24.160895+0000","last_fullsized":"2024-03-04T08:57:24.160895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:19:39.319965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813149+0000","last_change":"2024-03-04T08:56:30.649094+0000","last_active":"2024-03-04T08:57:23.813149+0000","last_peered":"2024-03-04T08:57:23.813149+0000","last_clean":"2024-03-04T08:57:23.813149+0000","last_became_active":"2024-03-04T08:56:30.648776+0000","last_became_peered":"2024-03-04T08:56:30.648776+0000","last_unstale":"2024-03-04T08:57:23.813149+0000","last_undegraded":"2024-03-04T08:57:23.813149+0000","last_fullsized":"2024-03-04T08:57:23.813149+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:23:17.738792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813970+0000","last_change":"2024-03-04T08:56:34.680923+0000","last_active":"2024-03-04T08:57:23.813970+0000","last_peered":"2024-03-04T08:57:23.813970+0000","last_clean":"2024-03-04T08:57:23.813970+0000","last_became_active":"2024-03-04T08:56:34.680114+0000","last_became_peered":"2024-03-04T08:56:34.680114+0000","last_unstale":"2024-03-04T08:57:23.813970+0000","last_undegraded":"2024-03-04T08:57:23.813970+0000","last_fullsized":"2024-03-04T08:57:23.813970+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:05:53.434735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.162742+0000","last_change":"2024-03-04T08:57:24.162162+0000","last_active":"2024-03-04T08:57:24.162742+0000","last_peered":"2024-03-04T08:57:24.162742+0000","last_clean":"2024-03-04T08:57:24.162742+0000","last_became_active":"2024-03-04T08:57:24.161676+0000","last_became_peered":"2024-03-04T08:57:24.161676+0000","last_unstale":"2024-03-04T08:57:24.162742+0000","last_undegraded":"2024-03-04T08:57:24.162742+0000","last_fullsized":"2024-03-04T08:57:24.162742+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:02:29.121039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.160522+0000","last_change":"2024-03-04T08:56:30.654617+0000","last_active":"2024-03-04T08:57:24.160522+0000","last_peered":"2024-03-04T08:57:24.160522+0000","last_clean":"2024-03-04T08:57:24.160522+0000","last_became_active":"2024-03-04T08:56:30.653341+0000","last_became_peered":"2024-03-04T08:56:30.653341+0000","last_unstale":"2024-03-04T08:57:24.160522+0000","last_undegraded":"2024-03-04T08:57:24.160522+0000","last_fullsized":"2024-03-04T08:57:24.160522+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:10:26.234514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813608+0000","last_change":"2024-03-04T08:56:34.860680+0000","last_active":"2024-03-04T08:57:23.813608+0000","last_peered":"2024-03-04T08:57:23.813608+0000","last_clean":"2024-03-04T08:57:23.813608+0000","last_became_active":"2024-03-04T08:56:34.860290+0000","last_became_peered":"2024-03-04T08:56:34.860290+0000","last_unstale":"2024-03-04T08:57:23.813608+0000","last_undegraded":"2024-03-04T08:57:23.813608+0000","last_fullsized":"2024-03-04T08:57:23.813608+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:14:32.126839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816054+0000","last_change":"2024-03-04T08:56:29.989843+0000","last_active":"2024-03-04T08:57:23.816054+0000","last_peered":"2024-03-04T08:57:23.816054+0000","last_clean":"2024-03-04T08:57:23.816054+0000","last_became_active":"2024-03-04T08:56:29.989508+0000","last_became_peered":"2024-03-04T08:56:29.989508+0000","last_unstale":"2024-03-04T08:57:23.816054+0000","last_undegraded":"2024-03-04T08:57:23.816054+0000","last_fullsized":"2024-03-04T08:57:23.816054+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:46:01.738580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816231+0000","last_change":"2024-03-04T08:56:30.644598+0000","last_active":"2024-03-04T08:57:23.816231+0000","last_peered":"2024-03-04T08:57:23.816231+0000","last_clean":"2024-03-04T08:57:23.816231+0000","last_became_active":"2024-03-04T08:56:30.644263+0000","last_became_peered":"2024-03-04T08:56:30.644263+0000","last_unstale":"2024-03-04T08:57:23.816231+0000","last_undegraded":"2024-03-04T08:57:23.816231+0000","last_fullsized":"2024-03-04T08:57:23.816231+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:47:07.013271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813407+0000","last_change":"2024-03-04T08:56:34.675116+0000","last_active":"2024-03-04T08:57:23.813407+0000","last_peered":"2024-03-04T08:57:23.813407+0000","last_clean":"2024-03-04T08:57:23.813407+0000","last_became_active":"2024-03-04T08:56:34.674727+0000","last_became_peered":"2024-03-04T08:56:34.674727+0000","last_unstale":"2024-03-04T08:57:23.813407+0000","last_undegraded":"2024-03-04T08:57:23.813407+0000","last_fullsized":"2024-03-04T08:57:23.813407+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:56:23.461299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.811967+0000","last_change":"2024-03-04T08:56:29.983545+0000","last_active":"2024-03-04T08:57:23.811967+0000","last_peered":"2024-03-04T08:57:23.811967+0000","last_clean":"2024-03-04T08:57:23.811967+0000","last_became_active":"2024-03-04T08:56:29.983244+0000","last_became_peered":"2024-03-04T08:56:29.983244+0000","last_unstale":"2024-03-04T08:57:23.811967+0000","last_undegraded":"2024-03-04T08:57:23.811967+0000","last_fullsized":"2024-03-04T08:57:23.811967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:59:43.475203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816720+0000","last_change":"2024-03-04T08:56:30.641244+0000","last_active":"2024-03-04T08:57:23.816720+0000","last_peered":"2024-03-04T08:57:23.816720+0000","last_clean":"2024-03-04T08:57:23.816720+0000","last_became_active":"2024-03-04T08:56:30.640952+0000","last_became_peered":"2024-03-04T08:56:30.640952+0000","last_unstale":"2024-03-04T08:57:23.816720+0000","last_undegraded":"2024-03-04T08:57:23.816720+0000","last_fullsized":"2024-03-04T08:57:23.816720+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:38:51.134697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.819456+0000","last_change":"2024-03-04T08:57:23.819456+0000","last_active":"2024-03-04T08:57:23.819456+0000","last_peered":"2024-03-04T08:57:23.819456+0000","last_clean":"2024-03-04T08:57:23.819456+0000","last_became_active":"2024-03-04T08:57:23.818994+0000","last_became_peered":"2024-03-04T08:57:23.818994+0000","last_unstale":"2024-03-04T08:57:23.819456+0000","last_undegraded":"2024-03-04T08:57:23.819456+0000","last_fullsized":"2024-03-04T08:57:23.819456+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:27:14.890246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,10,1,4],"acting":[0,10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.820654+0000","last_change":"2024-03-04T08:56:29.980781+0000","last_active":"2024-03-04T08:57:23.820654+0000","last_peered":"2024-03-04T08:57:23.820654+0000","last_clean":"2024-03-04T08:57:23.820654+0000","last_became_active":"2024-03-04T08:56:29.980446+0000","last_became_peered":"2024-03-04T08:56:29.980446+0000","last_unstale":"2024-03-04T08:57:23.820654+0000","last_undegraded":"2024-03-04T08:57:23.820654+0000","last_fullsized":"2024-03-04T08:57:23.820654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:05:37.325902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816274+0000","last_change":"2024-03-04T08:56:30.648350+0000","last_active":"2024-03-04T08:57:23.816274+0000","last_peered":"2024-03-04T08:57:23.816274+0000","last_clean":"2024-03-04T08:57:23.816274+0000","last_became_active":"2024-03-04T08:56:30.648059+0000","last_became_peered":"2024-03-04T08:56:30.648059+0000","last_unstale":"2024-03-04T08:57:23.816274+0000","last_undegraded":"2024-03-04T08:57:23.816274+0000","last_fullsized":"2024-03-04T08:57:23.816274+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:59:07.571168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.819082+0000","last_change":"2024-03-04T08:56:34.673401+0000","last_active":"2024-03-04T08:57:23.819082+0000","last_peered":"2024-03-04T08:57:23.819082+0000","last_clean":"2024-03-04T08:57:23.819082+0000","last_became_active":"2024-03-04T08:56:34.672596+0000","last_became_peered":"2024-03-04T08:56:34.672596+0000","last_unstale":"2024-03-04T08:57:23.819082+0000","last_undegraded":"2024-03-04T08:57:23.819082+0000","last_fullsized":"2024-03-04T08:57:23.819082+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:39:56.613366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813351+0000","last_change":"2024-03-04T08:56:29.988331+0000","last_active":"2024-03-04T08:57:23.813351+0000","last_peered":"2024-03-04T08:57:23.813351+0000","last_clean":"2024-03-04T08:57:23.813351+0000","last_became_active":"2024-03-04T08:56:29.988009+0000","last_became_peered":"2024-03-04T08:56:29.988009+0000","last_unstale":"2024-03-04T08:57:23.813351+0000","last_undegraded":"2024-03-04T08:57:23.813351+0000","last_fullsized":"2024-03-04T08:57:23.813351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:50:43.693924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816655+0000","last_change":"2024-03-04T08:56:30.647443+0000","last_active":"2024-03-04T08:57:23.816655+0000","last_peered":"2024-03-04T08:57:23.816655+0000","last_clean":"2024-03-04T08:57:23.816655+0000","last_became_active":"2024-03-04T08:56:30.647106+0000","last_became_peered":"2024-03-04T08:56:30.647106+0000","last_unstale":"2024-03-04T08:57:23.816655+0000","last_undegraded":"2024-03-04T08:57:23.816655+0000","last_fullsized":"2024-03-04T08:57:23.816655+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:09:56.320806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813326+0000","last_change":"2024-03-04T08:56:30.651243+0000","last_active":"2024-03-04T08:57:23.813326+0000","last_peered":"2024-03-04T08:57:23.813326+0000","last_clean":"2024-03-04T08:57:23.813326+0000","last_became_active":"2024-03-04T08:56:30.650781+0000","last_became_peered":"2024-03-04T08:56:30.650781+0000","last_unstale":"2024-03-04T08:57:23.813326+0000","last_undegraded":"2024-03-04T08:57:23.813326+0000","last_fullsized":"2024-03-04T08:57:23.813326+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:58:16.723542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.811680+0000","last_change":"2024-03-04T08:56:30.645737+0000","last_active":"2024-03-04T08:57:23.811680+0000","last_peered":"2024-03-04T08:57:23.811680+0000","last_clean":"2024-03-04T08:57:23.811680+0000","last_became_active":"2024-03-04T08:56:30.645293+0000","last_became_peered":"2024-03-04T08:56:30.645293+0000","last_unstale":"2024-03-04T08:57:23.811680+0000","last_undegraded":"2024-03-04T08:57:23.811680+0000","last_fullsized":"2024-03-04T08:57:23.811680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:56:04.302166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.811096+0000","last_change":"2024-03-04T08:56:30.649851+0000","last_active":"2024-03-04T08:57:23.811096+0000","last_peered":"2024-03-04T08:57:23.811096+0000","last_clean":"2024-03-04T08:57:23.811096+0000","last_became_active":"2024-03-04T08:56:30.649528+0000","last_became_peered":"2024-03-04T08:56:30.649528+0000","last_unstale":"2024-03-04T08:57:23.811096+0000","last_undegraded":"2024-03-04T08:57:23.811096+0000","last_fullsized":"2024-03-04T08:57:23.811096+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:36:50.431866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.817217+0000","last_change":"2024-03-04T08:56:30.646893+0000","last_active":"2024-03-04T08:57:23.817217+0000","last_peered":"2024-03-04T08:57:23.817217+0000","last_clean":"2024-03-04T08:57:23.817217+0000","last_became_active":"2024-03-04T08:56:30.646615+0000","last_became_peered":"2024-03-04T08:56:30.646615+0000","last_unstale":"2024-03-04T08:57:23.817217+0000","last_undegraded":"2024-03-04T08:57:23.817217+0000","last_fullsized":"2024-03-04T08:57:23.817217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:07:12.584686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813183+0000","last_change":"2024-03-04T08:56:30.649772+0000","last_active":"2024-03-04T08:57:23.813183+0000","last_peered":"2024-03-04T08:57:23.813183+0000","last_clean":"2024-03-04T08:57:23.813183+0000","last_became_active":"2024-03-04T08:56:30.649480+0000","last_became_peered":"2024-03-04T08:56:30.649480+0000","last_unstale":"2024-03-04T08:57:23.813183+0000","last_undegraded":"2024-03-04T08:57:23.813183+0000","last_fullsized":"2024-03-04T08:57:23.813183+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:14:10.553290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.819863+0000","last_change":"2024-03-04T08:57:23.819863+0000","last_active":"2024-03-04T08:57:23.819863+0000","last_peered":"2024-03-04T08:57:23.819863+0000","last_clean":"2024-03-04T08:57:23.819863+0000","last_became_active":"2024-03-04T08:57:23.818397+0000","last_became_peered":"2024-03-04T08:57:23.818397+0000","last_unstale":"2024-03-04T08:57:23.819863+0000","last_undegraded":"2024-03-04T08:57:23.819863+0000","last_fullsized":"2024-03-04T08:57:23.819863+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:47:26.005195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.811688+0000","last_change":"2024-03-04T08:56:30.646870+0000","last_active":"2024-03-04T08:57:23.811688+0000","last_peered":"2024-03-04T08:57:23.811688+0000","last_clean":"2024-03-04T08:57:23.811688+0000","last_became_active":"2024-03-04T08:56:30.646369+0000","last_became_peered":"2024-03-04T08:56:30.646369+0000","last_unstale":"2024-03-04T08:57:23.811688+0000","last_undegraded":"2024-03-04T08:57:23.811688+0000","last_fullsized":"2024-03-04T08:57:23.811688+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:25:34.969089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.160860+0000","last_change":"2024-03-04T08:56:30.640854+0000","last_active":"2024-03-04T08:57:24.160860+0000","last_peered":"2024-03-04T08:57:24.160860+0000","last_clean":"2024-03-04T08:57:24.160860+0000","last_became_active":"2024-03-04T08:56:30.640484+0000","last_became_peered":"2024-03-04T08:56:30.640484+0000","last_unstale":"2024-03-04T08:57:24.160860+0000","last_undegraded":"2024-03-04T08:57:24.160860+0000","last_fullsized":"2024-03-04T08:57:24.160860+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:16:01.298720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.819465+0000","last_change":"2024-03-04T08:56:30.648593+0000","last_active":"2024-03-04T08:57:23.819465+0000","last_peered":"2024-03-04T08:57:23.819465+0000","last_clean":"2024-03-04T08:57:23.819465+0000","last_became_active":"2024-03-04T08:56:30.648287+0000","last_became_peered":"2024-03-04T08:56:30.648287+0000","last_unstale":"2024-03-04T08:57:23.819465+0000","last_undegraded":"2024-03-04T08:57:23.819465+0000","last_fullsized":"2024-03-04T08:57:23.819465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:04:52.525357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.160999+0000","last_change":"2024-03-04T08:56:30.640284+0000","last_active":"2024-03-04T08:57:24.160999+0000","last_peered":"2024-03-04T08:57:24.160999+0000","last_clean":"2024-03-04T08:57:24.160999+0000","last_became_active":"2024-03-04T08:56:30.639968+0000","last_became_peered":"2024-03-04T08:56:30.639968+0000","last_unstale":"2024-03-04T08:57:24.160999+0000","last_undegraded":"2024-03-04T08:57:24.160999+0000","last_fullsized":"2024-03-04T08:57:24.160999+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:37:26.063318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813636+0000","last_change":"2024-03-04T08:56:30.649547+0000","last_active":"2024-03-04T08:57:23.813636+0000","last_peered":"2024-03-04T08:57:23.813636+0000","last_clean":"2024-03-04T08:57:23.813636+0000","last_became_active":"2024-03-04T08:56:30.649170+0000","last_became_peered":"2024-03-04T08:56:30.649170+0000","last_unstale":"2024-03-04T08:57:23.813636+0000","last_undegraded":"2024-03-04T08:57:23.813636+0000","last_fullsized":"2024-03-04T08:57:23.813636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:02:22.805496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813013+0000","last_change":"2024-03-04T08:56:30.650892+0000","last_active":"2024-03-04T08:57:23.813013+0000","last_peered":"2024-03-04T08:57:23.813013+0000","last_clean":"2024-03-04T08:57:23.813013+0000","last_became_active":"2024-03-04T08:56:30.650442+0000","last_became_peered":"2024-03-04T08:56:30.650442+0000","last_unstale":"2024-03-04T08:57:23.813013+0000","last_undegraded":"2024-03-04T08:57:23.813013+0000","last_fullsized":"2024-03-04T08:57:23.813013+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:40:38.217900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813415+0000","last_change":"2024-03-04T08:56:30.647282+0000","last_active":"2024-03-04T08:57:23.813415+0000","last_peered":"2024-03-04T08:57:23.813415+0000","last_clean":"2024-03-04T08:57:23.813415+0000","last_became_active":"2024-03-04T08:56:30.646916+0000","last_became_peered":"2024-03-04T08:56:30.646916+0000","last_unstale":"2024-03-04T08:57:23.813415+0000","last_undegraded":"2024-03-04T08:57:23.813415+0000","last_fullsized":"2024-03-04T08:57:23.813415+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:36:11.714223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64154,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":147,"num_read_kb":138,"num_write":79,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":63356,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":192462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318306,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":509082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579943,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1716224,"data_stored":1158904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743469,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":513760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005110,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":509686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299448,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1642496,"data_stored":1104128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462974,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":504986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1110016,"data_stored":568342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":505076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":505020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":505090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888138,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":572460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724613,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1646592,"data_stored":1099632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":63356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":63378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T08:58:08.501 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph pg dump --format=json 2024-03-04T08:58:08.743 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:58:09.155 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:08 smithi071 ceph-mon[32789]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T08:58:09.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:08 smithi150 ceph-mon[36807]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T08:58:09.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:08 smithi158 ceph-mon[37241]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T08:58:10.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:09 smithi071 ceph-mon[32789]: pgmap v376: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:10.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:09 smithi150 ceph-mon[36807]: pgmap v376: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:10.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:09 smithi158 ceph-mon[37241]: pgmap v376: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:10.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.586+0000 7f5f176ee700 1 -- 172.21.15.71:0/2475427341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f10102cb0 msgr2=0x7f5f10106d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:10.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.586+0000 7f5f176ee700 1 --2- 172.21.15.71:0/2475427341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f10102cb0 0x7f5f10106d00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5f000099c0 tx=0x7f5f00009cd0 comp rx=0 tx=0).stop 2024-03-04T08:58:10.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.586+0000 7f5f176ee700 1 -- 172.21.15.71:0/2475427341 shutdown_connections 2024-03-04T08:58:10.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.586+0000 7f5f176ee700 1 --2- 172.21.15.71:0/2475427341 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5f10102300 0x7f5f101026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:10.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.586+0000 7f5f176ee700 1 --2- 172.21.15.71:0/2475427341 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f101073e0 0x7f5f1010efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:10.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.586+0000 7f5f176ee700 1 --2- 172.21.15.71:0/2475427341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f10102cb0 0x7f5f10106d00 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:10.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.586+0000 7f5f176ee700 1 -- 172.21.15.71:0/2475427341 >> 172.21.15.71:0/2475427341 conn(0x7f5f100fdb70 msgr2=0x7f5f100fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:10.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.586+0000 7f5f176ee700 1 -- 172.21.15.71:0/2475427341 shutdown_connections 2024-03-04T08:58:10.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.586+0000 7f5f176ee700 1 -- 172.21.15.71:0/2475427341 wait complete. 2024-03-04T08:58:10.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.587+0000 7f5f176ee700 1 Processor -- start 2024-03-04T08:58:10.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.587+0000 7f5f176ee700 1 -- start start 2024-03-04T08:58:10.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.587+0000 7f5f176ee700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5f10102300 0x7f5f1019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:10.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.587+0000 7f5f176ee700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f10102cb0 0x7f5f1019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:10.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.587+0000 7f5f176ee700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f101073e0 0x7f5f10198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:10.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.587+0000 7f5f176ee700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f101123c0 con 0x7f5f101073e0 2024-03-04T08:58:10.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.587+0000 7f5f176ee700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f10112240 con 0x7f5f10102cb0 2024-03-04T08:58:10.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.587+0000 7f5f176ee700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f10112540 con 0x7f5f10102300 2024-03-04T08:58:10.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.587+0000 7f5f15c8b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f101073e0 0x7f5f10198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:10.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.587+0000 7f5f15c8b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f101073e0 0x7f5f10198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42990/0 (socket says 172.21.15.71:42990) 2024-03-04T08:58:10.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.587+0000 7f5f15c8b700 1 -- 172.21.15.71:0/2170346115 learned_addr learned my addr 172.21.15.71:0/2170346115 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:10.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.587+0000 7f5f14c89700 1 --2- 172.21.15.71:0/2170346115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f10102cb0 0x7f5f1019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:10.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.587+0000 7f5f1548a700 1 --2- 172.21.15.71:0/2170346115 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5f10102300 0x7f5f1019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:10.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.587+0000 7f5f15c8b700 1 -- 172.21.15.71:0/2170346115 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5f10102300 msgr2=0x7f5f1019e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:10.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.587+0000 7f5f15c8b700 1 --2- 172.21.15.71:0/2170346115 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5f10102300 0x7f5f1019e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:10.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.587+0000 7f5f15c8b700 1 -- 172.21.15.71:0/2170346115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f10102cb0 msgr2=0x7f5f1019eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:10.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.587+0000 7f5f15c8b700 1 --2- 172.21.15.71:0/2170346115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f10102cb0 0x7f5f1019eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:10.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.587+0000 7f5f15c8b700 1 -- 172.21.15.71:0/2170346115 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f10198fe0 con 0x7f5f101073e0 2024-03-04T08:58:10.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.587+0000 7f5f15c8b700 1 --2- 172.21.15.71:0/2170346115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f101073e0 0x7f5f10198700 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00e940 tx=0x7f5f0c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:10.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.588+0000 7f5f067fc700 1 -- 172.21.15.71:0/2170346115 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c00cbd0 con 0x7f5f101073e0 2024-03-04T08:58:10.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.588+0000 7f5f176ee700 1 -- 172.21.15.71:0/2170346115 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f101992d0 con 0x7f5f101073e0 2024-03-04T08:58:10.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.588+0000 7f5f067fc700 1 -- 172.21.15.71:0/2170346115 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f0c00ee90 con 0x7f5f101073e0 2024-03-04T08:58:10.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.588+0000 7f5f067fc700 1 -- 172.21.15.71:0/2170346115 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c0056c0 con 0x7f5f101073e0 2024-03-04T08:58:10.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.588+0000 7f5f176ee700 1 -- 172.21.15.71:0/2170346115 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f101a5560 con 0x7f5f101073e0 2024-03-04T08:58:10.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.589+0000 7f5f176ee700 1 -- 172.21.15.71:0/2170346115 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f101037e0 con 0x7f5f101073e0 2024-03-04T08:58:10.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.590+0000 7f5f067fc700 1 -- 172.21.15.71:0/2170346115 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f5f0c004750 con 0x7f5f101073e0 2024-03-04T08:58:10.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.590+0000 7f5f067fc700 1 --2- 172.21.15.71:0/2170346115 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f5efc0a1ea0 0x7f5efc0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:10.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.590+0000 7f5f1548a700 1 --2- 172.21.15.71:0/2170346115 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f5efc0a1ea0 0x7f5efc0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:10.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.590+0000 7f5f067fc700 1 -- 172.21.15.71:0/2170346115 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f5f0c0ce710 con 0x7f5f101073e0 2024-03-04T08:58:10.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.591+0000 7f5f1548a700 1 --2- 172.21.15.71:0/2170346115 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f5efc0a1ea0 0x7f5efc0a4360 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5f080060b0 tx=0x7f5f08007790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:10.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.594+0000 7f5f067fc700 1 -- 172.21.15.71:0/2170346115 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5f0c053280 con 0x7f5f101073e0 2024-03-04T08:58:10.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.783+0000 7f5f176ee700 1 -- 172.21.15.71:0/2170346115 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5f10199570 con 0x7f5efc0a1ea0 2024-03-04T08:58:10.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.793+0000 7f5f067fc700 1 -- 172.21.15.71:0/2170346115 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340295 (secure 0 0 0) 0x7f5f10199570 con 0x7f5efc0a1ea0 2024-03-04T08:58:10.793 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:58:10.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.796+0000 7f5f176ee700 1 -- 172.21.15.71:0/2170346115 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f5efc0a1ea0 msgr2=0x7f5efc0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:10.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.796+0000 7f5f176ee700 1 --2- 172.21.15.71:0/2170346115 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f5efc0a1ea0 0x7f5efc0a4360 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5f080060b0 tx=0x7f5f08007790 comp rx=0 tx=0).stop 2024-03-04T08:58:10.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.796+0000 7f5f176ee700 1 -- 172.21.15.71:0/2170346115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f101073e0 msgr2=0x7f5f10198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:10.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.796+0000 7f5f176ee700 1 --2- 172.21.15.71:0/2170346115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f101073e0 0x7f5f10198700 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00e940 tx=0x7f5f0c00ec50 comp rx=0 tx=0).stop 2024-03-04T08:58:10.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.796+0000 7f5f176ee700 1 -- 172.21.15.71:0/2170346115 shutdown_connections 2024-03-04T08:58:10.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.796+0000 7f5f176ee700 1 --2- 172.21.15.71:0/2170346115 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f5efc0a1ea0 0x7f5efc0a4360 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:10.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.796+0000 7f5f176ee700 1 --2- 172.21.15.71:0/2170346115 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5f10102300 0x7f5f1019e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:10.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.796+0000 7f5f176ee700 1 --2- 172.21.15.71:0/2170346115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f101073e0 0x7f5f10198700 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:10.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.796+0000 7f5f176ee700 1 --2- 172.21.15.71:0/2170346115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f10102cb0 0x7f5f1019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:10.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.796+0000 7f5f176ee700 1 -- 172.21.15.71:0/2170346115 >> 172.21.15.71:0/2170346115 conn(0x7f5f100fdb70 msgr2=0x7f5f100ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:10.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.796+0000 7f5f176ee700 1 -- 172.21.15.71:0/2170346115 shutdown_connections 2024-03-04T08:58:10.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:10.796+0000 7f5f176ee700 1 -- 172.21.15.71:0/2170346115 wait complete. 2024-03-04T08:58:10.797 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-03-04T08:58:11.575 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2024-03-04T08:58:10.715542+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654756,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":283,"num_read_kb":257,"num_write":262,"num_write_kb":3030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1244092,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359264,"kb_used_data":14192,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582560,"statfs":{"total":1151940427776,"available":1151572541440,"internally_reserved":0,"allocated":14532608,"data_stored":8056166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003309"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813847+0000","last_change":"2024-03-04T08:56:30.650344+0000","last_active":"2024-03-04T08:57:23.813847+0000","last_peered":"2024-03-04T08:57:23.813847+0000","last_clean":"2024-03-04T08:57:23.813847+0000","last_became_active":"2024-03-04T08:56:30.649842+0000","last_became_peered":"2024-03-04T08:56:30.649842+0000","last_unstale":"2024-03-04T08:57:23.813847+0000","last_undegraded":"2024-03-04T08:57:23.813847+0000","last_fullsized":"2024-03-04T08:57:23.813847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:04:57.162321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816882+0000","last_change":"2024-03-04T08:56:30.651616+0000","last_active":"2024-03-04T08:57:23.816882+0000","last_peered":"2024-03-04T08:57:23.816882+0000","last_clean":"2024-03-04T08:57:23.816882+0000","last_became_active":"2024-03-04T08:56:30.651283+0000","last_became_peered":"2024-03-04T08:56:30.651283+0000","last_unstale":"2024-03-04T08:57:23.816882+0000","last_undegraded":"2024-03-04T08:57:23.816882+0000","last_fullsized":"2024-03-04T08:57:23.816882+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:18:48.018257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816784+0000","last_change":"2024-03-04T08:56:30.657459+0000","last_active":"2024-03-04T08:57:23.816784+0000","last_peered":"2024-03-04T08:57:23.816784+0000","last_clean":"2024-03-04T08:57:23.816784+0000","last_became_active":"2024-03-04T08:56:30.656993+0000","last_became_peered":"2024-03-04T08:56:30.656993+0000","last_unstale":"2024-03-04T08:57:23.816784+0000","last_undegraded":"2024-03-04T08:57:23.816784+0000","last_fullsized":"2024-03-04T08:57:23.816784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:07:34.898068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.818684+0000","last_change":"2024-03-04T08:57:23.818097+0000","last_active":"2024-03-04T08:57:23.818684+0000","last_peered":"2024-03-04T08:57:23.818684+0000","last_clean":"2024-03-04T08:57:23.818684+0000","last_became_active":"2024-03-04T08:57:23.817619+0000","last_became_peered":"2024-03-04T08:57:23.817619+0000","last_unstale":"2024-03-04T08:57:23.818684+0000","last_undegraded":"2024-03-04T08:57:23.818684+0000","last_fullsized":"2024-03-04T08:57:23.818684+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:48:15.249967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.815939+0000","last_change":"2024-03-04T08:56:30.654640+0000","last_active":"2024-03-04T08:57:23.815939+0000","last_peered":"2024-03-04T08:57:23.815939+0000","last_clean":"2024-03-04T08:57:23.815939+0000","last_became_active":"2024-03-04T08:56:30.654312+0000","last_became_peered":"2024-03-04T08:56:30.654312+0000","last_unstale":"2024-03-04T08:57:23.815939+0000","last_undegraded":"2024-03-04T08:57:23.815939+0000","last_fullsized":"2024-03-04T08:57:23.815939+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:09:43.930713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.810547+0000","last_change":"2024-03-04T08:56:30.647445+0000","last_active":"2024-03-04T08:57:23.810547+0000","last_peered":"2024-03-04T08:57:23.810547+0000","last_clean":"2024-03-04T08:57:23.810547+0000","last_became_active":"2024-03-04T08:56:30.647130+0000","last_became_peered":"2024-03-04T08:56:30.647130+0000","last_unstale":"2024-03-04T08:57:23.810547+0000","last_undegraded":"2024-03-04T08:57:23.810547+0000","last_fullsized":"2024-03-04T08:57:23.810547+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:40:18.906604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816002+0000","last_change":"2024-03-04T08:56:30.645776+0000","last_active":"2024-03-04T08:57:23.816002+0000","last_peered":"2024-03-04T08:57:23.816002+0000","last_clean":"2024-03-04T08:57:23.816002+0000","last_became_active":"2024-03-04T08:56:30.645107+0000","last_became_peered":"2024-03-04T08:56:30.645107+0000","last_unstale":"2024-03-04T08:57:23.816002+0000","last_undegraded":"2024-03-04T08:57:23.816002+0000","last_fullsized":"2024-03-04T08:57:23.816002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:44:35.230491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.160395+0000","last_change":"2024-03-04T08:56:30.646424+0000","last_active":"2024-03-04T08:57:24.160395+0000","last_peered":"2024-03-04T08:57:24.160395+0000","last_clean":"2024-03-04T08:57:24.160395+0000","last_became_active":"2024-03-04T08:56:30.646115+0000","last_became_peered":"2024-03-04T08:56:30.646115+0000","last_unstale":"2024-03-04T08:57:24.160395+0000","last_undegraded":"2024-03-04T08:57:24.160395+0000","last_fullsized":"2024-03-04T08:57:24.160395+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:42:36.201984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813891+0000","last_change":"2024-03-04T08:56:30.655027+0000","last_active":"2024-03-04T08:57:23.813891+0000","last_peered":"2024-03-04T08:57:23.813891+0000","last_clean":"2024-03-04T08:57:23.813891+0000","last_became_active":"2024-03-04T08:56:30.654766+0000","last_became_peered":"2024-03-04T08:56:30.654766+0000","last_unstale":"2024-03-04T08:57:23.813891+0000","last_undegraded":"2024-03-04T08:57:23.813891+0000","last_fullsized":"2024-03-04T08:57:23.813891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:00:34.168740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.811005+0000","last_change":"2024-03-04T08:56:30.648268+0000","last_active":"2024-03-04T08:57:23.811005+0000","last_peered":"2024-03-04T08:57:23.811005+0000","last_clean":"2024-03-04T08:57:23.811005+0000","last_became_active":"2024-03-04T08:56:30.647905+0000","last_became_peered":"2024-03-04T08:56:30.647905+0000","last_unstale":"2024-03-04T08:57:23.811005+0000","last_undegraded":"2024-03-04T08:57:23.811005+0000","last_fullsized":"2024-03-04T08:57:23.811005+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:07:32.387510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816537+0000","last_change":"2024-03-04T08:56:30.648061+0000","last_active":"2024-03-04T08:57:23.816537+0000","last_peered":"2024-03-04T08:57:23.816537+0000","last_clean":"2024-03-04T08:57:23.816537+0000","last_became_active":"2024-03-04T08:56:30.647740+0000","last_became_peered":"2024-03-04T08:56:30.647740+0000","last_unstale":"2024-03-04T08:57:23.816537+0000","last_undegraded":"2024-03-04T08:57:23.816537+0000","last_fullsized":"2024-03-04T08:57:23.816537+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:35:23.065902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813785+0000","last_change":"2024-03-04T08:56:30.645802+0000","last_active":"2024-03-04T08:57:23.813785+0000","last_peered":"2024-03-04T08:57:23.813785+0000","last_clean":"2024-03-04T08:57:23.813785+0000","last_became_active":"2024-03-04T08:56:30.645474+0000","last_became_peered":"2024-03-04T08:56:30.645474+0000","last_unstale":"2024-03-04T08:57:23.813785+0000","last_undegraded":"2024-03-04T08:57:23.813785+0000","last_fullsized":"2024-03-04T08:57:23.813785+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:35:40.207215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.812978+0000","last_change":"2024-03-04T08:56:30.656149+0000","last_active":"2024-03-04T08:57:23.812978+0000","last_peered":"2024-03-04T08:57:23.812978+0000","last_clean":"2024-03-04T08:57:23.812978+0000","last_became_active":"2024-03-04T08:56:30.655493+0000","last_became_peered":"2024-03-04T08:56:30.655493+0000","last_unstale":"2024-03-04T08:57:23.812978+0000","last_undegraded":"2024-03-04T08:57:23.812978+0000","last_fullsized":"2024-03-04T08:57:23.812978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:23:28.320328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816645+0000","last_change":"2024-03-04T08:56:30.658309+0000","last_active":"2024-03-04T08:57:23.816645+0000","last_peered":"2024-03-04T08:57:23.816645+0000","last_clean":"2024-03-04T08:57:23.816645+0000","last_became_active":"2024-03-04T08:56:30.656773+0000","last_became_peered":"2024-03-04T08:56:30.656773+0000","last_unstale":"2024-03-04T08:57:23.816645+0000","last_undegraded":"2024-03-04T08:57:23.816645+0000","last_fullsized":"2024-03-04T08:57:23.816645+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:26:57.021442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816806+0000","last_change":"2024-03-04T08:56:30.650336+0000","last_active":"2024-03-04T08:57:23.816806+0000","last_peered":"2024-03-04T08:57:23.816806+0000","last_clean":"2024-03-04T08:57:23.816806+0000","last_became_active":"2024-03-04T08:56:30.648377+0000","last_became_peered":"2024-03-04T08:56:30.648377+0000","last_unstale":"2024-03-04T08:57:23.816806+0000","last_undegraded":"2024-03-04T08:57:23.816806+0000","last_fullsized":"2024-03-04T08:57:23.816806+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:56:41.358895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813248+0000","last_change":"2024-03-04T08:56:30.653327+0000","last_active":"2024-03-04T08:57:23.813248+0000","last_peered":"2024-03-04T08:57:23.813248+0000","last_clean":"2024-03-04T08:57:23.813248+0000","last_became_active":"2024-03-04T08:56:30.652523+0000","last_became_peered":"2024-03-04T08:56:30.652523+0000","last_unstale":"2024-03-04T08:57:23.813248+0000","last_undegraded":"2024-03-04T08:57:23.813248+0000","last_fullsized":"2024-03-04T08:57:23.813248+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:18:41.726921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813425+0000","last_change":"2024-03-04T08:56:30.647210+0000","last_active":"2024-03-04T08:57:23.813425+0000","last_peered":"2024-03-04T08:57:23.813425+0000","last_clean":"2024-03-04T08:57:23.813425+0000","last_became_active":"2024-03-04T08:56:30.646897+0000","last_became_peered":"2024-03-04T08:56:30.646897+0000","last_unstale":"2024-03-04T08:57:23.813425+0000","last_undegraded":"2024-03-04T08:57:23.813425+0000","last_fullsized":"2024-03-04T08:57:23.813425+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:40:35.021061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.814452+0000","last_change":"2024-03-04T08:56:30.648751+0000","last_active":"2024-03-04T08:57:23.814452+0000","last_peered":"2024-03-04T08:57:23.814452+0000","last_clean":"2024-03-04T08:57:23.814452+0000","last_became_active":"2024-03-04T08:56:30.648457+0000","last_became_peered":"2024-03-04T08:56:30.648457+0000","last_unstale":"2024-03-04T08:57:23.814452+0000","last_undegraded":"2024-03-04T08:57:23.814452+0000","last_fullsized":"2024-03-04T08:57:23.814452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:20:20.435769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.811779+0000","last_change":"2024-03-04T08:56:30.644523+0000","last_active":"2024-03-04T08:57:23.811779+0000","last_peered":"2024-03-04T08:57:23.811779+0000","last_clean":"2024-03-04T08:57:23.811779+0000","last_became_active":"2024-03-04T08:56:30.644183+0000","last_became_peered":"2024-03-04T08:56:30.644183+0000","last_unstale":"2024-03-04T08:57:23.811779+0000","last_undegraded":"2024-03-04T08:57:23.811779+0000","last_fullsized":"2024-03-04T08:57:23.811779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:58:16.065571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.811048+0000","last_change":"2024-03-04T08:56:34.681350+0000","last_active":"2024-03-04T08:57:23.811048+0000","last_peered":"2024-03-04T08:57:23.811048+0000","last_clean":"2024-03-04T08:57:23.811048+0000","last_became_active":"2024-03-04T08:56:34.680940+0000","last_became_peered":"2024-03-04T08:56:34.680940+0000","last_unstale":"2024-03-04T08:57:23.811048+0000","last_undegraded":"2024-03-04T08:57:23.811048+0000","last_fullsized":"2024-03-04T08:57:23.811048+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:41:00.690541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.814319+0000","last_change":"2024-03-04T08:56:29.989827+0000","last_active":"2024-03-04T08:57:23.814319+0000","last_peered":"2024-03-04T08:57:23.814319+0000","last_clean":"2024-03-04T08:57:23.814319+0000","last_became_active":"2024-03-04T08:56:29.989509+0000","last_became_peered":"2024-03-04T08:56:29.989509+0000","last_unstale":"2024-03-04T08:57:23.814319+0000","last_undegraded":"2024-03-04T08:57:23.814319+0000","last_fullsized":"2024-03-04T08:57:23.814319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:27:37.771477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.160609+0000","last_change":"2024-03-04T08:56:30.638354+0000","last_active":"2024-03-04T08:57:24.160609+0000","last_peered":"2024-03-04T08:57:24.160609+0000","last_clean":"2024-03-04T08:57:24.160609+0000","last_became_active":"2024-03-04T08:56:30.638058+0000","last_became_peered":"2024-03-04T08:56:30.638058+0000","last_unstale":"2024-03-04T08:57:24.160609+0000","last_undegraded":"2024-03-04T08:57:24.160609+0000","last_fullsized":"2024-03-04T08:57:24.160609+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:43:51.920411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813316+0000","last_change":"2024-03-04T08:56:34.680206+0000","last_active":"2024-03-04T08:57:23.813316+0000","last_peered":"2024-03-04T08:57:23.813316+0000","last_clean":"2024-03-04T08:57:23.813316+0000","last_became_active":"2024-03-04T08:56:34.679821+0000","last_became_peered":"2024-03-04T08:56:34.679821+0000","last_unstale":"2024-03-04T08:57:23.813316+0000","last_undegraded":"2024-03-04T08:57:23.813316+0000","last_fullsized":"2024-03-04T08:57:23.813316+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:18:39.945962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'1","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.161028+0000","last_change":"2024-03-04T08:56:29.994017+0000","last_active":"2024-03-04T08:57:24.161028+0000","last_peered":"2024-03-04T08:57:24.161028+0000","last_clean":"2024-03-04T08:57:24.161028+0000","last_became_active":"2024-03-04T08:56:29.993687+0000","last_became_peered":"2024-03-04T08:56:29.993687+0000","last_unstale":"2024-03-04T08:57:24.161028+0000","last_undegraded":"2024-03-04T08:57:24.161028+0000","last_fullsized":"2024-03-04T08:57:24.161028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:32:51.542130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816984+0000","last_change":"2024-03-04T08:56:30.654446+0000","last_active":"2024-03-04T08:57:23.816984+0000","last_peered":"2024-03-04T08:57:23.816984+0000","last_clean":"2024-03-04T08:57:23.816984+0000","last_became_active":"2024-03-04T08:56:30.653920+0000","last_became_peered":"2024-03-04T08:56:30.653920+0000","last_unstale":"2024-03-04T08:57:23.816984+0000","last_undegraded":"2024-03-04T08:57:23.816984+0000","last_fullsized":"2024-03-04T08:57:23.816984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:48:43.177893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813859+0000","last_change":"2024-03-04T08:56:34.680888+0000","last_active":"2024-03-04T08:57:23.813859+0000","last_peered":"2024-03-04T08:57:23.813859+0000","last_clean":"2024-03-04T08:57:23.813859+0000","last_became_active":"2024-03-04T08:56:34.679950+0000","last_became_peered":"2024-03-04T08:56:34.679950+0000","last_unstale":"2024-03-04T08:57:23.813859+0000","last_undegraded":"2024-03-04T08:57:23.813859+0000","last_fullsized":"2024-03-04T08:57:23.813859+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:08:26.660124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816585+0000","last_change":"2024-03-04T08:56:29.986476+0000","last_active":"2024-03-04T08:57:23.816585+0000","last_peered":"2024-03-04T08:57:23.816585+0000","last_clean":"2024-03-04T08:57:23.816585+0000","last_became_active":"2024-03-04T08:56:29.986106+0000","last_became_peered":"2024-03-04T08:56:29.986106+0000","last_unstale":"2024-03-04T08:57:23.816585+0000","last_undegraded":"2024-03-04T08:57:23.816585+0000","last_fullsized":"2024-03-04T08:57:23.816585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:45:15.775106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.814028+0000","last_change":"2024-03-04T08:56:30.645841+0000","last_active":"2024-03-04T08:57:23.814028+0000","last_peered":"2024-03-04T08:57:23.814028+0000","last_clean":"2024-03-04T08:57:23.814028+0000","last_became_active":"2024-03-04T08:56:30.645459+0000","last_became_peered":"2024-03-04T08:56:30.645459+0000","last_unstale":"2024-03-04T08:57:23.814028+0000","last_undegraded":"2024-03-04T08:57:23.814028+0000","last_fullsized":"2024-03-04T08:57:23.814028+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:37:51.204517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813447+0000","last_change":"2024-03-04T08:56:34.674562+0000","last_active":"2024-03-04T08:57:23.813447+0000","last_peered":"2024-03-04T08:57:23.813447+0000","last_clean":"2024-03-04T08:57:23.813447+0000","last_became_active":"2024-03-04T08:56:34.674239+0000","last_became_peered":"2024-03-04T08:56:34.674239+0000","last_unstale":"2024-03-04T08:57:23.813447+0000","last_undegraded":"2024-03-04T08:57:23.813447+0000","last_fullsized":"2024-03-04T08:57:23.813447+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:47:34.819487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816862+0000","last_change":"2024-03-04T08:56:29.984025+0000","last_active":"2024-03-04T08:57:23.816862+0000","last_peered":"2024-03-04T08:57:23.816862+0000","last_clean":"2024-03-04T08:57:23.816862+0000","last_became_active":"2024-03-04T08:56:29.983676+0000","last_became_peered":"2024-03-04T08:56:29.983676+0000","last_unstale":"2024-03-04T08:57:23.816862+0000","last_undegraded":"2024-03-04T08:57:23.816862+0000","last_fullsized":"2024-03-04T08:57:23.816862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:48:24.920513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.820557+0000","last_change":"2024-03-04T08:56:30.649301+0000","last_active":"2024-03-04T08:57:23.820557+0000","last_peered":"2024-03-04T08:57:23.820557+0000","last_clean":"2024-03-04T08:57:23.820557+0000","last_became_active":"2024-03-04T08:56:30.643987+0000","last_became_peered":"2024-03-04T08:56:30.643987+0000","last_unstale":"2024-03-04T08:57:23.820557+0000","last_undegraded":"2024-03-04T08:57:23.820557+0000","last_fullsized":"2024-03-04T08:57:23.820557+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:05:21.171953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.811629+0000","last_change":"2024-03-04T08:56:34.672893+0000","last_active":"2024-03-04T08:57:23.811629+0000","last_peered":"2024-03-04T08:57:23.811629+0000","last_clean":"2024-03-04T08:57:23.811629+0000","last_became_active":"2024-03-04T08:56:34.672364+0000","last_became_peered":"2024-03-04T08:56:34.672364+0000","last_unstale":"2024-03-04T08:57:23.811629+0000","last_undegraded":"2024-03-04T08:57:23.811629+0000","last_fullsized":"2024-03-04T08:57:23.811629+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:33:29.287772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'1","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.819456+0000","last_change":"2024-03-04T08:56:29.986822+0000","last_active":"2024-03-04T08:57:23.819456+0000","last_peered":"2024-03-04T08:57:23.819456+0000","last_clean":"2024-03-04T08:57:23.819456+0000","last_became_active":"2024-03-04T08:56:29.986498+0000","last_became_peered":"2024-03-04T08:56:29.986498+0000","last_unstale":"2024-03-04T08:57:23.819456+0000","last_undegraded":"2024-03-04T08:57:23.819456+0000","last_fullsized":"2024-03-04T08:57:23.819456+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:15:01.585997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.815645+0000","last_change":"2024-03-04T08:56:30.646164+0000","last_active":"2024-03-04T08:57:23.815645+0000","last_peered":"2024-03-04T08:57:23.815645+0000","last_clean":"2024-03-04T08:57:23.815645+0000","last_became_active":"2024-03-04T08:56:30.645843+0000","last_became_peered":"2024-03-04T08:56:30.645843+0000","last_unstale":"2024-03-04T08:57:23.815645+0000","last_undegraded":"2024-03-04T08:57:23.815645+0000","last_fullsized":"2024-03-04T08:57:23.815645+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:11:48.082226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.811857+0000","last_change":"2024-03-04T08:56:34.680387+0000","last_active":"2024-03-04T08:57:23.811857+0000","last_peered":"2024-03-04T08:57:23.811857+0000","last_clean":"2024-03-04T08:57:23.811857+0000","last_became_active":"2024-03-04T08:56:34.680005+0000","last_became_peered":"2024-03-04T08:56:34.680005+0000","last_unstale":"2024-03-04T08:57:23.811857+0000","last_undegraded":"2024-03-04T08:57:23.811857+0000","last_fullsized":"2024-03-04T08:57:23.811857+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:04:40.403187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'1","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.815686+0000","last_change":"2024-03-04T08:57:23.815686+0000","last_active":"2024-03-04T08:57:23.815686+0000","last_peered":"2024-03-04T08:57:23.815686+0000","last_clean":"2024-03-04T08:57:23.815686+0000","last_became_active":"2024-03-04T08:57:23.815243+0000","last_became_peered":"2024-03-04T08:57:23.815243+0000","last_unstale":"2024-03-04T08:57:23.815686+0000","last_undegraded":"2024-03-04T08:57:23.815686+0000","last_fullsized":"2024-03-04T08:57:23.815686+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:56:49.407722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.819641+0000","last_change":"2024-03-04T08:56:30.649321+0000","last_active":"2024-03-04T08:57:23.819641+0000","last_peered":"2024-03-04T08:57:23.819641+0000","last_clean":"2024-03-04T08:57:23.819641+0000","last_became_active":"2024-03-04T08:56:30.648705+0000","last_became_peered":"2024-03-04T08:56:30.648705+0000","last_unstale":"2024-03-04T08:57:23.819641+0000","last_undegraded":"2024-03-04T08:57:23.819641+0000","last_fullsized":"2024-03-04T08:57:23.819641+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:26:12.549471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813388+0000","last_change":"2024-03-04T08:56:34.682560+0000","last_active":"2024-03-04T08:57:23.813388+0000","last_peered":"2024-03-04T08:57:23.813388+0000","last_clean":"2024-03-04T08:57:23.813388+0000","last_became_active":"2024-03-04T08:56:34.682156+0000","last_became_peered":"2024-03-04T08:56:34.682156+0000","last_unstale":"2024-03-04T08:57:23.813388+0000","last_undegraded":"2024-03-04T08:57:23.813388+0000","last_fullsized":"2024-03-04T08:57:23.813388+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:57:49.132436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.812575+0000","last_change":"2024-03-04T08:56:29.984947+0000","last_active":"2024-03-04T08:57:23.812575+0000","last_peered":"2024-03-04T08:57:23.812575+0000","last_clean":"2024-03-04T08:57:23.812575+0000","last_became_active":"2024-03-04T08:56:29.984575+0000","last_became_peered":"2024-03-04T08:56:29.984575+0000","last_unstale":"2024-03-04T08:57:23.812575+0000","last_undegraded":"2024-03-04T08:57:23.812575+0000","last_fullsized":"2024-03-04T08:57:23.812575+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:01:16.698417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.160555+0000","last_change":"2024-03-04T08:56:30.649000+0000","last_active":"2024-03-04T08:57:24.160555+0000","last_peered":"2024-03-04T08:57:24.160555+0000","last_clean":"2024-03-04T08:57:24.160555+0000","last_became_active":"2024-03-04T08:56:30.648700+0000","last_became_peered":"2024-03-04T08:56:30.648700+0000","last_unstale":"2024-03-04T08:57:24.160555+0000","last_undegraded":"2024-03-04T08:57:24.160555+0000","last_fullsized":"2024-03-04T08:57:24.160555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:46:09.384040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816872+0000","last_change":"2024-03-04T08:56:34.671042+0000","last_active":"2024-03-04T08:57:23.816872+0000","last_peered":"2024-03-04T08:57:23.816872+0000","last_clean":"2024-03-04T08:57:23.816872+0000","last_became_active":"2024-03-04T08:56:34.670683+0000","last_became_peered":"2024-03-04T08:56:34.670683+0000","last_unstale":"2024-03-04T08:57:23.816872+0000","last_undegraded":"2024-03-04T08:57:23.816872+0000","last_fullsized":"2024-03-04T08:57:23.816872+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:13:09.166986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.819316+0000","last_change":"2024-03-04T08:56:29.983359+0000","last_active":"2024-03-04T08:57:23.819316+0000","last_peered":"2024-03-04T08:57:23.819316+0000","last_clean":"2024-03-04T08:57:23.819316+0000","last_became_active":"2024-03-04T08:56:29.983009+0000","last_became_peered":"2024-03-04T08:56:29.983009+0000","last_unstale":"2024-03-04T08:57:23.819316+0000","last_undegraded":"2024-03-04T08:57:23.819316+0000","last_fullsized":"2024-03-04T08:57:23.819316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:03:16.549109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813656+0000","last_change":"2024-03-04T08:56:30.649612+0000","last_active":"2024-03-04T08:57:23.813656+0000","last_peered":"2024-03-04T08:57:23.813656+0000","last_clean":"2024-03-04T08:57:23.813656+0000","last_became_active":"2024-03-04T08:56:30.649299+0000","last_became_peered":"2024-03-04T08:56:30.649299+0000","last_unstale":"2024-03-04T08:57:23.813656+0000","last_undegraded":"2024-03-04T08:57:23.813656+0000","last_fullsized":"2024-03-04T08:57:23.813656+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:57:50.571696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.160731+0000","last_change":"2024-03-04T08:56:34.677974+0000","last_active":"2024-03-04T08:57:24.160731+0000","last_peered":"2024-03-04T08:57:24.160731+0000","last_clean":"2024-03-04T08:57:24.160731+0000","last_became_active":"2024-03-04T08:56:34.676628+0000","last_became_peered":"2024-03-04T08:56:34.676628+0000","last_unstale":"2024-03-04T08:57:24.160731+0000","last_undegraded":"2024-03-04T08:57:24.160731+0000","last_fullsized":"2024-03-04T08:57:24.160731+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:18:40.301336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.810975+0000","last_change":"2024-03-04T08:56:29.976760+0000","last_active":"2024-03-04T08:57:23.810975+0000","last_peered":"2024-03-04T08:57:23.810975+0000","last_clean":"2024-03-04T08:57:23.810975+0000","last_became_active":"2024-03-04T08:56:29.976446+0000","last_became_peered":"2024-03-04T08:56:29.976446+0000","last_unstale":"2024-03-04T08:57:23.810975+0000","last_undegraded":"2024-03-04T08:57:23.810975+0000","last_fullsized":"2024-03-04T08:57:23.810975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:46:35.038622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813548+0000","last_change":"2024-03-04T08:56:30.644326+0000","last_active":"2024-03-04T08:57:23.813548+0000","last_peered":"2024-03-04T08:57:23.813548+0000","last_clean":"2024-03-04T08:57:23.813548+0000","last_became_active":"2024-03-04T08:56:30.644017+0000","last_became_peered":"2024-03-04T08:56:30.644017+0000","last_unstale":"2024-03-04T08:57:23.813548+0000","last_undegraded":"2024-03-04T08:57:23.813548+0000","last_fullsized":"2024-03-04T08:57:23.813548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:01:40.843915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813546+0000","last_change":"2024-03-04T08:56:34.675757+0000","last_active":"2024-03-04T08:57:23.813546+0000","last_peered":"2024-03-04T08:57:23.813546+0000","last_clean":"2024-03-04T08:57:23.813546+0000","last_became_active":"2024-03-04T08:56:34.675375+0000","last_became_peered":"2024-03-04T08:56:34.675375+0000","last_unstale":"2024-03-04T08:57:23.813546+0000","last_undegraded":"2024-03-04T08:57:23.813546+0000","last_fullsized":"2024-03-04T08:57:23.813546+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:16:12.762024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816521+0000","last_change":"2024-03-04T08:56:29.987614+0000","last_active":"2024-03-04T08:57:23.816521+0000","last_peered":"2024-03-04T08:57:23.816521+0000","last_clean":"2024-03-04T08:57:23.816521+0000","last_became_active":"2024-03-04T08:56:29.986639+0000","last_became_peered":"2024-03-04T08:56:29.986639+0000","last_unstale":"2024-03-04T08:57:23.816521+0000","last_undegraded":"2024-03-04T08:57:23.816521+0000","last_fullsized":"2024-03-04T08:57:23.816521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:18:50.320069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816761+0000","last_change":"2024-03-04T08:56:30.655145+0000","last_active":"2024-03-04T08:57:23.816761+0000","last_peered":"2024-03-04T08:57:23.816761+0000","last_clean":"2024-03-04T08:57:23.816761+0000","last_became_active":"2024-03-04T08:56:30.654834+0000","last_became_peered":"2024-03-04T08:56:30.654834+0000","last_unstale":"2024-03-04T08:57:23.816761+0000","last_undegraded":"2024-03-04T08:57:23.816761+0000","last_fullsized":"2024-03-04T08:57:23.816761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:45:19.567325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.814077+0000","last_change":"2024-03-04T08:56:34.680530+0000","last_active":"2024-03-04T08:57:23.814077+0000","last_peered":"2024-03-04T08:57:23.814077+0000","last_clean":"2024-03-04T08:57:23.814077+0000","last_became_active":"2024-03-04T08:56:34.679860+0000","last_became_peered":"2024-03-04T08:56:34.679860+0000","last_unstale":"2024-03-04T08:57:23.814077+0000","last_undegraded":"2024-03-04T08:57:23.814077+0000","last_fullsized":"2024-03-04T08:57:23.814077+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:25:09.036145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'3","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.820442+0000","last_change":"2024-03-04T08:56:29.982516+0000","last_active":"2024-03-04T08:57:23.820442+0000","last_peered":"2024-03-04T08:57:23.820442+0000","last_clean":"2024-03-04T08:57:23.820442+0000","last_became_active":"2024-03-04T08:56:29.982166+0000","last_became_peered":"2024-03-04T08:56:29.982166+0000","last_unstale":"2024-03-04T08:57:23.820442+0000","last_undegraded":"2024-03-04T08:57:23.820442+0000","last_fullsized":"2024-03-04T08:57:23.820442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:48:57.463584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.814193+0000","last_change":"2024-03-04T08:56:30.650030+0000","last_active":"2024-03-04T08:57:23.814193+0000","last_peered":"2024-03-04T08:57:23.814193+0000","last_clean":"2024-03-04T08:57:23.814193+0000","last_became_active":"2024-03-04T08:56:30.647826+0000","last_became_peered":"2024-03-04T08:56:30.647826+0000","last_unstale":"2024-03-04T08:57:23.814193+0000","last_undegraded":"2024-03-04T08:57:23.814193+0000","last_fullsized":"2024-03-04T08:57:23.814193+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:07:45.189935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813823+0000","last_change":"2024-03-04T08:56:34.679189+0000","last_active":"2024-03-04T08:57:23.813823+0000","last_peered":"2024-03-04T08:57:23.813823+0000","last_clean":"2024-03-04T08:57:23.813823+0000","last_became_active":"2024-03-04T08:56:34.678744+0000","last_became_peered":"2024-03-04T08:56:34.678744+0000","last_unstale":"2024-03-04T08:57:23.813823+0000","last_undegraded":"2024-03-04T08:57:23.813823+0000","last_fullsized":"2024-03-04T08:57:23.813823+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:05:15.128703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'30","reported_seq":223,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:58:05.879369+0000","last_change":"2024-03-04T08:57:23.825562+0000","last_active":"2024-03-04T08:58:05.879369+0000","last_peered":"2024-03-04T08:58:05.879369+0000","last_clean":"2024-03-04T08:58:05.879369+0000","last_became_active":"2024-03-04T08:57:23.814977+0000","last_became_peered":"2024-03-04T08:57:23.814977+0000","last_unstale":"2024-03-04T08:58:05.879369+0000","last_undegraded":"2024-03-04T08:58:05.879369+0000","last_fullsized":"2024-03-04T08:58:05.879369+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:29:26.493310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63356,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":141,"num_read_kb":132,"num_write":30,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":63356,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816374+0000","last_change":"2024-03-04T08:56:30.647764+0000","last_active":"2024-03-04T08:57:23.816374+0000","last_peered":"2024-03-04T08:57:23.816374+0000","last_clean":"2024-03-04T08:57:23.816374+0000","last_became_active":"2024-03-04T08:56:30.647445+0000","last_became_peered":"2024-03-04T08:56:30.647445+0000","last_unstale":"2024-03-04T08:57:23.816374+0000","last_undegraded":"2024-03-04T08:57:23.816374+0000","last_fullsized":"2024-03-04T08:57:23.816374+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:13:25.135993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.820947+0000","last_change":"2024-03-04T08:56:34.674674+0000","last_active":"2024-03-04T08:57:23.820947+0000","last_peered":"2024-03-04T08:57:23.820947+0000","last_clean":"2024-03-04T08:57:23.820947+0000","last_became_active":"2024-03-04T08:56:34.674138+0000","last_became_peered":"2024-03-04T08:56:34.674138+0000","last_unstale":"2024-03-04T08:57:23.820947+0000","last_undegraded":"2024-03-04T08:57:23.820947+0000","last_fullsized":"2024-03-04T08:57:23.820947+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:20:36.123111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.811491+0000","last_change":"2024-03-04T08:56:29.980997+0000","last_active":"2024-03-04T08:57:23.811491+0000","last_peered":"2024-03-04T08:57:23.811491+0000","last_clean":"2024-03-04T08:57:23.811491+0000","last_became_active":"2024-03-04T08:56:29.980651+0000","last_became_peered":"2024-03-04T08:56:29.980651+0000","last_unstale":"2024-03-04T08:57:23.811491+0000","last_undegraded":"2024-03-04T08:57:23.811491+0000","last_fullsized":"2024-03-04T08:57:23.811491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:23:33.434703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.812705+0000","last_change":"2024-03-04T08:56:30.657229+0000","last_active":"2024-03-04T08:57:23.812705+0000","last_peered":"2024-03-04T08:57:23.812705+0000","last_clean":"2024-03-04T08:57:23.812705+0000","last_became_active":"2024-03-04T08:56:30.656877+0000","last_became_peered":"2024-03-04T08:56:30.656877+0000","last_unstale":"2024-03-04T08:57:23.812705+0000","last_undegraded":"2024-03-04T08:57:23.812705+0000","last_fullsized":"2024-03-04T08:57:23.812705+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:17:55.063418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.818314+0000","last_change":"2024-03-04T08:57:23.818314+0000","last_active":"2024-03-04T08:57:23.818314+0000","last_peered":"2024-03-04T08:57:23.818314+0000","last_clean":"2024-03-04T08:57:23.818314+0000","last_became_active":"2024-03-04T08:57:23.817831+0000","last_became_peered":"2024-03-04T08:57:23.817831+0000","last_unstale":"2024-03-04T08:57:23.818314+0000","last_undegraded":"2024-03-04T08:57:23.818314+0000","last_fullsized":"2024-03-04T08:57:23.818314+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:06:54.889154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.819691+0000","last_change":"2024-03-04T08:56:29.987759+0000","last_active":"2024-03-04T08:57:23.819691+0000","last_peered":"2024-03-04T08:57:23.819691+0000","last_clean":"2024-03-04T08:57:23.819691+0000","last_became_active":"2024-03-04T08:56:29.987458+0000","last_became_peered":"2024-03-04T08:56:29.987458+0000","last_unstale":"2024-03-04T08:57:23.819691+0000","last_undegraded":"2024-03-04T08:57:23.819691+0000","last_fullsized":"2024-03-04T08:57:23.819691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:36:18.878556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.818888+0000","last_change":"2024-03-04T08:57:23.818888+0000","last_active":"2024-03-04T08:57:23.818888+0000","last_peered":"2024-03-04T08:57:23.818888+0000","last_clean":"2024-03-04T08:57:23.818888+0000","last_became_active":"2024-03-04T08:57:23.818510+0000","last_became_peered":"2024-03-04T08:57:23.818510+0000","last_unstale":"2024-03-04T08:57:23.818888+0000","last_undegraded":"2024-03-04T08:57:23.818888+0000","last_fullsized":"2024-03-04T08:57:23.818888+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:50:21.276520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.814191+0000","last_change":"2024-03-04T08:56:34.683727+0000","last_active":"2024-03-04T08:57:23.814191+0000","last_peered":"2024-03-04T08:57:23.814191+0000","last_clean":"2024-03-04T08:57:23.814191+0000","last_became_active":"2024-03-04T08:56:34.681865+0000","last_became_peered":"2024-03-04T08:56:34.681865+0000","last_unstale":"2024-03-04T08:57:23.814191+0000","last_undegraded":"2024-03-04T08:57:23.814191+0000","last_fullsized":"2024-03-04T08:57:23.814191+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:44:46.586791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813619+0000","last_change":"2024-03-04T08:56:29.983385+0000","last_active":"2024-03-04T08:57:23.813619+0000","last_peered":"2024-03-04T08:57:23.813619+0000","last_clean":"2024-03-04T08:57:23.813619+0000","last_became_active":"2024-03-04T08:56:29.983079+0000","last_became_peered":"2024-03-04T08:56:29.983079+0000","last_unstale":"2024-03-04T08:57:23.813619+0000","last_undegraded":"2024-03-04T08:57:23.813619+0000","last_fullsized":"2024-03-04T08:57:23.813619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:23:15.417389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.815799+0000","last_change":"2024-03-04T08:56:30.641187+0000","last_active":"2024-03-04T08:57:23.815799+0000","last_peered":"2024-03-04T08:57:23.815799+0000","last_clean":"2024-03-04T08:57:23.815799+0000","last_became_active":"2024-03-04T08:56:30.640863+0000","last_became_peered":"2024-03-04T08:56:30.640863+0000","last_unstale":"2024-03-04T08:57:23.815799+0000","last_undegraded":"2024-03-04T08:57:23.815799+0000","last_fullsized":"2024-03-04T08:57:23.815799+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:08:30.257806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.162018+0000","last_change":"2024-03-04T08:57:24.162018+0000","last_active":"2024-03-04T08:57:24.162018+0000","last_peered":"2024-03-04T08:57:24.162018+0000","last_clean":"2024-03-04T08:57:24.162018+0000","last_became_active":"2024-03-04T08:57:24.161551+0000","last_became_peered":"2024-03-04T08:57:24.161551+0000","last_unstale":"2024-03-04T08:57:24.162018+0000","last_undegraded":"2024-03-04T08:57:24.162018+0000","last_fullsized":"2024-03-04T08:57:24.162018+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:21:35.698806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'8","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.815252+0000","last_change":"2024-03-04T08:56:29.994016+0000","last_active":"2024-03-04T08:57:23.815252+0000","last_peered":"2024-03-04T08:57:23.815252+0000","last_clean":"2024-03-04T08:57:23.815252+0000","last_became_active":"2024-03-04T08:56:29.993683+0000","last_became_peered":"2024-03-04T08:56:29.993683+0000","last_unstale":"2024-03-04T08:57:23.815252+0000","last_undegraded":"2024-03-04T08:57:23.815252+0000","last_fullsized":"2024-03-04T08:57:23.815252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:25:37.931248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.819930+0000","last_change":"2024-03-04T08:56:30.647390+0000","last_active":"2024-03-04T08:57:23.819930+0000","last_peered":"2024-03-04T08:57:23.819930+0000","last_clean":"2024-03-04T08:57:23.819930+0000","last_became_active":"2024-03-04T08:56:30.643879+0000","last_became_peered":"2024-03-04T08:56:30.643879+0000","last_unstale":"2024-03-04T08:57:23.819930+0000","last_undegraded":"2024-03-04T08:57:23.819930+0000","last_fullsized":"2024-03-04T08:57:23.819930+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:41:13.030295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.162126+0000","last_change":"2024-03-04T08:57:24.162126+0000","last_active":"2024-03-04T08:57:24.162126+0000","last_peered":"2024-03-04T08:57:24.162126+0000","last_clean":"2024-03-04T08:57:24.162126+0000","last_became_active":"2024-03-04T08:57:24.161597+0000","last_became_peered":"2024-03-04T08:57:24.161597+0000","last_unstale":"2024-03-04T08:57:24.162126+0000","last_undegraded":"2024-03-04T08:57:24.162126+0000","last_fullsized":"2024-03-04T08:57:24.162126+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:56:01.936796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"86'1","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.815985+0000","last_change":"2024-03-04T08:56:29.983195+0000","last_active":"2024-03-04T08:57:23.815985+0000","last_peered":"2024-03-04T08:57:23.815985+0000","last_clean":"2024-03-04T08:57:23.815985+0000","last_became_active":"2024-03-04T08:56:29.982881+0000","last_became_peered":"2024-03-04T08:56:29.982881+0000","last_unstale":"2024-03-04T08:57:23.815985+0000","last_undegraded":"2024-03-04T08:57:23.815985+0000","last_fullsized":"2024-03-04T08:57:23.815985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:59:12.802430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813582+0000","last_change":"2024-03-04T08:56:30.650452+0000","last_active":"2024-03-04T08:57:23.813582+0000","last_peered":"2024-03-04T08:57:23.813582+0000","last_clean":"2024-03-04T08:57:23.813582+0000","last_became_active":"2024-03-04T08:56:30.650148+0000","last_became_peered":"2024-03-04T08:56:30.650148+0000","last_unstale":"2024-03-04T08:57:23.813582+0000","last_undegraded":"2024-03-04T08:57:23.813582+0000","last_fullsized":"2024-03-04T08:57:23.813582+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:22:55.741505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.815542+0000","last_change":"2024-03-04T08:56:34.860608+0000","last_active":"2024-03-04T08:57:23.815542+0000","last_peered":"2024-03-04T08:57:23.815542+0000","last_clean":"2024-03-04T08:57:23.815542+0000","last_became_active":"2024-03-04T08:56:34.860087+0000","last_became_peered":"2024-03-04T08:56:34.860087+0000","last_unstale":"2024-03-04T08:57:23.815542+0000","last_undegraded":"2024-03-04T08:57:23.815542+0000","last_fullsized":"2024-03-04T08:57:23.815542+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:33:35.675886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.814013+0000","last_change":"2024-03-04T08:56:29.980947+0000","last_active":"2024-03-04T08:57:23.814013+0000","last_peered":"2024-03-04T08:57:23.814013+0000","last_clean":"2024-03-04T08:57:23.814013+0000","last_became_active":"2024-03-04T08:56:29.980625+0000","last_became_peered":"2024-03-04T08:56:29.980625+0000","last_unstale":"2024-03-04T08:57:23.814013+0000","last_undegraded":"2024-03-04T08:57:23.814013+0000","last_fullsized":"2024-03-04T08:57:23.814013+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:53:53.944112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.810713+0000","last_change":"2024-03-04T08:56:30.642047+0000","last_active":"2024-03-04T08:57:23.810713+0000","last_peered":"2024-03-04T08:57:23.810713+0000","last_clean":"2024-03-04T08:57:23.810713+0000","last_became_active":"2024-03-04T08:56:30.641700+0000","last_became_peered":"2024-03-04T08:56:30.641700+0000","last_unstale":"2024-03-04T08:57:23.810713+0000","last_undegraded":"2024-03-04T08:57:23.810713+0000","last_fullsized":"2024-03-04T08:57:23.810713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:43:53.641512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813031+0000","last_change":"2024-03-04T08:56:34.674944+0000","last_active":"2024-03-04T08:57:23.813031+0000","last_peered":"2024-03-04T08:57:23.813031+0000","last_clean":"2024-03-04T08:57:23.813031+0000","last_became_active":"2024-03-04T08:56:34.674000+0000","last_became_peered":"2024-03-04T08:56:34.674000+0000","last_unstale":"2024-03-04T08:57:23.813031+0000","last_undegraded":"2024-03-04T08:57:23.813031+0000","last_fullsized":"2024-03-04T08:57:23.813031+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:26:19.386329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816135+0000","last_change":"2024-03-04T08:56:29.989020+0000","last_active":"2024-03-04T08:57:23.816135+0000","last_peered":"2024-03-04T08:57:23.816135+0000","last_clean":"2024-03-04T08:57:23.816135+0000","last_became_active":"2024-03-04T08:56:29.988751+0000","last_became_peered":"2024-03-04T08:56:29.988751+0000","last_unstale":"2024-03-04T08:57:23.816135+0000","last_undegraded":"2024-03-04T08:57:23.816135+0000","last_fullsized":"2024-03-04T08:57:23.816135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:38:45.003221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.810596+0000","last_change":"2024-03-04T08:56:30.649976+0000","last_active":"2024-03-04T08:57:23.810596+0000","last_peered":"2024-03-04T08:57:23.810596+0000","last_clean":"2024-03-04T08:57:23.810596+0000","last_became_active":"2024-03-04T08:56:30.649680+0000","last_became_peered":"2024-03-04T08:56:30.649680+0000","last_unstale":"2024-03-04T08:57:23.810596+0000","last_undegraded":"2024-03-04T08:57:23.810596+0000","last_fullsized":"2024-03-04T08:57:23.810596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:50:02.197087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813389+0000","last_change":"2024-03-04T08:56:34.860654+0000","last_active":"2024-03-04T08:57:23.813389+0000","last_peered":"2024-03-04T08:57:23.813389+0000","last_clean":"2024-03-04T08:57:23.813389+0000","last_became_active":"2024-03-04T08:56:34.860170+0000","last_became_peered":"2024-03-04T08:56:34.860170+0000","last_unstale":"2024-03-04T08:57:23.813389+0000","last_undegraded":"2024-03-04T08:57:23.813389+0000","last_fullsized":"2024-03-04T08:57:23.813389+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:08:49.703317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.819410+0000","last_change":"2024-03-04T08:56:30.643510+0000","last_active":"2024-03-04T08:57:23.819410+0000","last_peered":"2024-03-04T08:57:23.819410+0000","last_clean":"2024-03-04T08:57:23.819410+0000","last_became_active":"2024-03-04T08:56:30.643141+0000","last_became_peered":"2024-03-04T08:56:30.643141+0000","last_unstale":"2024-03-04T08:57:23.819410+0000","last_undegraded":"2024-03-04T08:57:23.819410+0000","last_fullsized":"2024-03-04T08:57:23.819410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:36:03.619735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.812466+0000","last_change":"2024-03-04T08:56:29.994030+0000","last_active":"2024-03-04T08:57:23.812466+0000","last_peered":"2024-03-04T08:57:23.812466+0000","last_clean":"2024-03-04T08:57:23.812466+0000","last_became_active":"2024-03-04T08:56:29.993698+0000","last_became_peered":"2024-03-04T08:56:29.993698+0000","last_unstale":"2024-03-04T08:57:23.812466+0000","last_undegraded":"2024-03-04T08:57:23.812466+0000","last_fullsized":"2024-03-04T08:57:23.812466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:28:38.609330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"91'1","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816388+0000","last_change":"2024-03-04T08:56:34.678337+0000","last_active":"2024-03-04T08:57:23.816388+0000","last_peered":"2024-03-04T08:57:23.816388+0000","last_clean":"2024-03-04T08:57:23.816388+0000","last_became_active":"2024-03-04T08:56:34.678091+0000","last_became_peered":"2024-03-04T08:56:34.678091+0000","last_unstale":"2024-03-04T08:57:23.816388+0000","last_undegraded":"2024-03-04T08:57:23.816388+0000","last_fullsized":"2024-03-04T08:57:23.816388+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:05:54.075257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.160651+0000","last_change":"2024-03-04T08:56:30.641609+0000","last_active":"2024-03-04T08:57:24.160651+0000","last_peered":"2024-03-04T08:57:24.160651+0000","last_clean":"2024-03-04T08:57:24.160651+0000","last_became_active":"2024-03-04T08:56:30.641323+0000","last_became_peered":"2024-03-04T08:56:30.641323+0000","last_unstale":"2024-03-04T08:57:24.160651+0000","last_undegraded":"2024-03-04T08:57:24.160651+0000","last_fullsized":"2024-03-04T08:57:24.160651+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:39:03.935424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813966+0000","last_change":"2024-03-04T08:56:29.976581+0000","last_active":"2024-03-04T08:57:23.813966+0000","last_peered":"2024-03-04T08:57:23.813966+0000","last_clean":"2024-03-04T08:57:23.813966+0000","last_became_active":"2024-03-04T08:56:29.976215+0000","last_became_peered":"2024-03-04T08:56:29.976215+0000","last_unstale":"2024-03-04T08:57:23.813966+0000","last_undegraded":"2024-03-04T08:57:23.813966+0000","last_fullsized":"2024-03-04T08:57:23.813966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:53:29.040404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":361,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.815720+0000","last_change":"2024-03-04T08:55:10.344909+0000","last_active":"2024-03-04T08:57:23.815720+0000","last_peered":"2024-03-04T08:57:23.815720+0000","last_clean":"2024-03-04T08:57:23.815720+0000","last_became_active":"2024-03-04T08:55:10.344424+0000","last_became_peered":"2024-03-04T08:55:10.344424+0000","last_unstale":"2024-03-04T08:57:23.815720+0000","last_undegraded":"2024-03-04T08:57:23.815720+0000","last_fullsized":"2024-03-04T08:57:23.815720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:51:15.334055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:51:15.334055+0000","last_clean_scrub_stamp":"2024-03-04T08:51:15.334055+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:36:29.766101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.820502+0000","last_change":"2024-03-04T08:56:34.861141+0000","last_active":"2024-03-04T08:57:23.820502+0000","last_peered":"2024-03-04T08:57:23.820502+0000","last_clean":"2024-03-04T08:57:23.820502+0000","last_became_active":"2024-03-04T08:56:34.860618+0000","last_became_peered":"2024-03-04T08:56:34.860618+0000","last_unstale":"2024-03-04T08:57:23.820502+0000","last_undegraded":"2024-03-04T08:57:23.820502+0000","last_fullsized":"2024-03-04T08:57:23.820502+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:53:20.918385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.812408+0000","last_change":"2024-03-04T08:56:30.644248+0000","last_active":"2024-03-04T08:57:23.812408+0000","last_peered":"2024-03-04T08:57:23.812408+0000","last_clean":"2024-03-04T08:57:23.812408+0000","last_became_active":"2024-03-04T08:56:30.643956+0000","last_became_peered":"2024-03-04T08:56:30.643956+0000","last_unstale":"2024-03-04T08:57:23.812408+0000","last_undegraded":"2024-03-04T08:57:23.812408+0000","last_fullsized":"2024-03-04T08:57:23.812408+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:19:24.298145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816701+0000","last_change":"2024-03-04T08:56:29.987584+0000","last_active":"2024-03-04T08:57:23.816701+0000","last_peered":"2024-03-04T08:57:23.816701+0000","last_clean":"2024-03-04T08:57:23.816701+0000","last_became_active":"2024-03-04T08:56:29.986741+0000","last_became_peered":"2024-03-04T08:56:29.986741+0000","last_unstale":"2024-03-04T08:57:23.816701+0000","last_undegraded":"2024-03-04T08:57:23.816701+0000","last_fullsized":"2024-03-04T08:57:23.816701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:46:19.672027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813250+0000","last_change":"2024-03-04T08:56:34.675485+0000","last_active":"2024-03-04T08:57:23.813250+0000","last_peered":"2024-03-04T08:57:23.813250+0000","last_clean":"2024-03-04T08:57:23.813250+0000","last_became_active":"2024-03-04T08:56:34.674961+0000","last_became_peered":"2024-03-04T08:56:34.674961+0000","last_unstale":"2024-03-04T08:57:23.813250+0000","last_undegraded":"2024-03-04T08:57:23.813250+0000","last_fullsized":"2024-03-04T08:57:23.813250+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:39:43.387849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816504+0000","last_change":"2024-03-04T08:56:29.992287+0000","last_active":"2024-03-04T08:57:23.816504+0000","last_peered":"2024-03-04T08:57:23.816504+0000","last_clean":"2024-03-04T08:57:23.816504+0000","last_became_active":"2024-03-04T08:56:29.991998+0000","last_became_peered":"2024-03-04T08:56:29.991998+0000","last_unstale":"2024-03-04T08:57:23.816504+0000","last_undegraded":"2024-03-04T08:57:23.816504+0000","last_fullsized":"2024-03-04T08:57:23.816504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:33:10.557684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813150+0000","last_change":"2024-03-04T08:56:30.644190+0000","last_active":"2024-03-04T08:57:23.813150+0000","last_peered":"2024-03-04T08:57:23.813150+0000","last_clean":"2024-03-04T08:57:23.813150+0000","last_became_active":"2024-03-04T08:56:30.643783+0000","last_became_peered":"2024-03-04T08:56:30.643783+0000","last_unstale":"2024-03-04T08:57:23.813150+0000","last_undegraded":"2024-03-04T08:57:23.813150+0000","last_fullsized":"2024-03-04T08:57:23.813150+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:25:12.886305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.160397+0000","last_change":"2024-03-04T08:56:34.860415+0000","last_active":"2024-03-04T08:57:24.160397+0000","last_peered":"2024-03-04T08:57:24.160397+0000","last_clean":"2024-03-04T08:57:24.160397+0000","last_became_active":"2024-03-04T08:56:34.860008+0000","last_became_peered":"2024-03-04T08:56:34.860008+0000","last_unstale":"2024-03-04T08:57:24.160397+0000","last_undegraded":"2024-03-04T08:57:24.160397+0000","last_fullsized":"2024-03-04T08:57:24.160397+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:55:45.673567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.814550+0000","last_change":"2024-03-04T08:56:29.989599+0000","last_active":"2024-03-04T08:57:23.814550+0000","last_peered":"2024-03-04T08:57:23.814550+0000","last_clean":"2024-03-04T08:57:23.814550+0000","last_became_active":"2024-03-04T08:56:29.989281+0000","last_became_peered":"2024-03-04T08:56:29.989281+0000","last_unstale":"2024-03-04T08:57:23.814550+0000","last_undegraded":"2024-03-04T08:57:23.814550+0000","last_fullsized":"2024-03-04T08:57:23.814550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:59:04.006117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.160732+0000","last_change":"2024-03-04T08:56:30.640315+0000","last_active":"2024-03-04T08:57:24.160732+0000","last_peered":"2024-03-04T08:57:24.160732+0000","last_clean":"2024-03-04T08:57:24.160732+0000","last_became_active":"2024-03-04T08:56:30.640001+0000","last_became_peered":"2024-03-04T08:56:30.640001+0000","last_unstale":"2024-03-04T08:57:24.160732+0000","last_undegraded":"2024-03-04T08:57:24.160732+0000","last_fullsized":"2024-03-04T08:57:24.160732+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:43:57.049948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.811032+0000","last_change":"2024-03-04T08:56:34.667973+0000","last_active":"2024-03-04T08:57:23.811032+0000","last_peered":"2024-03-04T08:57:23.811032+0000","last_clean":"2024-03-04T08:57:23.811032+0000","last_became_active":"2024-03-04T08:56:34.667527+0000","last_became_peered":"2024-03-04T08:56:34.667527+0000","last_unstale":"2024-03-04T08:57:23.811032+0000","last_undegraded":"2024-03-04T08:57:23.811032+0000","last_fullsized":"2024-03-04T08:57:23.811032+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:07:23.991933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813883+0000","last_change":"2024-03-04T08:56:29.982263+0000","last_active":"2024-03-04T08:57:23.813883+0000","last_peered":"2024-03-04T08:57:23.813883+0000","last_clean":"2024-03-04T08:57:23.813883+0000","last_became_active":"2024-03-04T08:56:29.981802+0000","last_became_peered":"2024-03-04T08:56:29.981802+0000","last_unstale":"2024-03-04T08:57:23.813883+0000","last_undegraded":"2024-03-04T08:57:23.813883+0000","last_fullsized":"2024-03-04T08:57:23.813883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:24:33.351428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816995+0000","last_change":"2024-03-04T08:56:30.657847+0000","last_active":"2024-03-04T08:57:23.816995+0000","last_peered":"2024-03-04T08:57:23.816995+0000","last_clean":"2024-03-04T08:57:23.816995+0000","last_became_active":"2024-03-04T08:56:30.657533+0000","last_became_peered":"2024-03-04T08:56:30.657533+0000","last_unstale":"2024-03-04T08:57:23.816995+0000","last_undegraded":"2024-03-04T08:57:23.816995+0000","last_fullsized":"2024-03-04T08:57:23.816995+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:11:01.914455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.814164+0000","last_change":"2024-03-04T08:56:34.861093+0000","last_active":"2024-03-04T08:57:23.814164+0000","last_peered":"2024-03-04T08:57:23.814164+0000","last_clean":"2024-03-04T08:57:23.814164+0000","last_became_active":"2024-03-04T08:56:34.860660+0000","last_became_peered":"2024-03-04T08:56:34.860660+0000","last_unstale":"2024-03-04T08:57:23.814164+0000","last_undegraded":"2024-03-04T08:57:23.814164+0000","last_fullsized":"2024-03-04T08:57:23.814164+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:37:38.424914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813376+0000","last_change":"2024-03-04T08:56:29.990546+0000","last_active":"2024-03-04T08:57:23.813376+0000","last_peered":"2024-03-04T08:57:23.813376+0000","last_clean":"2024-03-04T08:57:23.813376+0000","last_became_active":"2024-03-04T08:56:29.990342+0000","last_became_peered":"2024-03-04T08:56:29.990342+0000","last_unstale":"2024-03-04T08:57:23.813376+0000","last_undegraded":"2024-03-04T08:57:23.813376+0000","last_fullsized":"2024-03-04T08:57:23.813376+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:01:06.929112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.815687+0000","last_change":"2024-03-04T08:56:30.653228+0000","last_active":"2024-03-04T08:57:23.815687+0000","last_peered":"2024-03-04T08:57:23.815687+0000","last_clean":"2024-03-04T08:57:23.815687+0000","last_became_active":"2024-03-04T08:56:30.652926+0000","last_became_peered":"2024-03-04T08:56:30.652926+0000","last_unstale":"2024-03-04T08:57:23.815687+0000","last_undegraded":"2024-03-04T08:57:23.815687+0000","last_fullsized":"2024-03-04T08:57:23.815687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:06:27.728135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.814329+0000","last_change":"2024-03-04T08:56:34.681096+0000","last_active":"2024-03-04T08:57:23.814329+0000","last_peered":"2024-03-04T08:57:23.814329+0000","last_clean":"2024-03-04T08:57:23.814329+0000","last_became_active":"2024-03-04T08:56:34.680718+0000","last_became_peered":"2024-03-04T08:56:34.680718+0000","last_unstale":"2024-03-04T08:57:23.814329+0000","last_undegraded":"2024-03-04T08:57:23.814329+0000","last_fullsized":"2024-03-04T08:57:23.814329+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:04:58.908904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.160895+0000","last_change":"2024-03-04T08:56:29.988039+0000","last_active":"2024-03-04T08:57:24.160895+0000","last_peered":"2024-03-04T08:57:24.160895+0000","last_clean":"2024-03-04T08:57:24.160895+0000","last_became_active":"2024-03-04T08:56:29.987310+0000","last_became_peered":"2024-03-04T08:56:29.987310+0000","last_unstale":"2024-03-04T08:57:24.160895+0000","last_undegraded":"2024-03-04T08:57:24.160895+0000","last_fullsized":"2024-03-04T08:57:24.160895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:19:39.319965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813149+0000","last_change":"2024-03-04T08:56:30.649094+0000","last_active":"2024-03-04T08:57:23.813149+0000","last_peered":"2024-03-04T08:57:23.813149+0000","last_clean":"2024-03-04T08:57:23.813149+0000","last_became_active":"2024-03-04T08:56:30.648776+0000","last_became_peered":"2024-03-04T08:56:30.648776+0000","last_unstale":"2024-03-04T08:57:23.813149+0000","last_undegraded":"2024-03-04T08:57:23.813149+0000","last_fullsized":"2024-03-04T08:57:23.813149+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:23:17.738792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"91'1","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813970+0000","last_change":"2024-03-04T08:56:34.680923+0000","last_active":"2024-03-04T08:57:23.813970+0000","last_peered":"2024-03-04T08:57:23.813970+0000","last_clean":"2024-03-04T08:57:23.813970+0000","last_became_active":"2024-03-04T08:56:34.680114+0000","last_became_peered":"2024-03-04T08:56:34.680114+0000","last_unstale":"2024-03-04T08:57:23.813970+0000","last_undegraded":"2024-03-04T08:57:23.813970+0000","last_fullsized":"2024-03-04T08:57:23.813970+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:05:53.434735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.162742+0000","last_change":"2024-03-04T08:57:24.162162+0000","last_active":"2024-03-04T08:57:24.162742+0000","last_peered":"2024-03-04T08:57:24.162742+0000","last_clean":"2024-03-04T08:57:24.162742+0000","last_became_active":"2024-03-04T08:57:24.161676+0000","last_became_peered":"2024-03-04T08:57:24.161676+0000","last_unstale":"2024-03-04T08:57:24.162742+0000","last_undegraded":"2024-03-04T08:57:24.162742+0000","last_fullsized":"2024-03-04T08:57:24.162742+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:02:29.121039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.160522+0000","last_change":"2024-03-04T08:56:30.654617+0000","last_active":"2024-03-04T08:57:24.160522+0000","last_peered":"2024-03-04T08:57:24.160522+0000","last_clean":"2024-03-04T08:57:24.160522+0000","last_became_active":"2024-03-04T08:56:30.653341+0000","last_became_peered":"2024-03-04T08:56:30.653341+0000","last_unstale":"2024-03-04T08:57:24.160522+0000","last_undegraded":"2024-03-04T08:57:24.160522+0000","last_fullsized":"2024-03-04T08:57:24.160522+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:10:26.234514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813608+0000","last_change":"2024-03-04T08:56:34.860680+0000","last_active":"2024-03-04T08:57:23.813608+0000","last_peered":"2024-03-04T08:57:23.813608+0000","last_clean":"2024-03-04T08:57:23.813608+0000","last_became_active":"2024-03-04T08:56:34.860290+0000","last_became_peered":"2024-03-04T08:56:34.860290+0000","last_unstale":"2024-03-04T08:57:23.813608+0000","last_undegraded":"2024-03-04T08:57:23.813608+0000","last_fullsized":"2024-03-04T08:57:23.813608+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:14:32.126839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816054+0000","last_change":"2024-03-04T08:56:29.989843+0000","last_active":"2024-03-04T08:57:23.816054+0000","last_peered":"2024-03-04T08:57:23.816054+0000","last_clean":"2024-03-04T08:57:23.816054+0000","last_became_active":"2024-03-04T08:56:29.989508+0000","last_became_peered":"2024-03-04T08:56:29.989508+0000","last_unstale":"2024-03-04T08:57:23.816054+0000","last_undegraded":"2024-03-04T08:57:23.816054+0000","last_fullsized":"2024-03-04T08:57:23.816054+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:46:01.738580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816231+0000","last_change":"2024-03-04T08:56:30.644598+0000","last_active":"2024-03-04T08:57:23.816231+0000","last_peered":"2024-03-04T08:57:23.816231+0000","last_clean":"2024-03-04T08:57:23.816231+0000","last_became_active":"2024-03-04T08:56:30.644263+0000","last_became_peered":"2024-03-04T08:56:30.644263+0000","last_unstale":"2024-03-04T08:57:23.816231+0000","last_undegraded":"2024-03-04T08:57:23.816231+0000","last_fullsized":"2024-03-04T08:57:23.816231+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:47:07.013271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813407+0000","last_change":"2024-03-04T08:56:34.675116+0000","last_active":"2024-03-04T08:57:23.813407+0000","last_peered":"2024-03-04T08:57:23.813407+0000","last_clean":"2024-03-04T08:57:23.813407+0000","last_became_active":"2024-03-04T08:56:34.674727+0000","last_became_peered":"2024-03-04T08:56:34.674727+0000","last_unstale":"2024-03-04T08:57:23.813407+0000","last_undegraded":"2024-03-04T08:57:23.813407+0000","last_fullsized":"2024-03-04T08:57:23.813407+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:56:23.461299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.811967+0000","last_change":"2024-03-04T08:56:29.983545+0000","last_active":"2024-03-04T08:57:23.811967+0000","last_peered":"2024-03-04T08:57:23.811967+0000","last_clean":"2024-03-04T08:57:23.811967+0000","last_became_active":"2024-03-04T08:56:29.983244+0000","last_became_peered":"2024-03-04T08:56:29.983244+0000","last_unstale":"2024-03-04T08:57:23.811967+0000","last_undegraded":"2024-03-04T08:57:23.811967+0000","last_fullsized":"2024-03-04T08:57:23.811967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:59:43.475203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816720+0000","last_change":"2024-03-04T08:56:30.641244+0000","last_active":"2024-03-04T08:57:23.816720+0000","last_peered":"2024-03-04T08:57:23.816720+0000","last_clean":"2024-03-04T08:57:23.816720+0000","last_became_active":"2024-03-04T08:56:30.640952+0000","last_became_peered":"2024-03-04T08:56:30.640952+0000","last_unstale":"2024-03-04T08:57:23.816720+0000","last_undegraded":"2024-03-04T08:57:23.816720+0000","last_fullsized":"2024-03-04T08:57:23.816720+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:38:51.134697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.819456+0000","last_change":"2024-03-04T08:57:23.819456+0000","last_active":"2024-03-04T08:57:23.819456+0000","last_peered":"2024-03-04T08:57:23.819456+0000","last_clean":"2024-03-04T08:57:23.819456+0000","last_became_active":"2024-03-04T08:57:23.818994+0000","last_became_peered":"2024-03-04T08:57:23.818994+0000","last_unstale":"2024-03-04T08:57:23.819456+0000","last_undegraded":"2024-03-04T08:57:23.819456+0000","last_fullsized":"2024-03-04T08:57:23.819456+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:27:14.890246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,10,1,4],"acting":[0,10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.820654+0000","last_change":"2024-03-04T08:56:29.980781+0000","last_active":"2024-03-04T08:57:23.820654+0000","last_peered":"2024-03-04T08:57:23.820654+0000","last_clean":"2024-03-04T08:57:23.820654+0000","last_became_active":"2024-03-04T08:56:29.980446+0000","last_became_peered":"2024-03-04T08:56:29.980446+0000","last_unstale":"2024-03-04T08:57:23.820654+0000","last_undegraded":"2024-03-04T08:57:23.820654+0000","last_fullsized":"2024-03-04T08:57:23.820654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:05:37.325902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816274+0000","last_change":"2024-03-04T08:56:30.648350+0000","last_active":"2024-03-04T08:57:23.816274+0000","last_peered":"2024-03-04T08:57:23.816274+0000","last_clean":"2024-03-04T08:57:23.816274+0000","last_became_active":"2024-03-04T08:56:30.648059+0000","last_became_peered":"2024-03-04T08:56:30.648059+0000","last_unstale":"2024-03-04T08:57:23.816274+0000","last_undegraded":"2024-03-04T08:57:23.816274+0000","last_fullsized":"2024-03-04T08:57:23.816274+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:59:07.571168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.819082+0000","last_change":"2024-03-04T08:56:34.673401+0000","last_active":"2024-03-04T08:57:23.819082+0000","last_peered":"2024-03-04T08:57:23.819082+0000","last_clean":"2024-03-04T08:57:23.819082+0000","last_became_active":"2024-03-04T08:56:34.672596+0000","last_became_peered":"2024-03-04T08:56:34.672596+0000","last_unstale":"2024-03-04T08:57:23.819082+0000","last_undegraded":"2024-03-04T08:57:23.819082+0000","last_fullsized":"2024-03-04T08:57:23.819082+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:33.643752+0000","last_clean_scrub_stamp":"2024-03-04T08:56:33.643752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:39:56.613366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813351+0000","last_change":"2024-03-04T08:56:29.988331+0000","last_active":"2024-03-04T08:57:23.813351+0000","last_peered":"2024-03-04T08:57:23.813351+0000","last_clean":"2024-03-04T08:57:23.813351+0000","last_became_active":"2024-03-04T08:56:29.988009+0000","last_became_peered":"2024-03-04T08:56:29.988009+0000","last_unstale":"2024-03-04T08:57:23.813351+0000","last_undegraded":"2024-03-04T08:57:23.813351+0000","last_fullsized":"2024-03-04T08:57:23.813351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:28.961968+0000","last_clean_scrub_stamp":"2024-03-04T08:56:28.961968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:50:43.693924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.816655+0000","last_change":"2024-03-04T08:56:30.647443+0000","last_active":"2024-03-04T08:57:23.816655+0000","last_peered":"2024-03-04T08:57:23.816655+0000","last_clean":"2024-03-04T08:57:23.816655+0000","last_became_active":"2024-03-04T08:56:30.647106+0000","last_became_peered":"2024-03-04T08:56:30.647106+0000","last_unstale":"2024-03-04T08:57:23.816655+0000","last_undegraded":"2024-03-04T08:57:23.816655+0000","last_fullsized":"2024-03-04T08:57:23.816655+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:09:56.320806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813326+0000","last_change":"2024-03-04T08:56:30.651243+0000","last_active":"2024-03-04T08:57:23.813326+0000","last_peered":"2024-03-04T08:57:23.813326+0000","last_clean":"2024-03-04T08:57:23.813326+0000","last_became_active":"2024-03-04T08:56:30.650781+0000","last_became_peered":"2024-03-04T08:56:30.650781+0000","last_unstale":"2024-03-04T08:57:23.813326+0000","last_undegraded":"2024-03-04T08:57:23.813326+0000","last_fullsized":"2024-03-04T08:57:23.813326+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:58:16.723542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.811680+0000","last_change":"2024-03-04T08:56:30.645737+0000","last_active":"2024-03-04T08:57:23.811680+0000","last_peered":"2024-03-04T08:57:23.811680+0000","last_clean":"2024-03-04T08:57:23.811680+0000","last_became_active":"2024-03-04T08:56:30.645293+0000","last_became_peered":"2024-03-04T08:56:30.645293+0000","last_unstale":"2024-03-04T08:57:23.811680+0000","last_undegraded":"2024-03-04T08:57:23.811680+0000","last_fullsized":"2024-03-04T08:57:23.811680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:56:04.302166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.811096+0000","last_change":"2024-03-04T08:56:30.649851+0000","last_active":"2024-03-04T08:57:23.811096+0000","last_peered":"2024-03-04T08:57:23.811096+0000","last_clean":"2024-03-04T08:57:23.811096+0000","last_became_active":"2024-03-04T08:56:30.649528+0000","last_became_peered":"2024-03-04T08:56:30.649528+0000","last_unstale":"2024-03-04T08:57:23.811096+0000","last_undegraded":"2024-03-04T08:57:23.811096+0000","last_fullsized":"2024-03-04T08:57:23.811096+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:36:50.431866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.817217+0000","last_change":"2024-03-04T08:56:30.646893+0000","last_active":"2024-03-04T08:57:23.817217+0000","last_peered":"2024-03-04T08:57:23.817217+0000","last_clean":"2024-03-04T08:57:23.817217+0000","last_became_active":"2024-03-04T08:56:30.646615+0000","last_became_peered":"2024-03-04T08:56:30.646615+0000","last_unstale":"2024-03-04T08:57:23.817217+0000","last_undegraded":"2024-03-04T08:57:23.817217+0000","last_fullsized":"2024-03-04T08:57:23.817217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:07:12.584686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813183+0000","last_change":"2024-03-04T08:56:30.649772+0000","last_active":"2024-03-04T08:57:23.813183+0000","last_peered":"2024-03-04T08:57:23.813183+0000","last_clean":"2024-03-04T08:57:23.813183+0000","last_became_active":"2024-03-04T08:56:30.649480+0000","last_became_peered":"2024-03-04T08:56:30.649480+0000","last_unstale":"2024-03-04T08:57:23.813183+0000","last_undegraded":"2024-03-04T08:57:23.813183+0000","last_fullsized":"2024-03-04T08:57:23.813183+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:14:10.553290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.819863+0000","last_change":"2024-03-04T08:57:23.819863+0000","last_active":"2024-03-04T08:57:23.819863+0000","last_peered":"2024-03-04T08:57:23.819863+0000","last_clean":"2024-03-04T08:57:23.819863+0000","last_became_active":"2024-03-04T08:57:23.818397+0000","last_became_peered":"2024-03-04T08:57:23.818397+0000","last_unstale":"2024-03-04T08:57:23.819863+0000","last_undegraded":"2024-03-04T08:57:23.819863+0000","last_fullsized":"2024-03-04T08:57:23.819863+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:47:26.005195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.811688+0000","last_change":"2024-03-04T08:56:30.646870+0000","last_active":"2024-03-04T08:57:23.811688+0000","last_peered":"2024-03-04T08:57:23.811688+0000","last_clean":"2024-03-04T08:57:23.811688+0000","last_became_active":"2024-03-04T08:56:30.646369+0000","last_became_peered":"2024-03-04T08:56:30.646369+0000","last_unstale":"2024-03-04T08:57:23.811688+0000","last_undegraded":"2024-03-04T08:57:23.811688+0000","last_fullsized":"2024-03-04T08:57:23.811688+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:25:34.969089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.160860+0000","last_change":"2024-03-04T08:56:30.640854+0000","last_active":"2024-03-04T08:57:24.160860+0000","last_peered":"2024-03-04T08:57:24.160860+0000","last_clean":"2024-03-04T08:57:24.160860+0000","last_became_active":"2024-03-04T08:56:30.640484+0000","last_became_peered":"2024-03-04T08:56:30.640484+0000","last_unstale":"2024-03-04T08:57:24.160860+0000","last_undegraded":"2024-03-04T08:57:24.160860+0000","last_fullsized":"2024-03-04T08:57:24.160860+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:16:01.298720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.819465+0000","last_change":"2024-03-04T08:56:30.648593+0000","last_active":"2024-03-04T08:57:23.819465+0000","last_peered":"2024-03-04T08:57:23.819465+0000","last_clean":"2024-03-04T08:57:23.819465+0000","last_became_active":"2024-03-04T08:56:30.648287+0000","last_became_peered":"2024-03-04T08:56:30.648287+0000","last_unstale":"2024-03-04T08:57:23.819465+0000","last_undegraded":"2024-03-04T08:57:23.819465+0000","last_fullsized":"2024-03-04T08:57:23.819465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:04:52.525357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:24.160999+0000","last_change":"2024-03-04T08:56:30.640284+0000","last_active":"2024-03-04T08:57:24.160999+0000","last_peered":"2024-03-04T08:57:24.160999+0000","last_clean":"2024-03-04T08:57:24.160999+0000","last_became_active":"2024-03-04T08:56:30.639968+0000","last_became_peered":"2024-03-04T08:56:30.639968+0000","last_unstale":"2024-03-04T08:57:24.160999+0000","last_undegraded":"2024-03-04T08:57:24.160999+0000","last_fullsized":"2024-03-04T08:57:24.160999+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:37:26.063318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813636+0000","last_change":"2024-03-04T08:56:30.649547+0000","last_active":"2024-03-04T08:57:23.813636+0000","last_peered":"2024-03-04T08:57:23.813636+0000","last_clean":"2024-03-04T08:57:23.813636+0000","last_became_active":"2024-03-04T08:56:30.649170+0000","last_became_peered":"2024-03-04T08:56:30.649170+0000","last_unstale":"2024-03-04T08:57:23.813636+0000","last_undegraded":"2024-03-04T08:57:23.813636+0000","last_fullsized":"2024-03-04T08:57:23.813636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:02:22.805496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813013+0000","last_change":"2024-03-04T08:56:30.650892+0000","last_active":"2024-03-04T08:57:23.813013+0000","last_peered":"2024-03-04T08:57:23.813013+0000","last_clean":"2024-03-04T08:57:23.813013+0000","last_became_active":"2024-03-04T08:56:30.650442+0000","last_became_peered":"2024-03-04T08:56:30.650442+0000","last_unstale":"2024-03-04T08:57:23.813013+0000","last_undegraded":"2024-03-04T08:57:23.813013+0000","last_fullsized":"2024-03-04T08:57:23.813013+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:40:38.217900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-03-04T08:57:23.813415+0000","last_change":"2024-03-04T08:56:30.647282+0000","last_active":"2024-03-04T08:57:23.813415+0000","last_peered":"2024-03-04T08:57:23.813415+0000","last_clean":"2024-03-04T08:57:23.813415+0000","last_became_active":"2024-03-04T08:56:30.646916+0000","last_became_peered":"2024-03-04T08:56:30.646916+0000","last_unstale":"2024-03-04T08:57:23.813415+0000","last_undegraded":"2024-03-04T08:57:23.813415+0000","last_fullsized":"2024-03-04T08:57:23.813415+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T08:56:29.966896+0000","last_clean_scrub_stamp":"2024-03-04T08:56:29.966896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:36:11.714223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64154,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":147,"num_read_kb":138,"num_write":79,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":63356,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":192462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318307,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":509082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579943,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1716224,"data_stored":1158904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743469,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":513760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005111,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":509686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299449,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1642496,"data_stored":1104128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462974,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":504986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1110016,"data_stored":568342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":505076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":505020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149776,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":505090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888139,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":572460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724614,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1646592,"data_stored":1099632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":63356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":63378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T08:58:11.580 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-04T08:58:11.580 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-04T08:58:11.580 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-04T08:58:11.580 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph health --format=json 2024-03-04T08:58:11.825 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:58:12.075 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:11 smithi071 ceph-mon[32789]: pgmap v377: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:12.075 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:11 smithi071 ceph-mon[32789]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T08:58:12.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:11 smithi150 ceph-mon[36807]: pgmap v377: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:12.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:11 smithi150 ceph-mon[36807]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T08:58:12.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:11 smithi158 ceph-mon[37241]: pgmap v377: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:12.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:11 smithi158 ceph-mon[37241]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T08:58:13.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.594+0000 7f38e7829700 1 -- 172.21.15.71:0/891172287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38e0102110 msgr2=0x7f38e0102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:13.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.594+0000 7f38e7829700 1 --2- 172.21.15.71:0/891172287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38e0102110 0x7f38e0102590 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f38d400b0d0 tx=0x7f38d400b3e0 comp rx=0 tx=0).stop 2024-03-04T08:58:13.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.594+0000 7f38e7829700 1 -- 172.21.15.71:0/891172287 shutdown_connections 2024-03-04T08:58:13.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.594+0000 7f38e7829700 1 --2- 172.21.15.71:0/891172287 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f38e01080f0 0x7f38e01084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:13.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.594+0000 7f38e7829700 1 --2- 172.21.15.71:0/891172287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e0102ad0 0x7f38e010ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:13.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.594+0000 7f38e7829700 1 --2- 172.21.15.71:0/891172287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38e0102110 0x7f38e0102590 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:13.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.594+0000 7f38e7829700 1 -- 172.21.15.71:0/891172287 >> 172.21.15.71:0/891172287 conn(0x7f38e00791f0 msgr2=0x7f38e0079600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:13.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.594+0000 7f38e7829700 1 -- 172.21.15.71:0/891172287 shutdown_connections 2024-03-04T08:58:13.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.594+0000 7f38e7829700 1 -- 172.21.15.71:0/891172287 wait complete. 2024-03-04T08:58:13.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.595+0000 7f38e7829700 1 Processor -- start 2024-03-04T08:58:13.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.595+0000 7f38e7829700 1 -- start start 2024-03-04T08:58:13.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.595+0000 7f38e7829700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f38e0102110 0x7f38e00743c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:13.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.595+0000 7f38e7829700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38e0102ad0 0x7f38e0074900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:13.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.595+0000 7f38e7829700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e01080f0 0x7f38e0074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:13.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.595+0000 7f38e7829700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e0112390 con 0x7f38e0102ad0 2024-03-04T08:58:13.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.595+0000 7f38e7829700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e0112210 con 0x7f38e01080f0 2024-03-04T08:58:13.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.595+0000 7f38e7829700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e0112510 con 0x7f38e0102110 2024-03-04T08:58:13.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.595+0000 7f38e4dc4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38e0102ad0 0x7f38e0074900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:13.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.595+0000 7f38e4dc4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38e0102ad0 0x7f38e0074900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43008/0 (socket says 172.21.15.71:43008) 2024-03-04T08:58:13.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.595+0000 7f38e4dc4700 1 -- 172.21.15.71:0/3781036602 learned_addr learned my addr 172.21.15.71:0/3781036602 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:13.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.595+0000 7f38e5dc6700 1 --2- 172.21.15.71:0/3781036602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e01080f0 0x7f38e0074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:13.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.595+0000 7f38e55c5700 1 --2- 172.21.15.71:0/3781036602 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f38e0102110 0x7f38e00743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:13.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.596+0000 7f38e4dc4700 1 -- 172.21.15.71:0/3781036602 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f38e0102110 msgr2=0x7f38e00743c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:13.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.596+0000 7f38e4dc4700 1 --2- 172.21.15.71:0/3781036602 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f38e0102110 0x7f38e00743c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:13.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.596+0000 7f38e4dc4700 1 -- 172.21.15.71:0/3781036602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e01080f0 msgr2=0x7f38e0074e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:13.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.596+0000 7f38e4dc4700 1 --2- 172.21.15.71:0/3781036602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e01080f0 0x7f38e0074e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:13.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.596+0000 7f38e4dc4700 1 -- 172.21.15.71:0/3781036602 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e01a6d00 con 0x7f38e0102ad0 2024-03-04T08:58:13.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.596+0000 7f38e4dc4700 1 --2- 172.21.15.71:0/3781036602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38e0102ad0 0x7f38e0074900 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f38d400d010 tx=0x7f38d4007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:13.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.596+0000 7f38de7fc700 1 -- 172.21.15.71:0/3781036602 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4021070 con 0x7f38e0102ad0 2024-03-04T08:58:13.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.596+0000 7f38de7fc700 1 -- 172.21.15.71:0/3781036602 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f38d401db30 con 0x7f38e0102ad0 2024-03-04T08:58:13.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.596+0000 7f38e7829700 1 -- 172.21.15.71:0/3781036602 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e01a6f90 con 0x7f38e0102ad0 2024-03-04T08:58:13.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.596+0000 7f38de7fc700 1 -- 172.21.15.71:0/3781036602 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d401bb00 con 0x7f38e0102ad0 2024-03-04T08:58:13.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.596+0000 7f38e7829700 1 -- 172.21.15.71:0/3781036602 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e01a73f0 con 0x7f38e0102ad0 2024-03-04T08:58:13.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.597+0000 7f38e7829700 1 -- 172.21.15.71:0/3781036602 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e0068d80 con 0x7f38e0102ad0 2024-03-04T08:58:13.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.598+0000 7f38de7fc700 1 -- 172.21.15.71:0/3781036602 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f38d4024070 con 0x7f38e0102ad0 2024-03-04T08:58:13.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.599+0000 7f38de7fc700 1 --2- 172.21.15.71:0/3781036602 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f38c80a1f20 0x7f38c80a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:13.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.599+0000 7f38e55c5700 1 --2- 172.21.15.71:0/3781036602 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f38c80a1f20 0x7f38c80a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:13.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.599+0000 7f38de7fc700 1 -- 172.21.15.71:0/3781036602 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f38d400f020 con 0x7f38e0102ad0 2024-03-04T08:58:13.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.599+0000 7f38e55c5700 1 --2- 172.21.15.71:0/3781036602 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f38c80a1f20 0x7f38c80a43e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f38d80045b0 tx=0x7f38d800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:13.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.603+0000 7f38de7fc700 1 -- 172.21.15.71:0/3781036602 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f38d402b020 con 0x7f38e0102ad0 2024-03-04T08:58:13.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.844+0000 7f38e7829700 1 -- 172.21.15.71:0/3781036602 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f38e002dad0 con 0x7f38e0102ad0 2024-03-04T08:58:13.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.845+0000 7f38de7fc700 1 -- 172.21.15.71:0/3781036602 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f38d409c670 con 0x7f38e0102ad0 2024-03-04T08:58:13.841 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:58:13.841 INFO:teuthology.orchestra.run.smithi071.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-04T08:58:13.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.847+0000 7f38e7829700 1 -- 172.21.15.71:0/3781036602 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f38c80a1f20 msgr2=0x7f38c80a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:13.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.847+0000 7f38e7829700 1 --2- 172.21.15.71:0/3781036602 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f38c80a1f20 0x7f38c80a43e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f38d80045b0 tx=0x7f38d800b3c0 comp rx=0 tx=0).stop 2024-03-04T08:58:13.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.847+0000 7f38e7829700 1 -- 172.21.15.71:0/3781036602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38e0102ad0 msgr2=0x7f38e0074900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:13.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.847+0000 7f38e7829700 1 --2- 172.21.15.71:0/3781036602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38e0102ad0 0x7f38e0074900 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f38d400d010 tx=0x7f38d4007a20 comp rx=0 tx=0).stop 2024-03-04T08:58:13.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.847+0000 7f38e7829700 1 -- 172.21.15.71:0/3781036602 shutdown_connections 2024-03-04T08:58:13.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.847+0000 7f38e7829700 1 --2- 172.21.15.71:0/3781036602 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f38c80a1f20 0x7f38c80a43e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:13.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.847+0000 7f38e7829700 1 --2- 172.21.15.71:0/3781036602 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f38e0102110 0x7f38e00743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:13.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.847+0000 7f38e7829700 1 --2- 172.21.15.71:0/3781036602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38e01080f0 0x7f38e0074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:13.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.847+0000 7f38e7829700 1 --2- 172.21.15.71:0/3781036602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38e0102ad0 0x7f38e0074900 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:13.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.847+0000 7f38e7829700 1 -- 172.21.15.71:0/3781036602 >> 172.21.15.71:0/3781036602 conn(0x7f38e00791f0 msgr2=0x7f38e0100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:13.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.847+0000 7f38e7829700 1 -- 172.21.15.71:0/3781036602 shutdown_connections 2024-03-04T08:58:13.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:13.847+0000 7f38e7829700 1 -- 172.21.15.71:0/3781036602 wait complete. 2024-03-04T08:58:14.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:13 smithi071 ceph-mon[32789]: pgmap v378: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:58:14.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:13 smithi150 ceph-mon[36807]: pgmap v378: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:58:14.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:13 smithi158 ceph-mon[37241]: pgmap v378: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:58:14.633 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-04T08:58:14.633 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T08:58:14.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.744+0000 7f6a29f13640 1 -- 172.21.15.71:0/3071487017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a24100780 msgr2=0x7f6a24100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:14.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.744+0000 7f6a29f13640 1 --2- 172.21.15.71:0/3071487017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a24100780 0x7f6a24100c00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009930 tx=0x7f6a1802bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:14.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.744+0000 7f6a29f13640 1 -- 172.21.15.71:0/3071487017 shutdown_connections 2024-03-04T08:58:14.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.744+0000 7f6a29f13640 1 --2- 172.21.15.71:0/3071487017 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a24101140 0x7f6a241089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:14.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.744+0000 7f6a29f13640 1 --2- 172.21.15.71:0/3071487017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a24100780 0x7f6a24100c00 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:14.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.744+0000 7f6a29f13640 1 --2- 172.21.15.71:0/3071487017 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a24101ae0 0x7f6a24101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:14.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.744+0000 7f6a29f13640 1 -- 172.21.15.71:0/3071487017 >> 172.21.15.71:0/3071487017 conn(0x7f6a240fb1f0 msgr2=0x7f6a240fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:14.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.744+0000 7f6a29f13640 1 -- 172.21.15.71:0/3071487017 shutdown_connections 2024-03-04T08:58:14.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.744+0000 7f6a29f13640 1 -- 172.21.15.71:0/3071487017 wait complete. 2024-03-04T08:58:14.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.745+0000 7f6a29f13640 1 Processor -- start 2024-03-04T08:58:14.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.745+0000 7f6a29f13640 1 -- start start 2024-03-04T08:58:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.745+0000 7f6a29f13640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a24100780 0x7f6a24100200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.745+0000 7f6a29f13640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a24101140 0x7f6a2419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.745+0000 7f6a29f13640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a24101ae0 0x7f6a2419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.745+0000 7f6a29f13640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2410c1f0 con 0x7f6a24100780 2024-03-04T08:58:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.745+0000 7f6a29f13640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2410c070 con 0x7f6a24101ae0 2024-03-04T08:58:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.745+0000 7f6a29f13640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2410c370 con 0x7f6a24101140 2024-03-04T08:58:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.745+0000 7f6a237fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a24100780 0x7f6a24100200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.745+0000 7f6a237fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a24100780 0x7f6a24100200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43024/0 (socket says 172.21.15.71:43024) 2024-03-04T08:58:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.745+0000 7f6a237fe640 1 -- 172.21.15.71:0/1260435157 learned_addr learned my addr 172.21.15.71:0/1260435157 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.745+0000 7f6a23fff640 1 --2- 172.21.15.71:0/1260435157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a24101ae0 0x7f6a2419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:14.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.745+0000 7f6a22ffd640 1 --2- 172.21.15.71:0/1260435157 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a24101140 0x7f6a2419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:14.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.745+0000 7f6a237fe640 1 -- 172.21.15.71:0/1260435157 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a24101140 msgr2=0x7f6a2419c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:14.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.745+0000 7f6a237fe640 1 --2- 172.21.15.71:0/1260435157 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a24101140 0x7f6a2419c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:14.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.745+0000 7f6a237fe640 1 -- 172.21.15.71:0/1260435157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a24101ae0 msgr2=0x7f6a2419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:14.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.745+0000 7f6a237fe640 1 --2- 172.21.15.71:0/1260435157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a24101ae0 0x7f6a2419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:14.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.745+0000 7f6a237fe640 1 -- 172.21.15.71:0/1260435157 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2419f5c0 con 0x7f6a24100780 2024-03-04T08:58:14.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.745+0000 7f6a237fe640 1 --2- 172.21.15.71:0/1260435157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a24100780 0x7f6a24100200 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400e9c0 tx=0x7f6a1400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:14.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.746+0000 7f6a20ff9640 1 -- 172.21.15.71:0/1260435157 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1400cd80 con 0x7f6a24100780 2024-03-04T08:58:14.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.746+0000 7f6a29f13640 1 -- 172.21.15.71:0/1260435157 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a2419f8b0 con 0x7f6a24100780 2024-03-04T08:58:14.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.746+0000 7f6a20ff9640 1 -- 172.21.15.71:0/1260435157 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a140035f0 con 0x7f6a24100780 2024-03-04T08:58:14.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.746+0000 7f6a20ff9640 1 -- 172.21.15.71:0/1260435157 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a14010690 con 0x7f6a24100780 2024-03-04T08:58:14.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.746+0000 7f6a29f13640 1 -- 172.21.15.71:0/1260435157 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a2419fdd0 con 0x7f6a24100780 2024-03-04T08:58:14.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.747+0000 7f6a29f13640 1 -- 172.21.15.71:0/1260435157 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a241078c0 con 0x7f6a24100780 2024-03-04T08:58:14.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.748+0000 7f6a20ff9640 1 -- 172.21.15.71:0/1260435157 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f6a140037a0 con 0x7f6a24100780 2024-03-04T08:58:14.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.748+0000 7f6a20ff9640 1 --2- 172.21.15.71:0/1260435157 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6a0c0a1e10 0x7f6a0c0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:14.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.748+0000 7f6a22ffd640 1 --2- 172.21.15.71:0/1260435157 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6a0c0a1e10 0x7f6a0c0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:14.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.748+0000 7f6a20ff9640 1 -- 172.21.15.71:0/1260435157 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f6a140ce6d0 con 0x7f6a24100780 2024-03-04T08:58:14.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.749+0000 7f6a22ffd640 1 --2- 172.21.15.71:0/1260435157 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6a0c0a1e10 0x7f6a0c0a42d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009900 tx=0x7f6a1802cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:14.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.753+0000 7f6a20ff9640 1 -- 172.21.15.71:0/1260435157 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6a1409b640 con 0x7f6a24100780 2024-03-04T08:58:14.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:14 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3781036602' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-04T08:58:14.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.930+0000 7f6a29f13640 1 -- 172.21.15.71:0/1260435157 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a2402d260 con 0x7f6a24100780 2024-03-04T08:58:14.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.932+0000 7f6a20ff9640 1 -- 172.21.15.71:0/1260435157 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24624 (secure 0 0 0) 0x7f6a1409af40 con 0x7f6a24100780 2024-03-04T08:58:14.928 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:58:14.928 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":93,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","created":"2024-03-04T08:47:28.546102+0000","modified":"2024-03-04T08:57:23.805294+0000","last_up_change":"2024-03-04T08:55:30.744961+0000","last_in_change":"2024-03-04T08:55:11.570990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T08:51:14.933487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T08:56:28.253074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T08:56:29.543622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T08:56:32.884220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7e43b47d-8888-4631-bc96-8a96fcf8cd46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6803","nonce":1173641530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6805","nonce":1173641530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6809","nonce":1173641530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6807","nonce":1173641530}]},"public_addr":"172.21.15.71:6803/1173641530","cluster_addr":"172.21.15.71:6805/1173641530","heartbeat_back_addr":"172.21.15.71:6809/1173641530","heartbeat_front_addr":"172.21.15.71:6807/1173641530","state":["exists","up"]},{"osd":1,"uuid":"1209f214-f2e2-4209-bf69-4528a43a2123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6801","nonce":3383948110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6803","nonce":3383948110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6807","nonce":3383948110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6805","nonce":3383948110}]},"public_addr":"172.21.15.150:6801/3383948110","cluster_addr":"172.21.15.150:6803/3383948110","heartbeat_back_addr":"172.21.15.150:6807/3383948110","heartbeat_front_addr":"172.21.15.150:6805/3383948110","state":["exists","up"]},{"osd":2,"uuid":"b88c3082-93a9-4c9b-9221-a7105a44c91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6801","nonce":586944144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6803","nonce":586944144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6807","nonce":586944144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6805","nonce":586944144}]},"public_addr":"172.21.15.158:6801/586944144","cluster_addr":"172.21.15.158:6803/586944144","heartbeat_back_addr":"172.21.15.158:6807/586944144","heartbeat_front_addr":"172.21.15.158:6805/586944144","state":["exists","up"]},{"osd":3,"uuid":"e4fc7723-fd73-40b2-b9d0-7f9bda282ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6811","nonce":1311903262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6813","nonce":1311903262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6817","nonce":1311903262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6815","nonce":1311903262}]},"public_addr":"172.21.15.71:6811/1311903262","cluster_addr":"172.21.15.71:6813/1311903262","heartbeat_back_addr":"172.21.15.71:6817/1311903262","heartbeat_front_addr":"172.21.15.71:6815/1311903262","state":["exists","up"]},{"osd":4,"uuid":"e7eb72ac-f066-4ccd-b2bf-22a6a26daf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6809","nonce":2300837573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6811","nonce":2300837573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6815","nonce":2300837573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6813","nonce":2300837573}]},"public_addr":"172.21.15.150:6809/2300837573","cluster_addr":"172.21.15.150:6811/2300837573","heartbeat_back_addr":"172.21.15.150:6815/2300837573","heartbeat_front_addr":"172.21.15.150:6813/2300837573","state":["exists","up"]},{"osd":5,"uuid":"9b295840-e104-4a66-9cc4-7020627c1765","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6809","nonce":1922694045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6811","nonce":1922694045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6815","nonce":1922694045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6813","nonce":1922694045}]},"public_addr":"172.21.15.158:6809/1922694045","cluster_addr":"172.21.15.158:6811/1922694045","heartbeat_back_addr":"172.21.15.158:6815/1922694045","heartbeat_front_addr":"172.21.15.158:6813/1922694045","state":["exists","up"]},{"osd":6,"uuid":"6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6819","nonce":3848842707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6821","nonce":3848842707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6825","nonce":3848842707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6823","nonce":3848842707}]},"public_addr":"172.21.15.71:6819/3848842707","cluster_addr":"172.21.15.71:6821/3848842707","heartbeat_back_addr":"172.21.15.71:6825/3848842707","heartbeat_front_addr":"172.21.15.71:6823/3848842707","state":["exists","up"]},{"osd":7,"uuid":"6160a40a-d30f-4569-949c-0afb1bb3d133","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6817","nonce":1328330305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6819","nonce":1328330305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6823","nonce":1328330305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6821","nonce":1328330305}]},"public_addr":"172.21.15.150:6817/1328330305","cluster_addr":"172.21.15.150:6819/1328330305","heartbeat_back_addr":"172.21.15.150:6823/1328330305","heartbeat_front_addr":"172.21.15.150:6821/1328330305","state":["exists","up"]},{"osd":8,"uuid":"95ed2341-84a4-465c-9948-ac667680ce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6817","nonce":2195304577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6819","nonce":2195304577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6823","nonce":2195304577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6821","nonce":2195304577}]},"public_addr":"172.21.15.158:6817/2195304577","cluster_addr":"172.21.15.158:6819/2195304577","heartbeat_back_addr":"172.21.15.158:6823/2195304577","heartbeat_front_addr":"172.21.15.158:6821/2195304577","state":["exists","up"]},{"osd":9,"uuid":"088f6419-63a0-4768-afae-dbb8a5f5a565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6827","nonce":1913779347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6829","nonce":1913779347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6833","nonce":1913779347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6831","nonce":1913779347}]},"public_addr":"172.21.15.71:6827/1913779347","cluster_addr":"172.21.15.71:6829/1913779347","heartbeat_back_addr":"172.21.15.71:6833/1913779347","heartbeat_front_addr":"172.21.15.71:6831/1913779347","state":["exists","up"]},{"osd":10,"uuid":"18c85e24-068d-4f1e-a3b8-f7915b363234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6825","nonce":3295420996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6827","nonce":3295420996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6831","nonce":3295420996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6829","nonce":3295420996}]},"public_addr":"172.21.15.150:6825/3295420996","cluster_addr":"172.21.15.150:6827/3295420996","heartbeat_back_addr":"172.21.15.150:6831/3295420996","heartbeat_front_addr":"172.21.15.150:6829/3295420996","state":["exists","up"]},{"osd":11,"uuid":"4f6b3ddf-c482-4740-b1fd-de836e047c2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6825","nonce":3332083865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6827","nonce":3332083865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6831","nonce":3332083865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6829","nonce":3332083865}]},"public_addr":"172.21.15.158:6825/3332083865","cluster_addr":"172.21.15.158:6827/3332083865","heartbeat_back_addr":"172.21.15.158:6831/3332083865","heartbeat_front_addr":"172.21.15.158:6829/3332083865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:05.997832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:29.314235+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:08.819465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:41.194275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:06.032479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:38.814498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:13.067960+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:38.568479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:03.416713+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:38.349479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:04.096945+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:29.425250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1287712358":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/3770452860":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/718743094":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/1904717532":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:6800/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6800/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/2862184313":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/3621754630":"2024-03-05T08:48:22.593931+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T08:58:14.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.934+0000 7f6a29f13640 1 -- 172.21.15.71:0/1260435157 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6a0c0a1e10 msgr2=0x7f6a0c0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:14.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.934+0000 7f6a29f13640 1 --2- 172.21.15.71:0/1260435157 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6a0c0a1e10 0x7f6a0c0a42d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009900 tx=0x7f6a1802cf90 comp rx=0 tx=0).stop 2024-03-04T08:58:14.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.934+0000 7f6a29f13640 1 -- 172.21.15.71:0/1260435157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a24100780 msgr2=0x7f6a24100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:14.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.934+0000 7f6a29f13640 1 --2- 172.21.15.71:0/1260435157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a24100780 0x7f6a24100200 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400e9c0 tx=0x7f6a1400ee80 comp rx=0 tx=0).stop 2024-03-04T08:58:14.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.934+0000 7f6a29f13640 1 -- 172.21.15.71:0/1260435157 shutdown_connections 2024-03-04T08:58:14.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.934+0000 7f6a29f13640 1 --2- 172.21.15.71:0/1260435157 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6a0c0a1e10 0x7f6a0c0a42d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:14.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.934+0000 7f6a29f13640 1 --2- 172.21.15.71:0/1260435157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a24101ae0 0x7f6a2419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:14.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.934+0000 7f6a29f13640 1 --2- 172.21.15.71:0/1260435157 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a24101140 0x7f6a2419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:14.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.934+0000 7f6a29f13640 1 --2- 172.21.15.71:0/1260435157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a24100780 0x7f6a24100200 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:14.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.934+0000 7f6a29f13640 1 -- 172.21.15.71:0/1260435157 >> 172.21.15.71:0/1260435157 conn(0x7f6a240fb1f0 msgr2=0x7f6a24104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:14.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.934+0000 7f6a29f13640 1 -- 172.21.15.71:0/1260435157 shutdown_connections 2024-03-04T08:58:14.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:14.934+0000 7f6a29f13640 1 -- 172.21.15.71:0/1260435157 wait complete. 2024-03-04T08:58:14.941 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T08:51:14.933487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T08:56:28.253074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T08:56:29.543622+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T08:56:32.884220+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T08:58:14.942 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T08:58:15.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.051+0000 7f4fd35d0640 1 -- 172.21.15.71:0/643823060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4fcc1027f0 msgr2=0x7f4fcc102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:15.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.051+0000 7f4fd35d0640 1 --2- 172.21.15.71:0/643823060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4fcc1027f0 0x7f4fcc102c70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009900 tx=0x7f4fc802bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:15.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.051+0000 7f4fd35d0640 1 -- 172.21.15.71:0/643823060 shutdown_connections 2024-03-04T08:58:15.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.051+0000 7f4fd35d0640 1 --2- 172.21.15.71:0/643823060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fcc1031b0 0x7f4fcc109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.051+0000 7f4fd35d0640 1 --2- 172.21.15.71:0/643823060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4fcc1027f0 0x7f4fcc102c70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.051+0000 7f4fd35d0640 1 --2- 172.21.15.71:0/643823060 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4fcc101600 0x7f4fcc101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.051+0000 7f4fd35d0640 1 -- 172.21.15.71:0/643823060 >> 172.21.15.71:0/643823060 conn(0x7f4fcc0fcb70 msgr2=0x7f4fcc0fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:15.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.052+0000 7f4fd35d0640 1 -- 172.21.15.71:0/643823060 shutdown_connections 2024-03-04T08:58:15.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.052+0000 7f4fd35d0640 1 -- 172.21.15.71:0/643823060 wait complete. 2024-03-04T08:58:15.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.052+0000 7f4fd35d0640 1 Processor -- start 2024-03-04T08:58:15.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.052+0000 7f4fd35d0640 1 -- start start 2024-03-04T08:58:15.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.052+0000 7f4fd35d0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fcc101600 0x7f4fcc1978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:15.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.052+0000 7f4fd35d0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4fcc1027f0 0x7f4fcc197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:15.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.052+0000 7f4fd35d0640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4fcc1031b0 0x7f4fcc19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:15.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.052+0000 7f4fd35d0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc10d280 con 0x7f4fcc1027f0 2024-03-04T08:58:15.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.052+0000 7f4fd35d0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc10d100 con 0x7f4fcc101600 2024-03-04T08:58:15.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.052+0000 7f4fd35d0640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc10d400 con 0x7f4fcc1031b0 2024-03-04T08:58:15.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.052+0000 7f4fd0b6b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4fcc1027f0 0x7f4fcc197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:15.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.052+0000 7f4fd0b6b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4fcc1027f0 0x7f4fcc197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43040/0 (socket says 172.21.15.71:43040) 2024-03-04T08:58:15.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.052+0000 7f4fd0b6b640 1 -- 172.21.15.71:0/283122072 learned_addr learned my addr 172.21.15.71:0/283122072 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:15.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.052+0000 7f4fd136c640 1 --2- 172.21.15.71:0/283122072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fcc101600 0x7f4fcc1978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:15.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.053+0000 7f4fd1b6d640 1 --2- 172.21.15.71:0/283122072 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4fcc1031b0 0x7f4fcc19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.053+0000 7f4fd0b6b640 1 -- 172.21.15.71:0/283122072 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4fcc1031b0 msgr2=0x7f4fcc19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.053+0000 7f4fd0b6b640 1 --2- 172.21.15.71:0/283122072 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4fcc1031b0 0x7f4fcc19eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.053+0000 7f4fd0b6b640 1 -- 172.21.15.71:0/283122072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fcc101600 msgr2=0x7f4fcc1978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.053+0000 7f4fd0b6b640 1 --2- 172.21.15.71:0/283122072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fcc101600 0x7f4fcc1978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.053+0000 7f4fd0b6b640 1 -- 172.21.15.71:0/283122072 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc19f510 con 0x7f4fcc1027f0 2024-03-04T08:58:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.053+0000 7f4fd0b6b640 1 --2- 172.21.15.71:0/283122072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4fcc1027f0 0x7f4fcc197e30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009230 tx=0x7f4fc802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.053+0000 7f4fc27fc640 1 -- 172.21.15.71:0/283122072 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8041070 con 0x7f4fcc1027f0 2024-03-04T08:58:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.053+0000 7f4fc27fc640 1 -- 172.21.15.71:0/283122072 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4fc8004450 con 0x7f4fcc1027f0 2024-03-04T08:58:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.053+0000 7f4fd35d0640 1 -- 172.21.15.71:0/283122072 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fcc19f7a0 con 0x7f4fcc1027f0 2024-03-04T08:58:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.053+0000 7f4fc27fc640 1 -- 172.21.15.71:0/283122072 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8003e20 con 0x7f4fcc1027f0 2024-03-04T08:58:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.053+0000 7f4fd35d0640 1 -- 172.21.15.71:0/283122072 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fcc19fce0 con 0x7f4fcc1027f0 2024-03-04T08:58:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.054+0000 7f4fd35d0640 1 -- 172.21.15.71:0/283122072 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fcc108920 con 0x7f4fcc1027f0 2024-03-04T08:58:15.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.055+0000 7f4fc27fc640 1 -- 172.21.15.71:0/283122072 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f4fc8044070 con 0x7f4fcc1027f0 2024-03-04T08:58:15.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.055+0000 7f4fc27fc640 1 --2- 172.21.15.71:0/283122072 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f4fb80a1e90 0x7f4fb80a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:15.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.056+0000 7f4fd136c640 1 --2- 172.21.15.71:0/283122072 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f4fb80a1e90 0x7f4fb80a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:15.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.056+0000 7f4fc27fc640 1 -- 172.21.15.71:0/283122072 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f4fc80efd30 con 0x7f4fcc1027f0 2024-03-04T08:58:15.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.056+0000 7f4fd136c640 1 --2- 172.21.15.71:0/283122072 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f4fb80a1e90 0x7f4fb80a4350 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc102650 tx=0x7f4fbc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:15.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.060+0000 7f4fc27fc640 1 -- 172.21.15.71:0/283122072 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4fc80f4070 con 0x7f4fcc1027f0 2024-03-04T08:58:15.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.233+0000 7f4fd35d0640 1 -- 172.21.15.71:0/283122072 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4fcc02d240 con 0x7f4fcc1027f0 2024-03-04T08:58:15.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.233+0000 7f4fc27fc640 1 -- 172.21.15.71:0/283122072 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4fc80dae20 con 0x7f4fcc1027f0 2024-03-04T08:58:15.229 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-03-04T08:58:15.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.235+0000 7f4fd35d0640 1 -- 172.21.15.71:0/283122072 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f4fb80a1e90 msgr2=0x7f4fb80a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:15.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.235+0000 7f4fd35d0640 1 --2- 172.21.15.71:0/283122072 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f4fb80a1e90 0x7f4fb80a4350 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc102650 tx=0x7f4fbc005f90 comp rx=0 tx=0).stop 2024-03-04T08:58:15.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.235+0000 7f4fd35d0640 1 -- 172.21.15.71:0/283122072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4fcc1027f0 msgr2=0x7f4fcc197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:15.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.235+0000 7f4fd35d0640 1 --2- 172.21.15.71:0/283122072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4fcc1027f0 0x7f4fcc197e30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009230 tx=0x7f4fc802c650 comp rx=0 tx=0).stop 2024-03-04T08:58:15.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.235+0000 7f4fd35d0640 1 -- 172.21.15.71:0/283122072 shutdown_connections 2024-03-04T08:58:15.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.235+0000 7f4fd35d0640 1 --2- 172.21.15.71:0/283122072 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f4fb80a1e90 0x7f4fb80a4350 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.235+0000 7f4fd35d0640 1 --2- 172.21.15.71:0/283122072 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4fcc1031b0 0x7f4fcc19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.235+0000 7f4fd35d0640 1 --2- 172.21.15.71:0/283122072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4fcc1027f0 0x7f4fcc197e30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.235+0000 7f4fd35d0640 1 --2- 172.21.15.71:0/283122072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4fcc101600 0x7f4fcc1978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.235+0000 7f4fd35d0640 1 -- 172.21.15.71:0/283122072 >> 172.21.15.71:0/283122072 conn(0x7f4fcc0fcb70 msgr2=0x7f4fcc105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:15.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.235+0000 7f4fd35d0640 1 -- 172.21.15.71:0/283122072 shutdown_connections 2024-03-04T08:58:15.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.235+0000 7f4fd35d0640 1 -- 172.21.15.71:0/283122072 wait complete. 2024-03-04T08:58:15.242 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T08:58:15.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:14 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3781036602' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-04T08:58:15.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:14 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3781036602' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-04T08:58:15.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.351+0000 7fa16e902640 1 -- 172.21.15.71:0/3301495358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1681015e0 msgr2=0x7fa168101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:15.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.351+0000 7fa16e902640 1 --2- 172.21.15.71:0/3301495358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1681015e0 0x7fa168101a00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa158009900 tx=0x7fa15802bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:15.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.351+0000 7fa16e902640 1 -- 172.21.15.71:0/3301495358 shutdown_connections 2024-03-04T08:58:15.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.351+0000 7fa16e902640 1 --2- 172.21.15.71:0/3301495358 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa168103190 0x7fa168109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.351+0000 7fa16e902640 1 --2- 172.21.15.71:0/3301495358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1681027d0 0x7fa168102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.351+0000 7fa16e902640 1 --2- 172.21.15.71:0/3301495358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1681015e0 0x7fa168101a00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.351+0000 7fa16e902640 1 -- 172.21.15.71:0/3301495358 >> 172.21.15.71:0/3301495358 conn(0x7fa1680fcb70 msgr2=0x7fa1680fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:15.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.351+0000 7fa16e902640 1 -- 172.21.15.71:0/3301495358 shutdown_connections 2024-03-04T08:58:15.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.351+0000 7fa16e902640 1 -- 172.21.15.71:0/3301495358 wait complete. 2024-03-04T08:58:15.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.351+0000 7fa16e902640 1 Processor -- start 2024-03-04T08:58:15.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.352+0000 7fa16e902640 1 -- start start 2024-03-04T08:58:15.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.352+0000 7fa16e902640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1681015e0 0x7fa168197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:15.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.352+0000 7fa16e902640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa1681027d0 0x7fa168197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:15.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.352+0000 7fa16e902640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa168103190 0x7fa16819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:15.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.352+0000 7fa16e902640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16810d230 con 0x7fa168103190 2024-03-04T08:58:15.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.352+0000 7fa16e902640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16810d0b0 con 0x7fa1681015e0 2024-03-04T08:58:15.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.352+0000 7fa16e902640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16810d3b0 con 0x7fa1681027d0 2024-03-04T08:58:15.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.352+0000 7fa16ce9f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa168103190 0x7fa16819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:15.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.352+0000 7fa16ce9f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa168103190 0x7fa16819eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43056/0 (socket says 172.21.15.71:43056) 2024-03-04T08:58:15.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.352+0000 7fa16ce9f640 1 -- 172.21.15.71:0/275999035 learned_addr learned my addr 172.21.15.71:0/275999035 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:15.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.352+0000 7fa167fff640 1 --2- 172.21.15.71:0/275999035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1681015e0 0x7fa168197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:15.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.352+0000 7fa1677fe640 1 --2- 172.21.15.71:0/275999035 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa1681027d0 0x7fa168197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:15.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.352+0000 7fa16ce9f640 1 -- 172.21.15.71:0/275999035 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa1681027d0 msgr2=0x7fa168197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:15.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.352+0000 7fa16ce9f640 1 --2- 172.21.15.71:0/275999035 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa1681027d0 0x7fa168197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.352+0000 7fa16ce9f640 1 -- 172.21.15.71:0/275999035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1681015e0 msgr2=0x7fa168197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:15.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.352+0000 7fa16ce9f640 1 --2- 172.21.15.71:0/275999035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1681015e0 0x7fa168197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.352+0000 7fa16ce9f640 1 -- 172.21.15.71:0/275999035 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16819f520 con 0x7fa168103190 2024-03-04T08:58:15.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.352+0000 7fa16ce9f640 1 --2- 172.21.15.71:0/275999035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa168103190 0x7fa16819eeb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa16000eaa0 tx=0x7fa16000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:15.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.353+0000 7fa1657fa640 1 -- 172.21.15.71:0/275999035 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16000cdb0 con 0x7fa168103190 2024-03-04T08:58:15.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.353+0000 7fa16e902640 1 -- 172.21.15.71:0/275999035 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16819f810 con 0x7fa168103190 2024-03-04T08:58:15.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.353+0000 7fa1657fa640 1 -- 172.21.15.71:0/275999035 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa160005020 con 0x7fa168103190 2024-03-04T08:58:15.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.353+0000 7fa1657fa640 1 -- 172.21.15.71:0/275999035 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1600116b0 con 0x7fa168103190 2024-03-04T08:58:15.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.353+0000 7fa16e902640 1 -- 172.21.15.71:0/275999035 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16819fd10 con 0x7fa168103190 2024-03-04T08:58:15.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.354+0000 7fa16e902640 1 -- 172.21.15.71:0/275999035 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa168108a20 con 0x7fa168103190 2024-03-04T08:58:15.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.355+0000 7fa1657fa640 1 -- 172.21.15.71:0/275999035 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fa160005560 con 0x7fa168103190 2024-03-04T08:58:15.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.355+0000 7fa1657fa640 1 --2- 172.21.15.71:0/275999035 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa1500a1e10 0x7fa1500a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:15.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.355+0000 7fa167fff640 1 --2- 172.21.15.71:0/275999035 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa1500a1e10 0x7fa1500a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:15.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.355+0000 7fa1657fa640 1 -- 172.21.15.71:0/275999035 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fa1600ce520 con 0x7fa168103190 2024-03-04T08:58:15.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.356+0000 7fa167fff640 1 --2- 172.21.15.71:0/275999035 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa1500a1e10 0x7fa1500a42d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa158000c00 tx=0x7fa158005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:15.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.360+0000 7fa1657fa640 1 -- 172.21.15.71:0/275999035 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa160016030 con 0x7fa168103190 2024-03-04T08:58:15.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.532+0000 7fa16e902640 1 -- 172.21.15.71:0/275999035 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa1680621d0 con 0x7fa168103190 2024-03-04T08:58:15.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.532+0000 7fa1657fa640 1 -- 172.21.15.71:0/275999035 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa16009b490 con 0x7fa168103190 2024-03-04T08:58:15.529 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-03-04T08:58:15.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.534+0000 7fa16e902640 1 -- 172.21.15.71:0/275999035 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa1500a1e10 msgr2=0x7fa1500a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:15.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.534+0000 7fa16e902640 1 --2- 172.21.15.71:0/275999035 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa1500a1e10 0x7fa1500a42d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa158000c00 tx=0x7fa158005fd0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.534+0000 7fa16e902640 1 -- 172.21.15.71:0/275999035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa168103190 msgr2=0x7fa16819eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:15.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.534+0000 7fa16e902640 1 --2- 172.21.15.71:0/275999035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa168103190 0x7fa16819eeb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa16000eaa0 tx=0x7fa16000ef60 comp rx=0 tx=0).stop 2024-03-04T08:58:15.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.534+0000 7fa16e902640 1 -- 172.21.15.71:0/275999035 shutdown_connections 2024-03-04T08:58:15.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.534+0000 7fa16e902640 1 --2- 172.21.15.71:0/275999035 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa1500a1e10 0x7fa1500a42d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.534+0000 7fa16e902640 1 --2- 172.21.15.71:0/275999035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa168103190 0x7fa16819eeb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.534+0000 7fa16e902640 1 --2- 172.21.15.71:0/275999035 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa1681027d0 0x7fa168197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.534+0000 7fa16e902640 1 --2- 172.21.15.71:0/275999035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1681015e0 0x7fa168197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.534+0000 7fa16e902640 1 -- 172.21.15.71:0/275999035 >> 172.21.15.71:0/275999035 conn(0x7fa1680fcb70 msgr2=0x7fa168105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:15.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.534+0000 7fa16e902640 1 -- 172.21.15.71:0/275999035 shutdown_connections 2024-03-04T08:58:15.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.534+0000 7fa16e902640 1 -- 172.21.15.71:0/275999035 wait complete. 2024-03-04T08:58:15.542 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T08:58:15.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.650+0000 7fa7ebe0e640 1 -- 172.21.15.71:0/2754595034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7e41016c0 msgr2=0x7fa7e4101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:15.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.650+0000 7fa7ebe0e640 1 --2- 172.21.15.71:0/2754595034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7e41016c0 0x7fa7e4101b40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4009900 tx=0x7fa7d402bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:15.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.650+0000 7fa7ebe0e640 1 -- 172.21.15.71:0/2754595034 shutdown_connections 2024-03-04T08:58:15.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.650+0000 7fa7ebe0e640 1 --2- 172.21.15.71:0/2754595034 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7e4102080 0x7fa7e4108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.650+0000 7fa7ebe0e640 1 --2- 172.21.15.71:0/2754595034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7e41016c0 0x7fa7e4101b40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.650+0000 7fa7ebe0e640 1 --2- 172.21.15.71:0/2754595034 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa7e41004d0 0x7fa7e41008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.650+0000 7fa7ebe0e640 1 -- 172.21.15.71:0/2754595034 >> 172.21.15.71:0/2754595034 conn(0x7fa7e40fba60 msgr2=0x7fa7e40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:15.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.651+0000 7fa7ebe0e640 1 -- 172.21.15.71:0/2754595034 shutdown_connections 2024-03-04T08:58:15.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.651+0000 7fa7ebe0e640 1 -- 172.21.15.71:0/2754595034 wait complete. 2024-03-04T08:58:15.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.651+0000 7fa7ebe0e640 1 Processor -- start 2024-03-04T08:58:15.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.651+0000 7fa7ebe0e640 1 -- start start 2024-03-04T08:58:15.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.651+0000 7fa7ebe0e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7e41004d0 0x7fa7e4197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:15.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.651+0000 7fa7ebe0e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7e41016c0 0x7fa7e4197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:15.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.651+0000 7fa7ebe0e640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa7e4102080 0x7fa7e419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:15.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.651+0000 7fa7ebe0e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e410d270 con 0x7fa7e41004d0 2024-03-04T08:58:15.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.651+0000 7fa7ebe0e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e410d0f0 con 0x7fa7e41016c0 2024-03-04T08:58:15.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.651+0000 7fa7ebe0e640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e410d3f0 con 0x7fa7e4102080 2024-03-04T08:58:15.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.651+0000 7fa7e9baa640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7e41004d0 0x7fa7e4197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:15.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.651+0000 7fa7e9baa640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7e41004d0 0x7fa7e4197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35918/0 (socket says 172.21.15.71:35918) 2024-03-04T08:58:15.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.651+0000 7fa7e9baa640 1 -- 172.21.15.71:0/3983922836 learned_addr learned my addr 172.21.15.71:0/3983922836 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:15.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.651+0000 7fa7ea3ab640 1 --2- 172.21.15.71:0/3983922836 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa7e4102080 0x7fa7e419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:15.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.651+0000 7fa7e93a9640 1 --2- 172.21.15.71:0/3983922836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7e41016c0 0x7fa7e4197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:15.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.652+0000 7fa7e9baa640 1 -- 172.21.15.71:0/3983922836 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa7e4102080 msgr2=0x7fa7e419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:15.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.652+0000 7fa7e9baa640 1 --2- 172.21.15.71:0/3983922836 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa7e4102080 0x7fa7e419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.652+0000 7fa7e9baa640 1 -- 172.21.15.71:0/3983922836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7e41016c0 msgr2=0x7fa7e4197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:15.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.652+0000 7fa7e9baa640 1 --2- 172.21.15.71:0/3983922836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7e41016c0 0x7fa7e4197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.652+0000 7fa7e9baa640 1 -- 172.21.15.71:0/3983922836 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e419f530 con 0x7fa7e41004d0 2024-03-04T08:58:15.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.652+0000 7fa7e9baa640 1 --2- 172.21.15.71:0/3983922836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7e41004d0 0x7fa7e4197910 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000eaa0 tx=0x7fa7e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:15.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.652+0000 7fa7daffd640 1 -- 172.21.15.71:0/3983922836 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0016070 con 0x7fa7e41004d0 2024-03-04T08:58:15.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.652+0000 7fa7daffd640 1 -- 172.21.15.71:0/3983922836 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7e0003e50 con 0x7fa7e41004d0 2024-03-04T08:58:15.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.652+0000 7fa7ebe0e640 1 -- 172.21.15.71:0/3983922836 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e419f820 con 0x7fa7e41004d0 2024-03-04T08:58:15.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.652+0000 7fa7daffd640 1 -- 172.21.15.71:0/3983922836 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e00116d0 con 0x7fa7e41004d0 2024-03-04T08:58:15.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.652+0000 7fa7ebe0e640 1 -- 172.21.15.71:0/3983922836 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e419fcf0 con 0x7fa7e41004d0 2024-03-04T08:58:15.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.653+0000 7fa7ebe0e640 1 -- 172.21.15.71:0/3983922836 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e41077f0 con 0x7fa7e41004d0 2024-03-04T08:58:15.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.654+0000 7fa7daffd640 1 -- 172.21.15.71:0/3983922836 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fa7e0019070 con 0x7fa7e41004d0 2024-03-04T08:58:15.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.655+0000 7fa7daffd640 1 --2- 172.21.15.71:0/3983922836 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa7d00a1e10 0x7fa7d00a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:15.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.655+0000 7fa7e93a9640 1 --2- 172.21.15.71:0/3983922836 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa7d00a1e10 0x7fa7d00a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:15.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.655+0000 7fa7daffd640 1 -- 172.21.15.71:0/3983922836 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fa7e00ce300 con 0x7fa7e41004d0 2024-03-04T08:58:15.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.655+0000 7fa7e93a9640 1 --2- 172.21.15.71:0/3983922836 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa7d00a1e10 0x7fa7d00a42d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa7d402c7b0 tx=0x7fa7d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:15.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.659+0000 7fa7daffd640 1 -- 172.21.15.71:0/3983922836 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa7e009ac20 con 0x7fa7e41004d0 2024-03-04T08:58:15.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.834+0000 7fa7ebe0e640 1 -- 172.21.15.71:0/3983922836 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa7e40621d0 con 0x7fa7e41004d0 2024-03-04T08:58:15.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.834+0000 7fa7daffd640 1 -- 172.21.15.71:0/3983922836 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa7e0085020 con 0x7fa7e41004d0 2024-03-04T08:58:15.830 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2024-03-04T08:58:15.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.836+0000 7fa7ebe0e640 1 -- 172.21.15.71:0/3983922836 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa7d00a1e10 msgr2=0x7fa7d00a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:15.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.836+0000 7fa7ebe0e640 1 --2- 172.21.15.71:0/3983922836 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa7d00a1e10 0x7fa7d00a42d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa7d402c7b0 tx=0x7fa7d4005fd0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.836+0000 7fa7ebe0e640 1 -- 172.21.15.71:0/3983922836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7e41004d0 msgr2=0x7fa7e4197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:15.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.836+0000 7fa7ebe0e640 1 --2- 172.21.15.71:0/3983922836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7e41004d0 0x7fa7e4197910 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000eaa0 tx=0x7fa7e000ef60 comp rx=0 tx=0).stop 2024-03-04T08:58:15.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.836+0000 7fa7ebe0e640 1 -- 172.21.15.71:0/3983922836 shutdown_connections 2024-03-04T08:58:15.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.836+0000 7fa7ebe0e640 1 --2- 172.21.15.71:0/3983922836 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fa7d00a1e10 0x7fa7d00a42d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.836+0000 7fa7ebe0e640 1 --2- 172.21.15.71:0/3983922836 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fa7e4102080 0x7fa7e419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.836+0000 7fa7ebe0e640 1 --2- 172.21.15.71:0/3983922836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7e41016c0 0x7fa7e4197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.836+0000 7fa7ebe0e640 1 --2- 172.21.15.71:0/3983922836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7e41004d0 0x7fa7e4197910 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.836+0000 7fa7ebe0e640 1 -- 172.21.15.71:0/3983922836 >> 172.21.15.71:0/3983922836 conn(0x7fa7e40fba60 msgr2=0x7fa7e41049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:15.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.836+0000 7fa7ebe0e640 1 -- 172.21.15.71:0/3983922836 shutdown_connections 2024-03-04T08:58:15.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.836+0000 7fa7ebe0e640 1 -- 172.21.15.71:0/3983922836 wait complete. 2024-03-04T08:58:15.844 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T08:58:15.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.953+0000 7f062444e640 1 -- 172.21.15.71:0/3049055620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f061c1004d0 msgr2=0x7f061c1008f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:15.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.953+0000 7f062444e640 1 --2- 172.21.15.71:0/3049055620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f061c1004d0 0x7f061c1008f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0610009900 tx=0x7f061002bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:15.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.953+0000 7f062444e640 1 -- 172.21.15.71:0/3049055620 shutdown_connections 2024-03-04T08:58:15.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.953+0000 7f062444e640 1 --2- 172.21.15.71:0/3049055620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f061c102080 0x7f061c108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.953+0000 7f062444e640 1 --2- 172.21.15.71:0/3049055620 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f061c1016c0 0x7f061c101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.953+0000 7f062444e640 1 --2- 172.21.15.71:0/3049055620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f061c1004d0 0x7f061c1008f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.953+0000 7f062444e640 1 -- 172.21.15.71:0/3049055620 >> 172.21.15.71:0/3049055620 conn(0x7f061c0fba60 msgr2=0x7f061c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:15.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.953+0000 7f062444e640 1 -- 172.21.15.71:0/3049055620 shutdown_connections 2024-03-04T08:58:15.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.953+0000 7f062444e640 1 -- 172.21.15.71:0/3049055620 wait complete. 2024-03-04T08:58:15.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.953+0000 7f062444e640 1 Processor -- start 2024-03-04T08:58:15.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.953+0000 7f062444e640 1 -- start start 2024-03-04T08:58:15.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.953+0000 7f062444e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f061c1004d0 0x7f061c197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:15.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.953+0000 7f062444e640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f061c1016c0 0x7f061c197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:15.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.953+0000 7f062444e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f061c102080 0x7f061c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:15.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.953+0000 7f062444e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c10d270 con 0x7f061c1004d0 2024-03-04T08:58:15.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.953+0000 7f062444e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c10d0f0 con 0x7f061c102080 2024-03-04T08:58:15.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.953+0000 7f062444e640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c10d3f0 con 0x7f061c1016c0 2024-03-04T08:58:15.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.954+0000 7f06221ea640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f061c1004d0 0x7f061c197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:15.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.954+0000 7f06221ea640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f061c1004d0 0x7f061c197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35928/0 (socket says 172.21.15.71:35928) 2024-03-04T08:58:15.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.954+0000 7f06221ea640 1 -- 172.21.15.71:0/2401435434 learned_addr learned my addr 172.21.15.71:0/2401435434 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:15.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.954+0000 7f06229eb640 1 --2- 172.21.15.71:0/2401435434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f061c102080 0x7f061c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:15.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.954+0000 7f06219e9640 1 --2- 172.21.15.71:0/2401435434 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f061c1016c0 0x7f061c197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:15.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.954+0000 7f06221ea640 1 -- 172.21.15.71:0/2401435434 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f061c1016c0 msgr2=0x7f061c197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:15.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.954+0000 7f06221ea640 1 --2- 172.21.15.71:0/2401435434 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f061c1016c0 0x7f061c197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.954+0000 7f06221ea640 1 -- 172.21.15.71:0/2401435434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f061c102080 msgr2=0x7f061c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:15.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.954+0000 7f06221ea640 1 --2- 172.21.15.71:0/2401435434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f061c102080 0x7f061c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:15.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.954+0000 7f06221ea640 1 -- 172.21.15.71:0/2401435434 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061c19f590 con 0x7f061c1004d0 2024-03-04T08:58:15.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.954+0000 7f06221ea640 1 --2- 172.21.15.71:0/2401435434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f061c1004d0 0x7f061c197970 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0610000c00 tx=0x7f0610005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:15.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.954+0000 7f060f7fe640 1 -- 172.21.15.71:0/2401435434 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0610041070 con 0x7f061c1004d0 2024-03-04T08:58:15.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.954+0000 7f062444e640 1 -- 172.21.15.71:0/2401435434 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061c19f820 con 0x7f061c1004d0 2024-03-04T08:58:15.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.954+0000 7f060f7fe640 1 -- 172.21.15.71:0/2401435434 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06100044d0 con 0x7f061c1004d0 2024-03-04T08:58:15.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.954+0000 7f060f7fe640 1 -- 172.21.15.71:0/2401435434 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0610034440 con 0x7f061c1004d0 2024-03-04T08:58:15.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.954+0000 7f062444e640 1 -- 172.21.15.71:0/2401435434 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061c19fd10 con 0x7f061c1004d0 2024-03-04T08:58:15.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.955+0000 7f062444e640 1 -- 172.21.15.71:0/2401435434 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061c107910 con 0x7f061c1004d0 2024-03-04T08:58:15.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.957+0000 7f060f7fe640 1 -- 172.21.15.71:0/2401435434 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f0610044070 con 0x7f061c1004d0 2024-03-04T08:58:15.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.957+0000 7f060f7fe640 1 --2- 172.21.15.71:0/2401435434 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f06080a1e10 0x7f06080a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:15.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.958+0000 7f06219e9640 1 --2- 172.21.15.71:0/2401435434 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f06080a1e10 0x7f06080a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:15.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.958+0000 7f060f7fe640 1 -- 172.21.15.71:0/2401435434 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f06100efec0 con 0x7f061c1004d0 2024-03-04T08:58:15.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.958+0000 7f06219e9640 1 --2- 172.21.15.71:0/2401435434 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f06080a1e10 0x7f06080a42d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f061c198e80 tx=0x7f061800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:15.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:15.961+0000 7f060f7fe640 1 -- 172.21.15.71:0/2401435434 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f06100bc8f0 con 0x7f061c1004d0 2024-03-04T08:58:16.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.135+0000 7f062444e640 1 -- 172.21.15.71:0/2401435434 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f061c0621d0 con 0x7f061c1004d0 2024-03-04T08:58:16.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.136+0000 7f060f7fe640 1 -- 172.21.15.71:0/2401435434 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f06100c0990 con 0x7f061c1004d0 2024-03-04T08:58:16.132 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-03-04T08:58:16.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.137+0000 7f062444e640 1 -- 172.21.15.71:0/2401435434 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f06080a1e10 msgr2=0x7f06080a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:16.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.137+0000 7f062444e640 1 --2- 172.21.15.71:0/2401435434 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f06080a1e10 0x7f06080a42d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f061c198e80 tx=0x7f061800b390 comp rx=0 tx=0).stop 2024-03-04T08:58:16.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.137+0000 7f062444e640 1 -- 172.21.15.71:0/2401435434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f061c1004d0 msgr2=0x7f061c197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:16.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.137+0000 7f062444e640 1 --2- 172.21.15.71:0/2401435434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f061c1004d0 0x7f061c197970 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0610000c00 tx=0x7f0610005d50 comp rx=0 tx=0).stop 2024-03-04T08:58:16.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.138+0000 7f062444e640 1 -- 172.21.15.71:0/2401435434 shutdown_connections 2024-03-04T08:58:16.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.138+0000 7f062444e640 1 --2- 172.21.15.71:0/2401435434 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f06080a1e10 0x7f06080a42d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.138+0000 7f062444e640 1 --2- 172.21.15.71:0/2401435434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f061c102080 0x7f061c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.138+0000 7f062444e640 1 --2- 172.21.15.71:0/2401435434 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f061c1016c0 0x7f061c197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.138+0000 7f062444e640 1 --2- 172.21.15.71:0/2401435434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f061c1004d0 0x7f061c197970 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.138+0000 7f062444e640 1 -- 172.21.15.71:0/2401435434 >> 172.21.15.71:0/2401435434 conn(0x7f061c0fba60 msgr2=0x7f061c104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:16.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.138+0000 7f062444e640 1 -- 172.21.15.71:0/2401435434 shutdown_connections 2024-03-04T08:58:16.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.138+0000 7f062444e640 1 -- 172.21.15.71:0/2401435434 wait complete. 2024-03-04T08:58:16.144 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:15 smithi071 ceph-mon[32789]: pgmap v379: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:16.144 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:15 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1260435157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:58:16.144 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:15 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/283122072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:58:16.144 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:15 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/275999035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T08:58:16.145 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:58:16.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.251+0000 7fb85dcc9640 1 -- 172.21.15.71:0/3308716915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8581016e0 msgr2=0x7fb858101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:16.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.251+0000 7fb85dcc9640 1 --2- 172.21.15.71:0/3308716915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8581016e0 0x7fb858101b60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb840009900 tx=0x7fb84002bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:16.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.251+0000 7fb85dcc9640 1 -- 172.21.15.71:0/3308716915 shutdown_connections 2024-03-04T08:58:16.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.251+0000 7fb85dcc9640 1 --2- 172.21.15.71:0/3308716915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8581020a0 0x7fb858108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.251+0000 7fb85dcc9640 1 --2- 172.21.15.71:0/3308716915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8581016e0 0x7fb858101b60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.251+0000 7fb85dcc9640 1 --2- 172.21.15.71:0/3308716915 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8581004f0 0x7fb858100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.251+0000 7fb85dcc9640 1 -- 172.21.15.71:0/3308716915 >> 172.21.15.71:0/3308716915 conn(0x7fb8580fba60 msgr2=0x7fb8580fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:16.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.251+0000 7fb85dcc9640 1 -- 172.21.15.71:0/3308716915 shutdown_connections 2024-03-04T08:58:16.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.251+0000 7fb85dcc9640 1 -- 172.21.15.71:0/3308716915 wait complete. 2024-03-04T08:58:16.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.251+0000 7fb85dcc9640 1 Processor -- start 2024-03-04T08:58:16.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.251+0000 7fb85dcc9640 1 -- start start 2024-03-04T08:58:16.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.252+0000 7fb85dcc9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8581004f0 0x7fb85806a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:16.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.252+0000 7fb85dcc9640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8581016e0 0x7fb85819c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:16.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.252+0000 7fb85dcc9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8581020a0 0x7fb85819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:16.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.252+0000 7fb85dcc9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85810c180 con 0x7fb8581004f0 2024-03-04T08:58:16.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.252+0000 7fb85dcc9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85810c000 con 0x7fb8581020a0 2024-03-04T08:58:16.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.252+0000 7fb85dcc9640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85810c300 con 0x7fb8581016e0 2024-03-04T08:58:16.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.252+0000 7fb8577fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8581004f0 0x7fb85806a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:16.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.252+0000 7fb8577fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8581004f0 0x7fb85806a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35954/0 (socket says 172.21.15.71:35954) 2024-03-04T08:58:16.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.252+0000 7fb8577fe640 1 -- 172.21.15.71:0/3314571234 learned_addr learned my addr 172.21.15.71:0/3314571234 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:16.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.252+0000 7fb857fff640 1 --2- 172.21.15.71:0/3314571234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8581020a0 0x7fb85819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:16.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.252+0000 7fb84effd640 1 --2- 172.21.15.71:0/3314571234 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8581016e0 0x7fb85819c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:16.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.252+0000 7fb8577fe640 1 -- 172.21.15.71:0/3314571234 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8581016e0 msgr2=0x7fb85819c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:16.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.252+0000 7fb8577fe640 1 --2- 172.21.15.71:0/3314571234 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8581016e0 0x7fb85819c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.252+0000 7fb8577fe640 1 -- 172.21.15.71:0/3314571234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8581020a0 msgr2=0x7fb85819ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:16.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.252+0000 7fb8577fe640 1 --2- 172.21.15.71:0/3314571234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8581020a0 0x7fb85819ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.252+0000 7fb8577fe640 1 -- 172.21.15.71:0/3314571234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85819f520 con 0x7fb8581004f0 2024-03-04T08:58:16.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.252+0000 7fb84effd640 1 --2- 172.21.15.71:0/3314571234 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8581016e0 0x7fb85819c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:58:16.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.252+0000 7fb8577fe640 1 --2- 172.21.15.71:0/3314571234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8581004f0 0x7fb85806a810 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb84800d950 tx=0x7fb84800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:16.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.252+0000 7fb8557fa640 1 -- 172.21.15.71:0/3314571234 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb848017070 con 0x7fb8581004f0 2024-03-04T08:58:16.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.252+0000 7fb85dcc9640 1 -- 172.21.15.71:0/3314571234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb85819f810 con 0x7fb8581004f0 2024-03-04T08:58:16.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.252+0000 7fb8557fa640 1 -- 172.21.15.71:0/3314571234 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb84800be10 con 0x7fb8581004f0 2024-03-04T08:58:16.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.252+0000 7fb8557fa640 1 -- 172.21.15.71:0/3314571234 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84800f620 con 0x7fb8581004f0 2024-03-04T08:58:16.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.252+0000 7fb85dcc9640 1 -- 172.21.15.71:0/3314571234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb85819fd30 con 0x7fb8581004f0 2024-03-04T08:58:16.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.253+0000 7fb85dcc9640 1 -- 172.21.15.71:0/3314571234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb858107830 con 0x7fb8581004f0 2024-03-04T08:58:16.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.254+0000 7fb8557fa640 1 -- 172.21.15.71:0/3314571234 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fb84801a070 con 0x7fb8581004f0 2024-03-04T08:58:16.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.255+0000 7fb8557fa640 1 --2- 172.21.15.71:0/3314571234 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb8440a1e10 0x7fb8440a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:16.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.255+0000 7fb84effd640 1 --2- 172.21.15.71:0/3314571234 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb8440a1e10 0x7fb8440a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:16.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.255+0000 7fb8557fa640 1 -- 172.21.15.71:0/3314571234 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fb8480cdd50 con 0x7fb8581004f0 2024-03-04T08:58:16.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.255+0000 7fb84effd640 1 --2- 172.21.15.71:0/3314571234 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb8440a1e10 0x7fb8440a42d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb840006010 tx=0x7fb84002c8a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:16.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.258+0000 7fb8557fa640 1 -- 172.21.15.71:0/3314571234 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb84809acc0 con 0x7fb8581004f0 2024-03-04T08:58:16.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:15 smithi150 ceph-mon[36807]: pgmap v379: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:16.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:15 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1260435157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:58:16.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:15 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/283122072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:58:16.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:15 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/275999035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T08:58:16.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:15 smithi158 ceph-mon[37241]: pgmap v379: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:16.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:15 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1260435157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:58:16.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:15 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/283122072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:58:16.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:15 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/275999035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T08:58:16.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.475+0000 7fb85dcc9640 1 -- 172.21.15.71:0/3314571234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb85802cff0 con 0x7fb8581004f0 2024-03-04T08:58:16.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.475+0000 7fb8557fa640 1 -- 172.21.15.71:0/3314571234 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11339 (secure 0 0 0) 0x7fb84809a5c0 con 0x7fb8581004f0 2024-03-04T08:58:16.472 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:58:16.473 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3975379807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3975379807},{"type":"v1","addr":"172.21.15.71:6835","nonce":3975379807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1962929347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1962929347},{"type":"v1","addr":"172.21.15.71:6837","nonce":1962929347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/1459252176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1459252176},{"type":"v1","addr":"172.21.15.71:6839","nonce":1459252176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6841/1083546600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":1083546600},{"type":"v1","addr":"172.21.15.71:6841","nonce":1083546600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/178179528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":178179528},{"type":"v1","addr":"172.21.15.158:6833","nonce":178179528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2468877675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2468877675},{"type":"v1","addr":"172.21.15.150:6833","nonce":2468877675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1652094309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1652094309},{"type":"v1","addr":"172.21.15.158:6835","nonce":1652094309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1802416462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1802416462},{"type":"v1","addr":"172.21.15.150:6835","nonce":1802416462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/617058380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":617058380},{"type":"v1","addr":"172.21.15.158:6837","nonce":617058380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/913465600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":913465600},{"type":"v1","addr":"172.21.15.150:6837","nonce":913465600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T08:56:30.343357+0000","modified":"2024-03-04T08:56:54.901214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.158:6839/2921704804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6838","nonce":2921704804},{"type":"v1","addr":"172.21.15.158:6839","nonce":2921704804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/2639278083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2639278083},{"type":"v1","addr":"172.21.15.150:6839","nonce":2639278083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491]},"id":1}]} 2024-03-04T08:58:16.473 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 20 2024-03-04T08:58:16.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.478+0000 7fb85dcc9640 1 -- 172.21.15.71:0/3314571234 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb8440a1e10 msgr2=0x7fb8440a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:16.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.478+0000 7fb85dcc9640 1 --2- 172.21.15.71:0/3314571234 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb8440a1e10 0x7fb8440a42d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb840006010 tx=0x7fb84002c8a0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.478+0000 7fb85dcc9640 1 -- 172.21.15.71:0/3314571234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8581004f0 msgr2=0x7fb85806a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:16.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.478+0000 7fb85dcc9640 1 --2- 172.21.15.71:0/3314571234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8581004f0 0x7fb85806a810 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb84800d950 tx=0x7fb84800dd50 comp rx=0 tx=0).stop 2024-03-04T08:58:16.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.478+0000 7fb85dcc9640 1 -- 172.21.15.71:0/3314571234 shutdown_connections 2024-03-04T08:58:16.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.478+0000 7fb85dcc9640 1 --2- 172.21.15.71:0/3314571234 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fb8440a1e10 0x7fb8440a42d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.478+0000 7fb85dcc9640 1 --2- 172.21.15.71:0/3314571234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8581020a0 0x7fb85819ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.478+0000 7fb85dcc9640 1 --2- 172.21.15.71:0/3314571234 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8581016e0 0x7fb85819c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.478+0000 7fb85dcc9640 1 --2- 172.21.15.71:0/3314571234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8581004f0 0x7fb85806a810 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.478+0000 7fb85dcc9640 1 -- 172.21.15.71:0/3314571234 >> 172.21.15.71:0/3314571234 conn(0x7fb8580fba60 msgr2=0x7fb858104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:16.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.478+0000 7fb85dcc9640 1 -- 172.21.15.71:0/3314571234 shutdown_connections 2024-03-04T08:58:16.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.478+0000 7fb85dcc9640 1 -- 172.21.15.71:0/3314571234 wait complete. 2024-03-04T08:58:16.486 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T08:58:16.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.591+0000 7f0707dc9640 1 -- 172.21.15.71:0/3068124367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07001020a0 msgr2=0x7f0700108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:16.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.591+0000 7f0707dc9640 1 --2- 172.21.15.71:0/3068124367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07001020a0 0x7f0700108930 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009ca0 tx=0x7f06f802c0f0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.591+0000 7f0707dc9640 1 -- 172.21.15.71:0/3068124367 shutdown_connections 2024-03-04T08:58:16.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.591+0000 7f0707dc9640 1 --2- 172.21.15.71:0/3068124367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07001020a0 0x7f0700108930 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.591+0000 7f0707dc9640 1 --2- 172.21.15.71:0/3068124367 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f07001016e0 0x7f0700101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.591+0000 7f0707dc9640 1 --2- 172.21.15.71:0/3068124367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07001004f0 0x7f0700100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.591+0000 7f0707dc9640 1 -- 172.21.15.71:0/3068124367 >> 172.21.15.71:0/3068124367 conn(0x7f07000fba60 msgr2=0x7f07000fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:16.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.591+0000 7f0707dc9640 1 -- 172.21.15.71:0/3068124367 shutdown_connections 2024-03-04T08:58:16.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.591+0000 7f0707dc9640 1 -- 172.21.15.71:0/3068124367 wait complete. 2024-03-04T08:58:16.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.592+0000 7f0707dc9640 1 Processor -- start 2024-03-04T08:58:16.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.592+0000 7f0707dc9640 1 -- start start 2024-03-04T08:58:16.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.592+0000 7f0707dc9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07001004f0 0x7f0700197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:16.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.592+0000 7f0707dc9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07001016e0 0x7f0700197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:16.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.592+0000 7f0707dc9640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f07001020a0 0x7f070019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:16.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.592+0000 7f0707dc9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070010c180 con 0x7f07001016e0 2024-03-04T08:58:16.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.592+0000 7f0707dc9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070010c000 con 0x7f07001004f0 2024-03-04T08:58:16.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.592+0000 7f0707dc9640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070010c300 con 0x7f07001020a0 2024-03-04T08:58:16.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.592+0000 7f0705364640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07001016e0 0x7f0700197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:16.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.592+0000 7f0705364640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07001016e0 0x7f0700197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35970/0 (socket says 172.21.15.71:35970) 2024-03-04T08:58:16.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.592+0000 7f0705364640 1 -- 172.21.15.71:0/1180705474 learned_addr learned my addr 172.21.15.71:0/1180705474 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:16.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.592+0000 7f0705b65640 1 --2- 172.21.15.71:0/1180705474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07001004f0 0x7f0700197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:16.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.592+0000 7f0706366640 1 --2- 172.21.15.71:0/1180705474 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f07001020a0 0x7f070019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:16.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.592+0000 7f0705364640 1 -- 172.21.15.71:0/1180705474 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f07001020a0 msgr2=0x7f070019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:16.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.592+0000 7f0705364640 1 --2- 172.21.15.71:0/1180705474 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f07001020a0 0x7f070019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.592+0000 7f0705364640 1 -- 172.21.15.71:0/1180705474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07001004f0 msgr2=0x7f0700197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:16.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.592+0000 7f0705364640 1 --2- 172.21.15.71:0/1180705474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07001004f0 0x7f0700197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.592+0000 7f0705364640 1 -- 172.21.15.71:0/1180705474 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070019f490 con 0x7f07001016e0 2024-03-04T08:58:16.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.592+0000 7f0706366640 1 --2- 172.21.15.71:0/1180705474 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f07001020a0 0x7f070019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:58:16.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.593+0000 7f0705364640 1 --2- 172.21.15.71:0/1180705474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07001016e0 0x7f0700197db0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f06f000d770 tx=0x7f06f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:16.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.593+0000 7f06f6ffd640 1 -- 172.21.15.71:0/1180705474 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f0017070 con 0x7f07001016e0 2024-03-04T08:58:16.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.593+0000 7f06f6ffd640 1 -- 172.21.15.71:0/1180705474 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06f0005500 con 0x7f07001016e0 2024-03-04T08:58:16.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.593+0000 7f0707dc9640 1 -- 172.21.15.71:0/1180705474 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f070019f780 con 0x7f07001016e0 2024-03-04T08:58:16.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.593+0000 7f06f6ffd640 1 -- 172.21.15.71:0/1180705474 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f0004e00 con 0x7f07001016e0 2024-03-04T08:58:16.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.593+0000 7f0707dc9640 1 -- 172.21.15.71:0/1180705474 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f070019fc80 con 0x7f07001016e0 2024-03-04T08:58:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.594+0000 7f0707dc9640 1 -- 172.21.15.71:0/1180705474 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f070019fee0 con 0x7f07001016e0 2024-03-04T08:58:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.595+0000 7f06f6ffd640 1 -- 172.21.15.71:0/1180705474 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f06f0004fa0 con 0x7f07001016e0 2024-03-04T08:58:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.595+0000 7f06f6ffd640 1 --2- 172.21.15.71:0/1180705474 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f06ec0a1e10 0x7f06ec0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.596+0000 7f0705b65640 1 --2- 172.21.15.71:0/1180705474 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f06ec0a1e10 0x7f06ec0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.596+0000 7f06f6ffd640 1 -- 172.21.15.71:0/1180705474 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f06f00cdcd0 con 0x7f07001016e0 2024-03-04T08:58:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.596+0000 7f0705b65640 1 --2- 172.21.15.71:0/1180705474 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f06ec0a1e10 0x7f06ec0a42d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0700101540 tx=0x7f06fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:16.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.600+0000 7f06f6ffd640 1 -- 172.21.15.71:0/1180705474 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f06f009ac40 con 0x7f07001016e0 2024-03-04T08:58:16.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.775+0000 7f0707dc9640 1 -- 172.21.15.71:0/1180705474 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07001a00b0 con 0x7f07001016e0 2024-03-04T08:58:16.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.777+0000 7f06f6ffd640 1 -- 172.21.15.71:0/1180705474 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24624 (secure 0 0 0) 0x7f06f009a540 con 0x7f07001016e0 2024-03-04T08:58:16.773 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:58:16.773 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":93,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","created":"2024-03-04T08:47:28.546102+0000","modified":"2024-03-04T08:57:23.805294+0000","last_up_change":"2024-03-04T08:55:30.744961+0000","last_in_change":"2024-03-04T08:55:11.570990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T08:51:14.933487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T08:56:28.253074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T08:56:29.543622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T08:56:32.884220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7e43b47d-8888-4631-bc96-8a96fcf8cd46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6803","nonce":1173641530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6805","nonce":1173641530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6809","nonce":1173641530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6807","nonce":1173641530}]},"public_addr":"172.21.15.71:6803/1173641530","cluster_addr":"172.21.15.71:6805/1173641530","heartbeat_back_addr":"172.21.15.71:6809/1173641530","heartbeat_front_addr":"172.21.15.71:6807/1173641530","state":["exists","up"]},{"osd":1,"uuid":"1209f214-f2e2-4209-bf69-4528a43a2123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6801","nonce":3383948110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6803","nonce":3383948110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6807","nonce":3383948110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6805","nonce":3383948110}]},"public_addr":"172.21.15.150:6801/3383948110","cluster_addr":"172.21.15.150:6803/3383948110","heartbeat_back_addr":"172.21.15.150:6807/3383948110","heartbeat_front_addr":"172.21.15.150:6805/3383948110","state":["exists","up"]},{"osd":2,"uuid":"b88c3082-93a9-4c9b-9221-a7105a44c91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6801","nonce":586944144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6803","nonce":586944144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6807","nonce":586944144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6805","nonce":586944144}]},"public_addr":"172.21.15.158:6801/586944144","cluster_addr":"172.21.15.158:6803/586944144","heartbeat_back_addr":"172.21.15.158:6807/586944144","heartbeat_front_addr":"172.21.15.158:6805/586944144","state":["exists","up"]},{"osd":3,"uuid":"e4fc7723-fd73-40b2-b9d0-7f9bda282ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6811","nonce":1311903262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6813","nonce":1311903262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6817","nonce":1311903262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6815","nonce":1311903262}]},"public_addr":"172.21.15.71:6811/1311903262","cluster_addr":"172.21.15.71:6813/1311903262","heartbeat_back_addr":"172.21.15.71:6817/1311903262","heartbeat_front_addr":"172.21.15.71:6815/1311903262","state":["exists","up"]},{"osd":4,"uuid":"e7eb72ac-f066-4ccd-b2bf-22a6a26daf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6809","nonce":2300837573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6811","nonce":2300837573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6815","nonce":2300837573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6813","nonce":2300837573}]},"public_addr":"172.21.15.150:6809/2300837573","cluster_addr":"172.21.15.150:6811/2300837573","heartbeat_back_addr":"172.21.15.150:6815/2300837573","heartbeat_front_addr":"172.21.15.150:6813/2300837573","state":["exists","up"]},{"osd":5,"uuid":"9b295840-e104-4a66-9cc4-7020627c1765","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6809","nonce":1922694045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6811","nonce":1922694045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6815","nonce":1922694045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6813","nonce":1922694045}]},"public_addr":"172.21.15.158:6809/1922694045","cluster_addr":"172.21.15.158:6811/1922694045","heartbeat_back_addr":"172.21.15.158:6815/1922694045","heartbeat_front_addr":"172.21.15.158:6813/1922694045","state":["exists","up"]},{"osd":6,"uuid":"6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6819","nonce":3848842707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6821","nonce":3848842707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6825","nonce":3848842707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6823","nonce":3848842707}]},"public_addr":"172.21.15.71:6819/3848842707","cluster_addr":"172.21.15.71:6821/3848842707","heartbeat_back_addr":"172.21.15.71:6825/3848842707","heartbeat_front_addr":"172.21.15.71:6823/3848842707","state":["exists","up"]},{"osd":7,"uuid":"6160a40a-d30f-4569-949c-0afb1bb3d133","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6817","nonce":1328330305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6819","nonce":1328330305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6823","nonce":1328330305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6821","nonce":1328330305}]},"public_addr":"172.21.15.150:6817/1328330305","cluster_addr":"172.21.15.150:6819/1328330305","heartbeat_back_addr":"172.21.15.150:6823/1328330305","heartbeat_front_addr":"172.21.15.150:6821/1328330305","state":["exists","up"]},{"osd":8,"uuid":"95ed2341-84a4-465c-9948-ac667680ce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6817","nonce":2195304577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6819","nonce":2195304577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6823","nonce":2195304577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6821","nonce":2195304577}]},"public_addr":"172.21.15.158:6817/2195304577","cluster_addr":"172.21.15.158:6819/2195304577","heartbeat_back_addr":"172.21.15.158:6823/2195304577","heartbeat_front_addr":"172.21.15.158:6821/2195304577","state":["exists","up"]},{"osd":9,"uuid":"088f6419-63a0-4768-afae-dbb8a5f5a565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6827","nonce":1913779347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6829","nonce":1913779347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6833","nonce":1913779347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6831","nonce":1913779347}]},"public_addr":"172.21.15.71:6827/1913779347","cluster_addr":"172.21.15.71:6829/1913779347","heartbeat_back_addr":"172.21.15.71:6833/1913779347","heartbeat_front_addr":"172.21.15.71:6831/1913779347","state":["exists","up"]},{"osd":10,"uuid":"18c85e24-068d-4f1e-a3b8-f7915b363234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6825","nonce":3295420996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6827","nonce":3295420996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6831","nonce":3295420996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6829","nonce":3295420996}]},"public_addr":"172.21.15.150:6825/3295420996","cluster_addr":"172.21.15.150:6827/3295420996","heartbeat_back_addr":"172.21.15.150:6831/3295420996","heartbeat_front_addr":"172.21.15.150:6829/3295420996","state":["exists","up"]},{"osd":11,"uuid":"4f6b3ddf-c482-4740-b1fd-de836e047c2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6825","nonce":3332083865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6827","nonce":3332083865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6831","nonce":3332083865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6829","nonce":3332083865}]},"public_addr":"172.21.15.158:6825/3332083865","cluster_addr":"172.21.15.158:6827/3332083865","heartbeat_back_addr":"172.21.15.158:6831/3332083865","heartbeat_front_addr":"172.21.15.158:6829/3332083865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:05.997832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:29.314235+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:08.819465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:41.194275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:06.032479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:38.814498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:13.067960+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:38.568479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:03.416713+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:38.349479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:04.096945+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:29.425250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1287712358":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/3770452860":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/718743094":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/1904717532":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:6800/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6800/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/2862184313":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/3621754630":"2024-03-05T08:48:22.593931+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T08:58:16.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.779+0000 7f0707dc9640 1 -- 172.21.15.71:0/1180705474 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f06ec0a1e10 msgr2=0x7f06ec0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:16.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.779+0000 7f0707dc9640 1 --2- 172.21.15.71:0/1180705474 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f06ec0a1e10 0x7f06ec0a42d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0700101540 tx=0x7f06fc005f90 comp rx=0 tx=0).stop 2024-03-04T08:58:16.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.779+0000 7f0707dc9640 1 -- 172.21.15.71:0/1180705474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07001016e0 msgr2=0x7f0700197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:16.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.779+0000 7f0707dc9640 1 --2- 172.21.15.71:0/1180705474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07001016e0 0x7f0700197db0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f06f000d770 tx=0x7f06f000dc30 comp rx=0 tx=0).stop 2024-03-04T08:58:16.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.779+0000 7f0707dc9640 1 -- 172.21.15.71:0/1180705474 shutdown_connections 2024-03-04T08:58:16.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.779+0000 7f0707dc9640 1 --2- 172.21.15.71:0/1180705474 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f06ec0a1e10 0x7f06ec0a42d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.779+0000 7f0707dc9640 1 --2- 172.21.15.71:0/1180705474 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f07001020a0 0x7f070019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.779+0000 7f0707dc9640 1 --2- 172.21.15.71:0/1180705474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07001016e0 0x7f0700197db0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.779+0000 7f0707dc9640 1 --2- 172.21.15.71:0/1180705474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07001004f0 0x7f0700197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.779+0000 7f0707dc9640 1 -- 172.21.15.71:0/1180705474 >> 172.21.15.71:0/1180705474 conn(0x7f07000fba60 msgr2=0x7f0700069c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:16.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.779+0000 7f0707dc9640 1 -- 172.21.15.71:0/1180705474 shutdown_connections 2024-03-04T08:58:16.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.779+0000 7f0707dc9640 1 -- 172.21.15.71:0/1180705474 wait complete. 2024-03-04T08:58:16.787 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T08:51:14.933487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T08:56:28.253074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T08:56:29.543622+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T08:56:32.884220+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T08:58:16.787 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T08:58:16.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.896+0000 7f7d131bc640 1 -- 172.21.15.71:0/2351847043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d0c102060 msgr2=0x7f7d0c104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:16.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.896+0000 7f7d131bc640 1 --2- 172.21.15.71:0/2351847043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d0c102060 0x7f7d0c104450 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7d00009900 tx=0x7f7d0002bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:16.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.896+0000 7f7d131bc640 1 -- 172.21.15.71:0/2351847043 shutdown_connections 2024-03-04T08:58:16.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.896+0000 7f7d131bc640 1 --2- 172.21.15.71:0/2351847043 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7d0c1072c0 0x7f7d0c109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.896+0000 7f7d131bc640 1 --2- 172.21.15.71:0/2351847043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d0c104990 0x7f7d0c106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.896+0000 7f7d131bc640 1 --2- 172.21.15.71:0/2351847043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d0c102060 0x7f7d0c104450 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.896+0000 7f7d131bc640 1 -- 172.21.15.71:0/2351847043 >> 172.21.15.71:0/2351847043 conn(0x7f7d0c0fba70 msgr2=0x7f7d0c0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:16.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.896+0000 7f7d131bc640 1 -- 172.21.15.71:0/2351847043 shutdown_connections 2024-03-04T08:58:16.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.896+0000 7f7d131bc640 1 -- 172.21.15.71:0/2351847043 wait complete. 2024-03-04T08:58:16.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.897+0000 7f7d131bc640 1 Processor -- start 2024-03-04T08:58:16.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.897+0000 7f7d131bc640 1 -- start start 2024-03-04T08:58:16.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.897+0000 7f7d131bc640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7d0c102060 0x7f7d0c1019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:16.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.897+0000 7f7d131bc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d0c104990 0x7f7d0c100010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:16.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.897+0000 7f7d131bc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d0c1072c0 0x7f7d0c100550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:16.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.897+0000 7f7d131bc640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c10d250 con 0x7f7d0c104990 2024-03-04T08:58:16.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.897+0000 7f7d131bc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c10d0d0 con 0x7f7d0c1072c0 2024-03-04T08:58:16.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.897+0000 7f7d131bc640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c10d3d0 con 0x7f7d0c102060 2024-03-04T08:58:16.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.897+0000 7f7d0bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d0c104990 0x7f7d0c100010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:16.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.897+0000 7f7d0bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d0c104990 0x7f7d0c100010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35996/0 (socket says 172.21.15.71:35996) 2024-03-04T08:58:16.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.897+0000 7f7d0bfff640 1 -- 172.21.15.71:0/2985461643 learned_addr learned my addr 172.21.15.71:0/2985461643 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:16.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.897+0000 7f7d11759640 1 --2- 172.21.15.71:0/2985461643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d0c1072c0 0x7f7d0c100550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:16.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.897+0000 7f7d10f58640 1 --2- 172.21.15.71:0/2985461643 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7d0c102060 0x7f7d0c1019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:16.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.897+0000 7f7d0bfff640 1 -- 172.21.15.71:0/2985461643 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7d0c102060 msgr2=0x7f7d0c1019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:16.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.897+0000 7f7d0bfff640 1 --2- 172.21.15.71:0/2985461643 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7d0c102060 0x7f7d0c1019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.897+0000 7f7d0bfff640 1 -- 172.21.15.71:0/2985461643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d0c1072c0 msgr2=0x7f7d0c100550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:16.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.897+0000 7f7d0bfff640 1 --2- 172.21.15.71:0/2985461643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d0c1072c0 0x7f7d0c100550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:16.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.897+0000 7f7d0bfff640 1 -- 172.21.15.71:0/2985461643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0c100de0 con 0x7f7d0c104990 2024-03-04T08:58:16.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.897+0000 7f7d10f58640 1 --2- 172.21.15.71:0/2985461643 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7d0c102060 0x7f7d0c1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:58:16.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.897+0000 7f7d0bfff640 1 --2- 172.21.15.71:0/2985461643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d0c104990 0x7f7d0c100010 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800e9a0 tx=0x7f7cf800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:16.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.897+0000 7f7d09ffb640 1 -- 172.21.15.71:0/2985461643 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf800cd80 con 0x7f7d0c104990 2024-03-04T08:58:16.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.897+0000 7f7d09ffb640 1 -- 172.21.15.71:0/2985461643 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7cf80035f0 con 0x7f7d0c104990 2024-03-04T08:58:16.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.897+0000 7f7d131bc640 1 -- 172.21.15.71:0/2985461643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d0c1a1aa0 con 0x7f7d0c104990 2024-03-04T08:58:16.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.897+0000 7f7d09ffb640 1 -- 172.21.15.71:0/2985461643 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8010670 con 0x7f7d0c104990 2024-03-04T08:58:16.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.898+0000 7f7d131bc640 1 -- 172.21.15.71:0/2985461643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d0c1a1ff0 con 0x7f7d0c104990 2024-03-04T08:58:16.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.899+0000 7f7d131bc640 1 -- 172.21.15.71:0/2985461643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d0c0fd560 con 0x7f7d0c104990 2024-03-04T08:58:16.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.899+0000 7f7d09ffb640 1 -- 172.21.15.71:0/2985461643 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f7cf80037a0 con 0x7f7d0c104990 2024-03-04T08:58:16.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.900+0000 7f7d09ffb640 1 --2- 172.21.15.71:0/2985461643 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f7cfc0a1e10 0x7f7cfc0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:16.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.900+0000 7f7d10f58640 1 --2- 172.21.15.71:0/2985461643 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f7cfc0a1e10 0x7f7cfc0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:16.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.900+0000 7f7d09ffb640 1 -- 172.21.15.71:0/2985461643 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f7cf80ce600 con 0x7f7d0c104990 2024-03-04T08:58:16.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.901+0000 7f7d10f58640 1 --2- 172.21.15.71:0/2985461643 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f7cfc0a1e10 0x7f7cfc0a42d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7d00006010 tx=0x7f7d0002c8a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:16.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:16.904+0000 7f7d09ffb640 1 -- 172.21.15.71:0/2985461643 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7cf809b570 con 0x7f7d0c104990 2024-03-04T08:58:17.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.077+0000 7f7d131bc640 1 -- 172.21.15.71:0/2985461643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7d0c02d240 con 0x7f7d0c104990 2024-03-04T08:58:17.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.077+0000 7f7d09ffb640 1 -- 172.21.15.71:0/2985461643 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7cf809ae70 con 0x7f7d0c104990 2024-03-04T08:58:17.074 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-03-04T08:58:17.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.079+0000 7f7d131bc640 1 -- 172.21.15.71:0/2985461643 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f7cfc0a1e10 msgr2=0x7f7cfc0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:17.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.079+0000 7f7d131bc640 1 --2- 172.21.15.71:0/2985461643 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f7cfc0a1e10 0x7f7cfc0a42d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7d00006010 tx=0x7f7d0002c8a0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.079+0000 7f7d131bc640 1 -- 172.21.15.71:0/2985461643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d0c104990 msgr2=0x7f7d0c100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:17.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.079+0000 7f7d131bc640 1 --2- 172.21.15.71:0/2985461643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d0c104990 0x7f7d0c100010 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800e9a0 tx=0x7f7cf800ee60 comp rx=0 tx=0).stop 2024-03-04T08:58:17.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.079+0000 7f7d131bc640 1 -- 172.21.15.71:0/2985461643 shutdown_connections 2024-03-04T08:58:17.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.079+0000 7f7d131bc640 1 --2- 172.21.15.71:0/2985461643 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f7cfc0a1e10 0x7f7cfc0a42d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.079+0000 7f7d131bc640 1 --2- 172.21.15.71:0/2985461643 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d0c1072c0 0x7f7d0c100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.079+0000 7f7d131bc640 1 --2- 172.21.15.71:0/2985461643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d0c104990 0x7f7d0c100010 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.079+0000 7f7d131bc640 1 --2- 172.21.15.71:0/2985461643 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7d0c102060 0x7f7d0c1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.079+0000 7f7d131bc640 1 -- 172.21.15.71:0/2985461643 >> 172.21.15.71:0/2985461643 conn(0x7f7d0c0fba70 msgr2=0x7f7d0c102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:17.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.079+0000 7f7d131bc640 1 -- 172.21.15.71:0/2985461643 shutdown_connections 2024-03-04T08:58:17.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.079+0000 7f7d131bc640 1 -- 172.21.15.71:0/2985461643 wait complete. 2024-03-04T08:58:17.085 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:16 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3983922836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T08:58:17.086 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:16 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2401435434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T08:58:17.086 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:16 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3314571234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:58:17.086 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:16 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1180705474' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:58:17.087 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T08:58:17.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.195+0000 7ff2a941c640 1 -- 172.21.15.71:0/3055767299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a41016c0 msgr2=0x7ff2a4101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:17.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.195+0000 7ff2a941c640 1 --2- 172.21.15.71:0/3055767299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a41016c0 0x7ff2a4101b40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009900 tx=0x7ff28c02bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:17.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.195+0000 7ff2a941c640 1 -- 172.21.15.71:0/3055767299 shutdown_connections 2024-03-04T08:58:17.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.195+0000 7ff2a941c640 1 --2- 172.21.15.71:0/3055767299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2a4102080 0x7ff2a4108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.195+0000 7ff2a941c640 1 --2- 172.21.15.71:0/3055767299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a41016c0 0x7ff2a4101b40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.195+0000 7ff2a941c640 1 --2- 172.21.15.71:0/3055767299 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff2a41004d0 0x7ff2a41008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.195+0000 7ff2a941c640 1 -- 172.21.15.71:0/3055767299 >> 172.21.15.71:0/3055767299 conn(0x7ff2a40fba60 msgr2=0x7ff2a40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:17.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.195+0000 7ff2a941c640 1 -- 172.21.15.71:0/3055767299 shutdown_connections 2024-03-04T08:58:17.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.195+0000 7ff2a941c640 1 -- 172.21.15.71:0/3055767299 wait complete. 2024-03-04T08:58:17.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.196+0000 7ff2a941c640 1 Processor -- start 2024-03-04T08:58:17.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.196+0000 7ff2a941c640 1 -- start start 2024-03-04T08:58:17.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.196+0000 7ff2a941c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2a41004d0 0x7ff2a406a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:17.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.196+0000 7ff2a941c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a41016c0 0x7ff2a419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:17.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.196+0000 7ff2a941c640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff2a4102080 0x7ff2a419ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:17.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.196+0000 7ff2a941c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a410c160 con 0x7ff2a41016c0 2024-03-04T08:58:17.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.196+0000 7ff2a941c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a410bfe0 con 0x7ff2a41004d0 2024-03-04T08:58:17.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.196+0000 7ff2a941c640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a410c2e0 con 0x7ff2a4102080 2024-03-04T08:58:17.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.196+0000 7ff29bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a41016c0 0x7ff2a419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:17.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.196+0000 7ff29bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a41016c0 0x7ff2a419c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36024/0 (socket says 172.21.15.71:36024) 2024-03-04T08:58:17.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.196+0000 7ff29bfff640 1 -- 172.21.15.71:0/1859876886 learned_addr learned my addr 172.21.15.71:0/1859876886 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:17.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.196+0000 7ff2a2ffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2a41004d0 0x7ff2a406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:17.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.196+0000 7ff2a37fe640 1 --2- 172.21.15.71:0/1859876886 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff2a4102080 0x7ff2a419ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:17.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.196+0000 7ff29bfff640 1 -- 172.21.15.71:0/1859876886 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff2a4102080 msgr2=0x7ff2a419ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:17.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.196+0000 7ff29bfff640 1 --2- 172.21.15.71:0/1859876886 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff2a4102080 0x7ff2a419ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.196+0000 7ff29bfff640 1 -- 172.21.15.71:0/1859876886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2a41004d0 msgr2=0x7ff2a406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:17.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.196+0000 7ff29bfff640 1 --2- 172.21.15.71:0/1859876886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2a41004d0 0x7ff2a406a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.196+0000 7ff29bfff640 1 -- 172.21.15.71:0/1859876886 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a419f550 con 0x7ff2a41016c0 2024-03-04T08:58:17.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.196+0000 7ff29bfff640 1 --2- 172.21.15.71:0/1859876886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a41016c0 0x7ff2a419c650 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009230 tx=0x7ff28c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:17.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.197+0000 7ff2a0ff9640 1 -- 172.21.15.71:0/1859876886 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c041070 con 0x7ff2a41016c0 2024-03-04T08:58:17.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.197+0000 7ff2a941c640 1 -- 172.21.15.71:0/1859876886 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a419f7e0 con 0x7ff2a41016c0 2024-03-04T08:58:17.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.197+0000 7ff2a0ff9640 1 -- 172.21.15.71:0/1859876886 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff28c0044d0 con 0x7ff2a41016c0 2024-03-04T08:58:17.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.197+0000 7ff2a0ff9640 1 -- 172.21.15.71:0/1859876886 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c034440 con 0x7ff2a41016c0 2024-03-04T08:58:17.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.197+0000 7ff2a941c640 1 -- 172.21.15.71:0/1859876886 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a419fd20 con 0x7ff2a41016c0 2024-03-04T08:58:17.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.198+0000 7ff2a941c640 1 -- 172.21.15.71:0/1859876886 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a4107870 con 0x7ff2a41016c0 2024-03-04T08:58:17.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.199+0000 7ff2a0ff9640 1 -- 172.21.15.71:0/1859876886 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7ff28c044070 con 0x7ff2a41016c0 2024-03-04T08:58:17.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.199+0000 7ff2a0ff9640 1 --2- 172.21.15.71:0/1859876886 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff2900a1e10 0x7ff2900a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:17.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.199+0000 7ff2a2ffd640 1 --2- 172.21.15.71:0/1859876886 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff2900a1e10 0x7ff2900a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:17.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.199+0000 7ff2a0ff9640 1 -- 172.21.15.71:0/1859876886 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7ff28c0efdf0 con 0x7ff2a41016c0 2024-03-04T08:58:17.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.200+0000 7ff2a2ffd640 1 --2- 172.21.15.71:0/1859876886 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff2900a1e10 0x7ff2900a42d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4101520 tx=0x7ff29400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:17.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.204+0000 7ff2a0ff9640 1 -- 172.21.15.71:0/1859876886 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff28c0bce70 con 0x7ff2a41016c0 2024-03-04T08:58:17.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:16 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3983922836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T08:58:17.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:16 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2401435434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T08:58:17.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:16 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3314571234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:58:17.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:16 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1180705474' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:58:17.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:16 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3983922836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T08:58:17.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:16 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2401435434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T08:58:17.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:16 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3314571234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:58:17.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:16 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1180705474' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:58:17.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.376+0000 7ff2a941c640 1 -- 172.21.15.71:0/1859876886 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff2a402d1c0 con 0x7ff2a41016c0 2024-03-04T08:58:17.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.376+0000 7ff2a0ff9640 1 -- 172.21.15.71:0/1859876886 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff28c0bc770 con 0x7ff2a41016c0 2024-03-04T08:58:17.373 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-03-04T08:58:17.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.378+0000 7ff2a941c640 1 -- 172.21.15.71:0/1859876886 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff2900a1e10 msgr2=0x7ff2900a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:17.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.378+0000 7ff2a941c640 1 --2- 172.21.15.71:0/1859876886 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff2900a1e10 0x7ff2900a42d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4101520 tx=0x7ff29400b390 comp rx=0 tx=0).stop 2024-03-04T08:58:17.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.378+0000 7ff2a941c640 1 -- 172.21.15.71:0/1859876886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a41016c0 msgr2=0x7ff2a419c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:17.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.378+0000 7ff2a941c640 1 --2- 172.21.15.71:0/1859876886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a41016c0 0x7ff2a419c650 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009230 tx=0x7ff28c005d50 comp rx=0 tx=0).stop 2024-03-04T08:58:17.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.378+0000 7ff2a941c640 1 -- 172.21.15.71:0/1859876886 shutdown_connections 2024-03-04T08:58:17.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.378+0000 7ff2a941c640 1 --2- 172.21.15.71:0/1859876886 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ff2900a1e10 0x7ff2900a42d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.378+0000 7ff2a941c640 1 --2- 172.21.15.71:0/1859876886 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff2a4102080 0x7ff2a419ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.378+0000 7ff2a941c640 1 --2- 172.21.15.71:0/1859876886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a41016c0 0x7ff2a419c650 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.378+0000 7ff2a941c640 1 --2- 172.21.15.71:0/1859876886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2a41004d0 0x7ff2a406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.378+0000 7ff2a941c640 1 -- 172.21.15.71:0/1859876886 >> 172.21.15.71:0/1859876886 conn(0x7ff2a40fba60 msgr2=0x7ff2a4104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:17.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.379+0000 7ff2a941c640 1 -- 172.21.15.71:0/1859876886 shutdown_connections 2024-03-04T08:58:17.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.379+0000 7ff2a941c640 1 -- 172.21.15.71:0/1859876886 wait complete. 2024-03-04T08:58:17.386 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T08:58:17.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.494+0000 7f5e512af640 1 -- 172.21.15.71:0/2914778604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e4c102090 msgr2=0x7f5e4c108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:17.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.494+0000 7f5e512af640 1 --2- 172.21.15.71:0/2914778604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e4c102090 0x7f5e4c108920 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5e44009be0 tx=0x7f5e4402bef0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.494+0000 7f5e512af640 1 -- 172.21.15.71:0/2914778604 shutdown_connections 2024-03-04T08:58:17.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.494+0000 7f5e512af640 1 --2- 172.21.15.71:0/2914778604 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e4c102090 0x7f5e4c108920 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.494+0000 7f5e512af640 1 --2- 172.21.15.71:0/2914778604 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5e4c1016d0 0x7f5e4c101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.494+0000 7f5e512af640 1 --2- 172.21.15.71:0/2914778604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e4c1004e0 0x7f5e4c100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.494+0000 7f5e512af640 1 -- 172.21.15.71:0/2914778604 >> 172.21.15.71:0/2914778604 conn(0x7f5e4c0fba70 msgr2=0x7f5e4c0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:17.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.494+0000 7f5e512af640 1 -- 172.21.15.71:0/2914778604 shutdown_connections 2024-03-04T08:58:17.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.494+0000 7f5e512af640 1 -- 172.21.15.71:0/2914778604 wait complete. 2024-03-04T08:58:17.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.494+0000 7f5e512af640 1 Processor -- start 2024-03-04T08:58:17.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.495+0000 7f5e512af640 1 -- start start 2024-03-04T08:58:17.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.495+0000 7f5e512af640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e4c1004e0 0x7f5e4c06a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:17.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.495+0000 7f5e512af640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e4c1016d0 0x7f5e4c19c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:17.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.495+0000 7f5e512af640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5e4c102090 0x7f5e4c19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:17.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.495+0000 7f5e512af640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c10c170 con 0x7f5e4c1004e0 2024-03-04T08:58:17.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.495+0000 7f5e512af640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c10bff0 con 0x7f5e4c1016d0 2024-03-04T08:58:17.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.495+0000 7f5e512af640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c10c2f0 con 0x7f5e4c102090 2024-03-04T08:58:17.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.495+0000 7f5e4affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e4c1004e0 0x7f5e4c06a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:17.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.495+0000 7f5e4affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e4c1004e0 0x7f5e4c06a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36042/0 (socket says 172.21.15.71:36042) 2024-03-04T08:58:17.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.495+0000 7f5e4affd640 1 -- 172.21.15.71:0/2486284821 learned_addr learned my addr 172.21.15.71:0/2486284821 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:17.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.495+0000 7f5e4a7fc640 1 --2- 172.21.15.71:0/2486284821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e4c1016d0 0x7f5e4c19c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:17.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.495+0000 7f5e4b7fe640 1 --2- 172.21.15.71:0/2486284821 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5e4c102090 0x7f5e4c19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:17.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.495+0000 7f5e4affd640 1 -- 172.21.15.71:0/2486284821 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5e4c102090 msgr2=0x7f5e4c19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:17.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.495+0000 7f5e4affd640 1 --2- 172.21.15.71:0/2486284821 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5e4c102090 0x7f5e4c19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.495+0000 7f5e4affd640 1 -- 172.21.15.71:0/2486284821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e4c1016d0 msgr2=0x7f5e4c19c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:17.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.495+0000 7f5e4affd640 1 --2- 172.21.15.71:0/2486284821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e4c1016d0 0x7f5e4c19c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.495+0000 7f5e4affd640 1 -- 172.21.15.71:0/2486284821 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c19f4a0 con 0x7f5e4c1004e0 2024-03-04T08:58:17.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.495+0000 7f5e4b7fe640 1 --2- 172.21.15.71:0/2486284821 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5e4c102090 0x7f5e4c19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:58:17.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.495+0000 7f5e4affd640 1 --2- 172.21.15.71:0/2486284821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e4c1004e0 0x7f5e4c06a6d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00e9f0 tx=0x7f5e3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:17.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.496+0000 7f5e33fff640 1 -- 172.21.15.71:0/2486284821 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c00ccc0 con 0x7f5e4c1004e0 2024-03-04T08:58:17.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.496+0000 7f5e512af640 1 -- 172.21.15.71:0/2486284821 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4c19f790 con 0x7f5e4c1004e0 2024-03-04T08:58:17.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.496+0000 7f5e33fff640 1 -- 172.21.15.71:0/2486284821 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5e3c0035f0 con 0x7f5e4c1004e0 2024-03-04T08:58:17.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.496+0000 7f5e33fff640 1 -- 172.21.15.71:0/2486284821 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c00ce60 con 0x7f5e4c1004e0 2024-03-04T08:58:17.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.496+0000 7f5e512af640 1 -- 172.21.15.71:0/2486284821 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4c19fd30 con 0x7f5e4c1004e0 2024-03-04T08:58:17.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.497+0000 7f5e512af640 1 -- 172.21.15.71:0/2486284821 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e4c068810 con 0x7f5e4c1004e0 2024-03-04T08:58:17.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.497+0000 7f5e33fff640 1 -- 172.21.15.71:0/2486284821 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f5e3c0037a0 con 0x7f5e4c1004e0 2024-03-04T08:58:17.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.498+0000 7f5e33fff640 1 --2- 172.21.15.71:0/2486284821 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f5e340a1e10 0x7f5e340a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:17.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.498+0000 7f5e4a7fc640 1 --2- 172.21.15.71:0/2486284821 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f5e340a1e10 0x7f5e340a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:17.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.498+0000 7f5e33fff640 1 -- 172.21.15.71:0/2486284821 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f5e3c0ce590 con 0x7f5e4c1004e0 2024-03-04T08:58:17.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.498+0000 7f5e4a7fc640 1 --2- 172.21.15.71:0/2486284821 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f5e340a1e10 0x7f5e340a42d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c06a440 tx=0x7f5e4000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:17.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.502+0000 7f5e33fff640 1 -- 172.21.15.71:0/2486284821 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5e3c09b500 con 0x7f5e4c1004e0 2024-03-04T08:58:17.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.676+0000 7f5e512af640 1 -- 172.21.15.71:0/2486284821 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5e4c02cff0 con 0x7f5e4c1004e0 2024-03-04T08:58:17.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.676+0000 7f5e33fff640 1 -- 172.21.15.71:0/2486284821 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5e3c09ae00 con 0x7f5e4c1004e0 2024-03-04T08:58:17.672 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2024-03-04T08:58:17.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.678+0000 7f5e512af640 1 -- 172.21.15.71:0/2486284821 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f5e340a1e10 msgr2=0x7f5e340a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:17.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.678+0000 7f5e512af640 1 --2- 172.21.15.71:0/2486284821 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f5e340a1e10 0x7f5e340a42d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c06a440 tx=0x7f5e4000b390 comp rx=0 tx=0).stop 2024-03-04T08:58:17.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.678+0000 7f5e512af640 1 -- 172.21.15.71:0/2486284821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e4c1004e0 msgr2=0x7f5e4c06a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:17.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.678+0000 7f5e512af640 1 --2- 172.21.15.71:0/2486284821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e4c1004e0 0x7f5e4c06a6d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00e9f0 tx=0x7f5e3c00eeb0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.678+0000 7f5e512af640 1 -- 172.21.15.71:0/2486284821 shutdown_connections 2024-03-04T08:58:17.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.678+0000 7f5e512af640 1 --2- 172.21.15.71:0/2486284821 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f5e340a1e10 0x7f5e340a42d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.678+0000 7f5e512af640 1 --2- 172.21.15.71:0/2486284821 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5e4c102090 0x7f5e4c19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.678+0000 7f5e512af640 1 --2- 172.21.15.71:0/2486284821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e4c1016d0 0x7f5e4c19c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.678+0000 7f5e512af640 1 --2- 172.21.15.71:0/2486284821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e4c1004e0 0x7f5e4c06a6d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.678+0000 7f5e512af640 1 -- 172.21.15.71:0/2486284821 >> 172.21.15.71:0/2486284821 conn(0x7f5e4c0fba70 msgr2=0x7f5e4c1049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:17.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.678+0000 7f5e512af640 1 -- 172.21.15.71:0/2486284821 shutdown_connections 2024-03-04T08:58:17.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.678+0000 7f5e512af640 1 -- 172.21.15.71:0/2486284821 wait complete. 2024-03-04T08:58:17.685 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T08:58:17.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.794+0000 7f5415850640 1 -- 172.21.15.71:0/3189191042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5410107390 msgr2=0x7f5410109780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:17.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.794+0000 7f5415850640 1 --2- 172.21.15.71:0/3189191042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5410107390 0x7f5410109780 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5408009900 tx=0x7f540802bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:17.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.794+0000 7f5415850640 1 -- 172.21.15.71:0/3189191042 shutdown_connections 2024-03-04T08:58:17.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.794+0000 7f5415850640 1 --2- 172.21.15.71:0/3189191042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5410107390 0x7f5410109780 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.794+0000 7f5415850640 1 --2- 172.21.15.71:0/3189191042 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5410104a60 0x7f5410106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.794+0000 7f5415850640 1 --2- 172.21.15.71:0/3189191042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5410102130 0x7f5410104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.794+0000 7f5415850640 1 -- 172.21.15.71:0/3189191042 >> 172.21.15.71:0/3189191042 conn(0x7f54100fba80 msgr2=0x7f54100fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:17.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.794+0000 7f5415850640 1 -- 172.21.15.71:0/3189191042 shutdown_connections 2024-03-04T08:58:17.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.794+0000 7f5415850640 1 -- 172.21.15.71:0/3189191042 wait complete. 2024-03-04T08:58:17.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.795+0000 7f5415850640 1 Processor -- start 2024-03-04T08:58:17.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.795+0000 7f5415850640 1 -- start start 2024-03-04T08:58:17.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.795+0000 7f5415850640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5410102130 0x7f5410197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:17.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.795+0000 7f5415850640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5410104a60 0x7f5410197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:17.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.795+0000 7f5415850640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5410107390 0x7f541019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:17.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.795+0000 7f5415850640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541010d290 con 0x7f5410107390 2024-03-04T08:58:17.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.795+0000 7f5415850640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541010d110 con 0x7f5410104a60 2024-03-04T08:58:17.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.795+0000 7f5415850640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541010d410 con 0x7f5410102130 2024-03-04T08:58:17.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.795+0000 7f540f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5410107390 0x7f541019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:17.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.795+0000 7f540f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5410107390 0x7f541019eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36074/0 (socket says 172.21.15.71:36074) 2024-03-04T08:58:17.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.795+0000 7f540f7fe640 1 -- 172.21.15.71:0/1593945244 learned_addr learned my addr 172.21.15.71:0/1593945244 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:17.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.795+0000 7f540e7fc640 1 --2- 172.21.15.71:0/1593945244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5410104a60 0x7f5410197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:17.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.795+0000 7f540effd640 1 --2- 172.21.15.71:0/1593945244 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5410102130 0x7f5410197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:17.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.795+0000 7f540f7fe640 1 -- 172.21.15.71:0/1593945244 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5410102130 msgr2=0x7f5410197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:17.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.795+0000 7f540f7fe640 1 --2- 172.21.15.71:0/1593945244 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5410102130 0x7f5410197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.795+0000 7f540f7fe640 1 -- 172.21.15.71:0/1593945244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5410104a60 msgr2=0x7f5410197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:17.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.795+0000 7f540f7fe640 1 --2- 172.21.15.71:0/1593945244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5410104a60 0x7f5410197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.795+0000 7f540f7fe640 1 -- 172.21.15.71:0/1593945244 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541019f550 con 0x7f5410107390 2024-03-04T08:58:17.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.796+0000 7f540f7fe640 1 --2- 172.21.15.71:0/1593945244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5410107390 0x7f541019eee0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f540802c7b0 tx=0x7f5408005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:17.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.796+0000 7f541484e640 1 -- 172.21.15.71:0/1593945244 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5408041070 con 0x7f5410107390 2024-03-04T08:58:17.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.796+0000 7f541484e640 1 -- 172.21.15.71:0/1593945244 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54080054e0 con 0x7f5410107390 2024-03-04T08:58:17.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.796+0000 7f541484e640 1 -- 172.21.15.71:0/1593945244 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5408034440 con 0x7f5410107390 2024-03-04T08:58:17.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.796+0000 7f5415850640 1 -- 172.21.15.71:0/1593945244 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541019f7e0 con 0x7f5410107390 2024-03-04T08:58:17.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.796+0000 7f5415850640 1 -- 172.21.15.71:0/1593945244 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541019fcd0 con 0x7f5410107390 2024-03-04T08:58:17.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.797+0000 7f5415850640 1 -- 172.21.15.71:0/1593945244 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54100fd570 con 0x7f5410107390 2024-03-04T08:58:17.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.798+0000 7f541484e640 1 -- 172.21.15.71:0/1593945244 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f5408044070 con 0x7f5410107390 2024-03-04T08:58:17.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.798+0000 7f541484e640 1 --2- 172.21.15.71:0/1593945244 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f53f80a1e10 0x7f53f80a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:17.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.798+0000 7f540effd640 1 --2- 172.21.15.71:0/1593945244 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f53f80a1e10 0x7f53f80a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:17.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.798+0000 7f541484e640 1 -- 172.21.15.71:0/1593945244 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f54080efe50 con 0x7f5410107390 2024-03-04T08:58:17.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.799+0000 7f540effd640 1 --2- 172.21.15.71:0/1593945244 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f53f80a1e10 0x7f53f80a42d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f54000095f0 tx=0x7f5400005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:17.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.803+0000 7f541484e640 1 -- 172.21.15.71:0/1593945244 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f54080bced0 con 0x7f5410107390 2024-03-04T08:58:17.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.977+0000 7f5415850640 1 -- 172.21.15.71:0/1593945244 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f54100621d0 con 0x7f5410107390 2024-03-04T08:58:17.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.977+0000 7f541484e640 1 -- 172.21.15.71:0/1593945244 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f54080bc7d0 con 0x7f5410107390 2024-03-04T08:58:17.973 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-03-04T08:58:17.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.979+0000 7f5415850640 1 -- 172.21.15.71:0/1593945244 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f53f80a1e10 msgr2=0x7f53f80a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:17.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.979+0000 7f5415850640 1 --2- 172.21.15.71:0/1593945244 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f53f80a1e10 0x7f53f80a42d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f54000095f0 tx=0x7f5400005f90 comp rx=0 tx=0).stop 2024-03-04T08:58:17.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.979+0000 7f5415850640 1 -- 172.21.15.71:0/1593945244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5410107390 msgr2=0x7f541019eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:17.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.979+0000 7f5415850640 1 --2- 172.21.15.71:0/1593945244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5410107390 0x7f541019eee0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f540802c7b0 tx=0x7f5408005dd0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.979+0000 7f5415850640 1 -- 172.21.15.71:0/1593945244 shutdown_connections 2024-03-04T08:58:17.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.979+0000 7f5415850640 1 --2- 172.21.15.71:0/1593945244 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f53f80a1e10 0x7f53f80a42d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.979+0000 7f5415850640 1 --2- 172.21.15.71:0/1593945244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5410107390 0x7f541019eee0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.979+0000 7f5415850640 1 --2- 172.21.15.71:0/1593945244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5410104a60 0x7f5410197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.979+0000 7f5415850640 1 --2- 172.21.15.71:0/1593945244 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5410102130 0x7f5410197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:17.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.979+0000 7f5415850640 1 -- 172.21.15.71:0/1593945244 >> 172.21.15.71:0/1593945244 conn(0x7f54100fba80 msgr2=0x7f54101029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:17.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.979+0000 7f5415850640 1 -- 172.21.15.71:0/1593945244 shutdown_connections 2024-03-04T08:58:17.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:17.979+0000 7f5415850640 1 -- 172.21.15.71:0/1593945244 wait complete. 2024-03-04T08:58:17.985 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:17 smithi071 ceph-mon[32789]: pgmap v380: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:58:17.985 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:17 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2985461643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:58:17.985 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:17 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1859876886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T08:58:17.986 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:17 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2486284821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T08:58:17.987 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:58:18.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.091+0000 7feb38f56640 1 -- 172.21.15.71:0/3922144198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb341072d0 msgr2=0x7feb34109720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:18.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.091+0000 7feb38f56640 1 --2- 172.21.15.71:0/3922144198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb341072d0 0x7feb34109720 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009900 tx=0x7feb2c02bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:18.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.091+0000 7feb38f56640 1 -- 172.21.15.71:0/3922144198 shutdown_connections 2024-03-04T08:58:18.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.091+0000 7feb38f56640 1 --2- 172.21.15.71:0/3922144198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb341072d0 0x7feb34109720 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.091+0000 7feb38f56640 1 --2- 172.21.15.71:0/3922144198 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb341049a0 0x7feb34106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.091+0000 7feb38f56640 1 --2- 172.21.15.71:0/3922144198 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feb34102070 0x7feb34104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.091+0000 7feb38f56640 1 -- 172.21.15.71:0/3922144198 >> 172.21.15.71:0/3922144198 conn(0x7feb340fba60 msgr2=0x7feb340fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:18.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.092+0000 7feb38f56640 1 -- 172.21.15.71:0/3922144198 shutdown_connections 2024-03-04T08:58:18.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.092+0000 7feb38f56640 1 -- 172.21.15.71:0/3922144198 wait complete. 2024-03-04T08:58:18.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.092+0000 7feb38f56640 1 Processor -- start 2024-03-04T08:58:18.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.092+0000 7feb38f56640 1 -- start start 2024-03-04T08:58:18.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.092+0000 7feb38f56640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb34102070 0x7feb341978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:18.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.092+0000 7feb38f56640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb341049a0 0x7feb34197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:18.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.092+0000 7feb38f56640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feb341072d0 0x7feb3419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:18.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.092+0000 7feb38f56640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3410d260 con 0x7feb341049a0 2024-03-04T08:58:18.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.092+0000 7feb38f56640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3410d0e0 con 0x7feb34102070 2024-03-04T08:58:18.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.092+0000 7feb38f56640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3410d3e0 con 0x7feb341072d0 2024-03-04T08:58:18.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.093+0000 7feb31d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb341049a0 0x7feb34197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:18.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.093+0000 7feb31d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb341049a0 0x7feb34197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36102/0 (socket says 172.21.15.71:36102) 2024-03-04T08:58:18.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.093+0000 7feb31d9b640 1 -- 172.21.15.71:0/2944752809 learned_addr learned my addr 172.21.15.71:0/2944752809 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:18.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.093+0000 7feb3259c640 1 --2- 172.21.15.71:0/2944752809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb34102070 0x7feb341978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:18.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.093+0000 7feb32d9d640 1 --2- 172.21.15.71:0/2944752809 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feb341072d0 0x7feb3419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:18.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.093+0000 7feb31d9b640 1 -- 172.21.15.71:0/2944752809 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feb341072d0 msgr2=0x7feb3419ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:18.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.093+0000 7feb31d9b640 1 --2- 172.21.15.71:0/2944752809 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feb341072d0 0x7feb3419ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.093+0000 7feb31d9b640 1 -- 172.21.15.71:0/2944752809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb34102070 msgr2=0x7feb341978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:18.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.093+0000 7feb31d9b640 1 --2- 172.21.15.71:0/2944752809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb34102070 0x7feb341978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.093+0000 7feb31d9b640 1 -- 172.21.15.71:0/2944752809 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3419f4d0 con 0x7feb341049a0 2024-03-04T08:58:18.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.093+0000 7feb31d9b640 1 --2- 172.21.15.71:0/2944752809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb341049a0 0x7feb34197df0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7feb2800e9a0 tx=0x7feb2800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:18.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.093+0000 7feb237fe640 1 -- 172.21.15.71:0/2944752809 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2800cd80 con 0x7feb341049a0 2024-03-04T08:58:18.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.093+0000 7feb237fe640 1 -- 172.21.15.71:0/2944752809 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb280035f0 con 0x7feb341049a0 2024-03-04T08:58:18.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.093+0000 7feb38f56640 1 -- 172.21.15.71:0/2944752809 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb3419f7c0 con 0x7feb341049a0 2024-03-04T08:58:18.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.093+0000 7feb237fe640 1 -- 172.21.15.71:0/2944752809 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb28010670 con 0x7feb341049a0 2024-03-04T08:58:18.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.093+0000 7feb38f56640 1 -- 172.21.15.71:0/2944752809 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb3419fd10 con 0x7feb341049a0 2024-03-04T08:58:18.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.094+0000 7feb38f56640 1 -- 172.21.15.71:0/2944752809 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb340fd550 con 0x7feb341049a0 2024-03-04T08:58:18.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.095+0000 7feb237fe640 1 -- 172.21.15.71:0/2944752809 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7feb280037a0 con 0x7feb341049a0 2024-03-04T08:58:18.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.096+0000 7feb237fe640 1 --2- 172.21.15.71:0/2944752809 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7feb1c0a1e10 0x7feb1c0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:18.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.096+0000 7feb3259c640 1 --2- 172.21.15.71:0/2944752809 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7feb1c0a1e10 0x7feb1c0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:18.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.096+0000 7feb237fe640 1 -- 172.21.15.71:0/2944752809 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7feb280ce520 con 0x7feb341049a0 2024-03-04T08:58:18.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.096+0000 7feb3259c640 1 --2- 172.21.15.71:0/2944752809 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7feb1c0a1e10 0x7feb1c0a42d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7feb24005fd0 tx=0x7feb24009760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:18.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.100+0000 7feb237fe640 1 -- 172.21.15.71:0/2944752809 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7feb2809b490 con 0x7feb341049a0 2024-03-04T08:58:18.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:17 smithi150 ceph-mon[36807]: pgmap v380: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:58:18.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:17 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2985461643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:58:18.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:17 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1859876886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T08:58:18.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:17 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2486284821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T08:58:18.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:17 smithi158 ceph-mon[37241]: pgmap v380: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:58:18.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:17 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2985461643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:58:18.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:17 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1859876886' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T08:58:18.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:17 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2486284821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T08:58:18.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.309+0000 7feb38f56640 1 -- 172.21.15.71:0/2944752809 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb3402cff0 con 0x7feb341049a0 2024-03-04T08:58:18.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.310+0000 7feb237fe640 1 -- 172.21.15.71:0/2944752809 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11339 (secure 0 0 0) 0x7feb2809ad90 con 0x7feb341049a0 2024-03-04T08:58:18.306 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:58:18.306 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3975379807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3975379807},{"type":"v1","addr":"172.21.15.71:6835","nonce":3975379807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1962929347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1962929347},{"type":"v1","addr":"172.21.15.71:6837","nonce":1962929347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/1459252176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1459252176},{"type":"v1","addr":"172.21.15.71:6839","nonce":1459252176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6841/1083546600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":1083546600},{"type":"v1","addr":"172.21.15.71:6841","nonce":1083546600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/178179528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":178179528},{"type":"v1","addr":"172.21.15.158:6833","nonce":178179528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2468877675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2468877675},{"type":"v1","addr":"172.21.15.150:6833","nonce":2468877675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1652094309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1652094309},{"type":"v1","addr":"172.21.15.158:6835","nonce":1652094309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1802416462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1802416462},{"type":"v1","addr":"172.21.15.150:6835","nonce":1802416462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/617058380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":617058380},{"type":"v1","addr":"172.21.15.158:6837","nonce":617058380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/913465600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":913465600},{"type":"v1","addr":"172.21.15.150:6837","nonce":913465600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T08:56:30.343357+0000","modified":"2024-03-04T08:56:54.901214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.158:6839/2921704804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6838","nonce":2921704804},{"type":"v1","addr":"172.21.15.158:6839","nonce":2921704804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/2639278083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2639278083},{"type":"v1","addr":"172.21.15.150:6839","nonce":2639278083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491]},"id":1}]} 2024-03-04T08:58:18.306 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 20 2024-03-04T08:58:18.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.312+0000 7feb38f56640 1 -- 172.21.15.71:0/2944752809 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7feb1c0a1e10 msgr2=0x7feb1c0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:18.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.312+0000 7feb38f56640 1 --2- 172.21.15.71:0/2944752809 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7feb1c0a1e10 0x7feb1c0a42d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7feb24005fd0 tx=0x7feb24009760 comp rx=0 tx=0).stop 2024-03-04T08:58:18.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.312+0000 7feb38f56640 1 -- 172.21.15.71:0/2944752809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb341049a0 msgr2=0x7feb34197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:18.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.312+0000 7feb38f56640 1 --2- 172.21.15.71:0/2944752809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb341049a0 0x7feb34197df0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7feb2800e9a0 tx=0x7feb2800ee60 comp rx=0 tx=0).stop 2024-03-04T08:58:18.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.312+0000 7feb38f56640 1 -- 172.21.15.71:0/2944752809 shutdown_connections 2024-03-04T08:58:18.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.312+0000 7feb38f56640 1 --2- 172.21.15.71:0/2944752809 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7feb1c0a1e10 0x7feb1c0a42d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.312+0000 7feb38f56640 1 --2- 172.21.15.71:0/2944752809 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7feb341072d0 0x7feb3419ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.312+0000 7feb38f56640 1 --2- 172.21.15.71:0/2944752809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb341049a0 0x7feb34197df0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.312+0000 7feb38f56640 1 --2- 172.21.15.71:0/2944752809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb34102070 0x7feb341978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.312+0000 7feb38f56640 1 -- 172.21.15.71:0/2944752809 >> 172.21.15.71:0/2944752809 conn(0x7feb340fba60 msgr2=0x7feb341029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:18.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.312+0000 7feb38f56640 1 -- 172.21.15.71:0/2944752809 shutdown_connections 2024-03-04T08:58:18.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.312+0000 7feb38f56640 1 -- 172.21.15.71:0/2944752809 wait complete. 2024-03-04T08:58:18.319 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T08:58:18.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.424+0000 7f01d2a22640 1 -- 172.21.15.71:0/2345633899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01cc1004f0 msgr2=0x7f01cc100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:18.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.424+0000 7f01d2a22640 1 --2- 172.21.15.71:0/2345633899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01cc1004f0 0x7f01cc100910 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f01bc009900 tx=0x7f01bc02bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:18.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.424+0000 7f01d2a22640 1 -- 172.21.15.71:0/2345633899 shutdown_connections 2024-03-04T08:58:18.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.424+0000 7f01d2a22640 1 --2- 172.21.15.71:0/2345633899 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f01cc1020a0 0x7f01cc108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.424+0000 7f01d2a22640 1 --2- 172.21.15.71:0/2345633899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01cc1016e0 0x7f01cc101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.424+0000 7f01d2a22640 1 --2- 172.21.15.71:0/2345633899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01cc1004f0 0x7f01cc100910 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.424+0000 7f01d2a22640 1 -- 172.21.15.71:0/2345633899 >> 172.21.15.71:0/2345633899 conn(0x7f01cc0fba60 msgr2=0x7f01cc0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:18.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.424+0000 7f01d2a22640 1 -- 172.21.15.71:0/2345633899 shutdown_connections 2024-03-04T08:58:18.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.425+0000 7f01d2a22640 1 -- 172.21.15.71:0/2345633899 wait complete. 2024-03-04T08:58:18.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.425+0000 7f01d2a22640 1 Processor -- start 2024-03-04T08:58:18.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.425+0000 7f01d2a22640 1 -- start start 2024-03-04T08:58:18.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.425+0000 7f01d2a22640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01cc1004f0 0x7f01cc196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:18.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.425+0000 7f01d2a22640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f01cc1016e0 0x7f01cc196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:18.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.425+0000 7f01d2a22640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01cc1020a0 0x7f01cc19ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:18.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.425+0000 7f01d2a22640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc10c180 con 0x7f01cc1004f0 2024-03-04T08:58:18.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.425+0000 7f01d2a22640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc10c000 con 0x7f01cc1020a0 2024-03-04T08:58:18.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.425+0000 7f01d2a22640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc10c300 con 0x7f01cc1016e0 2024-03-04T08:58:18.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.425+0000 7f01cbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01cc1004f0 0x7f01cc196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:18.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.425+0000 7f01cbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01cc1004f0 0x7f01cc196800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36126/0 (socket says 172.21.15.71:36126) 2024-03-04T08:58:18.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.425+0000 7f01cbfff640 1 -- 172.21.15.71:0/2055911321 learned_addr learned my addr 172.21.15.71:0/2055911321 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:18.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.426+0000 7f01d0fbf640 1 --2- 172.21.15.71:0/2055911321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01cc1020a0 0x7f01cc19ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:18.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.426+0000 7f01cb7fe640 1 --2- 172.21.15.71:0/2055911321 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f01cc1016e0 0x7f01cc196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:18.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.426+0000 7f01cbfff640 1 -- 172.21.15.71:0/2055911321 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f01cc1016e0 msgr2=0x7f01cc196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:18.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.426+0000 7f01cbfff640 1 --2- 172.21.15.71:0/2055911321 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f01cc1016e0 0x7f01cc196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.426+0000 7f01cbfff640 1 -- 172.21.15.71:0/2055911321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01cc1020a0 msgr2=0x7f01cc19ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:18.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.426+0000 7f01cbfff640 1 --2- 172.21.15.71:0/2055911321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01cc1020a0 0x7f01cc19ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.426+0000 7f01cbfff640 1 -- 172.21.15.71:0/2055911321 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01cc19e420 con 0x7f01cc1004f0 2024-03-04T08:58:18.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.426+0000 7f01d0fbf640 1 --2- 172.21.15.71:0/2055911321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01cc1020a0 0x7f01cc19ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:58:18.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.426+0000 7f01cbfff640 1 --2- 172.21.15.71:0/2055911321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01cc1004f0 0x7f01cc196800 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f01bc000c00 tx=0x7f01bc02c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:18.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.426+0000 7f01c97fa640 1 -- 172.21.15.71:0/2055911321 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01bc041070 con 0x7f01cc1004f0 2024-03-04T08:58:18.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.426+0000 7f01c97fa640 1 -- 172.21.15.71:0/2055911321 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f01bc004450 con 0x7f01cc1004f0 2024-03-04T08:58:18.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.426+0000 7f01d2a22640 1 -- 172.21.15.71:0/2055911321 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01cc19e6b0 con 0x7f01cc1004f0 2024-03-04T08:58:18.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.426+0000 7f01c97fa640 1 -- 172.21.15.71:0/2055911321 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01bc034440 con 0x7f01cc1004f0 2024-03-04T08:58:18.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.426+0000 7f01d2a22640 1 -- 172.21.15.71:0/2055911321 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01cc19eba0 con 0x7f01cc1004f0 2024-03-04T08:58:18.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.427+0000 7f01d2a22640 1 -- 172.21.15.71:0/2055911321 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01cc107890 con 0x7f01cc1004f0 2024-03-04T08:58:18.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.428+0000 7f01c97fa640 1 -- 172.21.15.71:0/2055911321 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f01bc044070 con 0x7f01cc1004f0 2024-03-04T08:58:18.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.429+0000 7f01c97fa640 1 --2- 172.21.15.71:0/2055911321 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f01b40a1e10 0x7f01b40a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:18.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.429+0000 7f01cb7fe640 1 --2- 172.21.15.71:0/2055911321 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f01b40a1e10 0x7f01b40a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:18.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.429+0000 7f01c97fa640 1 -- 172.21.15.71:0/2055911321 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f01bc0efc00 con 0x7f01cc1004f0 2024-03-04T08:58:18.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.429+0000 7f01cb7fe640 1 --2- 172.21.15.71:0/2055911321 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f01b40a1e10 0x7f01b40a42d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f01c00095f0 tx=0x7f01c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:18.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.433+0000 7f01c97fa640 1 -- 172.21.15.71:0/2055911321 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f01bc0bc6c0 con 0x7f01cc1004f0 2024-03-04T08:58:18.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.609+0000 7f01d2a22640 1 -- 172.21.15.71:0/2055911321 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01cc02d210 con 0x7f01cc1004f0 2024-03-04T08:58:18.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.610+0000 7f01c97fa640 1 -- 172.21.15.71:0/2055911321 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24624 (secure 0 0 0) 0x7f01bc0c0760 con 0x7f01cc1004f0 2024-03-04T08:58:18.606 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:58:18.607 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":93,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","created":"2024-03-04T08:47:28.546102+0000","modified":"2024-03-04T08:57:23.805294+0000","last_up_change":"2024-03-04T08:55:30.744961+0000","last_in_change":"2024-03-04T08:55:11.570990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T08:51:14.933487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T08:56:28.253074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T08:56:29.543622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T08:56:32.884220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7e43b47d-8888-4631-bc96-8a96fcf8cd46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6803","nonce":1173641530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6805","nonce":1173641530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6809","nonce":1173641530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6807","nonce":1173641530}]},"public_addr":"172.21.15.71:6803/1173641530","cluster_addr":"172.21.15.71:6805/1173641530","heartbeat_back_addr":"172.21.15.71:6809/1173641530","heartbeat_front_addr":"172.21.15.71:6807/1173641530","state":["exists","up"]},{"osd":1,"uuid":"1209f214-f2e2-4209-bf69-4528a43a2123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6801","nonce":3383948110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6803","nonce":3383948110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6807","nonce":3383948110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6805","nonce":3383948110}]},"public_addr":"172.21.15.150:6801/3383948110","cluster_addr":"172.21.15.150:6803/3383948110","heartbeat_back_addr":"172.21.15.150:6807/3383948110","heartbeat_front_addr":"172.21.15.150:6805/3383948110","state":["exists","up"]},{"osd":2,"uuid":"b88c3082-93a9-4c9b-9221-a7105a44c91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6801","nonce":586944144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6803","nonce":586944144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6807","nonce":586944144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6805","nonce":586944144}]},"public_addr":"172.21.15.158:6801/586944144","cluster_addr":"172.21.15.158:6803/586944144","heartbeat_back_addr":"172.21.15.158:6807/586944144","heartbeat_front_addr":"172.21.15.158:6805/586944144","state":["exists","up"]},{"osd":3,"uuid":"e4fc7723-fd73-40b2-b9d0-7f9bda282ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6811","nonce":1311903262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6813","nonce":1311903262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6817","nonce":1311903262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6815","nonce":1311903262}]},"public_addr":"172.21.15.71:6811/1311903262","cluster_addr":"172.21.15.71:6813/1311903262","heartbeat_back_addr":"172.21.15.71:6817/1311903262","heartbeat_front_addr":"172.21.15.71:6815/1311903262","state":["exists","up"]},{"osd":4,"uuid":"e7eb72ac-f066-4ccd-b2bf-22a6a26daf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6809","nonce":2300837573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6811","nonce":2300837573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6815","nonce":2300837573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6813","nonce":2300837573}]},"public_addr":"172.21.15.150:6809/2300837573","cluster_addr":"172.21.15.150:6811/2300837573","heartbeat_back_addr":"172.21.15.150:6815/2300837573","heartbeat_front_addr":"172.21.15.150:6813/2300837573","state":["exists","up"]},{"osd":5,"uuid":"9b295840-e104-4a66-9cc4-7020627c1765","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6809","nonce":1922694045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6811","nonce":1922694045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6815","nonce":1922694045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6813","nonce":1922694045}]},"public_addr":"172.21.15.158:6809/1922694045","cluster_addr":"172.21.15.158:6811/1922694045","heartbeat_back_addr":"172.21.15.158:6815/1922694045","heartbeat_front_addr":"172.21.15.158:6813/1922694045","state":["exists","up"]},{"osd":6,"uuid":"6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6819","nonce":3848842707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6821","nonce":3848842707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6825","nonce":3848842707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6823","nonce":3848842707}]},"public_addr":"172.21.15.71:6819/3848842707","cluster_addr":"172.21.15.71:6821/3848842707","heartbeat_back_addr":"172.21.15.71:6825/3848842707","heartbeat_front_addr":"172.21.15.71:6823/3848842707","state":["exists","up"]},{"osd":7,"uuid":"6160a40a-d30f-4569-949c-0afb1bb3d133","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6817","nonce":1328330305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6819","nonce":1328330305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6823","nonce":1328330305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6821","nonce":1328330305}]},"public_addr":"172.21.15.150:6817/1328330305","cluster_addr":"172.21.15.150:6819/1328330305","heartbeat_back_addr":"172.21.15.150:6823/1328330305","heartbeat_front_addr":"172.21.15.150:6821/1328330305","state":["exists","up"]},{"osd":8,"uuid":"95ed2341-84a4-465c-9948-ac667680ce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6817","nonce":2195304577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6819","nonce":2195304577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6823","nonce":2195304577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6821","nonce":2195304577}]},"public_addr":"172.21.15.158:6817/2195304577","cluster_addr":"172.21.15.158:6819/2195304577","heartbeat_back_addr":"172.21.15.158:6823/2195304577","heartbeat_front_addr":"172.21.15.158:6821/2195304577","state":["exists","up"]},{"osd":9,"uuid":"088f6419-63a0-4768-afae-dbb8a5f5a565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6827","nonce":1913779347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6829","nonce":1913779347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6833","nonce":1913779347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6831","nonce":1913779347}]},"public_addr":"172.21.15.71:6827/1913779347","cluster_addr":"172.21.15.71:6829/1913779347","heartbeat_back_addr":"172.21.15.71:6833/1913779347","heartbeat_front_addr":"172.21.15.71:6831/1913779347","state":["exists","up"]},{"osd":10,"uuid":"18c85e24-068d-4f1e-a3b8-f7915b363234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6825","nonce":3295420996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6827","nonce":3295420996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6831","nonce":3295420996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6829","nonce":3295420996}]},"public_addr":"172.21.15.150:6825/3295420996","cluster_addr":"172.21.15.150:6827/3295420996","heartbeat_back_addr":"172.21.15.150:6831/3295420996","heartbeat_front_addr":"172.21.15.150:6829/3295420996","state":["exists","up"]},{"osd":11,"uuid":"4f6b3ddf-c482-4740-b1fd-de836e047c2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6825","nonce":3332083865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6827","nonce":3332083865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6831","nonce":3332083865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6829","nonce":3332083865}]},"public_addr":"172.21.15.158:6825/3332083865","cluster_addr":"172.21.15.158:6827/3332083865","heartbeat_back_addr":"172.21.15.158:6831/3332083865","heartbeat_front_addr":"172.21.15.158:6829/3332083865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:05.997832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:29.314235+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:08.819465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:41.194275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:06.032479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:38.814498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:13.067960+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:38.568479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:03.416713+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:38.349479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:04.096945+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:29.425250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1287712358":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/3770452860":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/718743094":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/1904717532":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:6800/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6800/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/2862184313":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/3621754630":"2024-03-05T08:48:22.593931+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T08:58:18.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.612+0000 7f01d2a22640 1 -- 172.21.15.71:0/2055911321 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f01b40a1e10 msgr2=0x7f01b40a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:18.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.612+0000 7f01d2a22640 1 --2- 172.21.15.71:0/2055911321 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f01b40a1e10 0x7f01b40a42d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f01c00095f0 tx=0x7f01c0005f90 comp rx=0 tx=0).stop 2024-03-04T08:58:18.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.612+0000 7f01d2a22640 1 -- 172.21.15.71:0/2055911321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01cc1004f0 msgr2=0x7f01cc196800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:18.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.612+0000 7f01d2a22640 1 --2- 172.21.15.71:0/2055911321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01cc1004f0 0x7f01cc196800 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f01bc000c00 tx=0x7f01bc02c640 comp rx=0 tx=0).stop 2024-03-04T08:58:18.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.612+0000 7f01d2a22640 1 -- 172.21.15.71:0/2055911321 shutdown_connections 2024-03-04T08:58:18.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.612+0000 7f01d2a22640 1 --2- 172.21.15.71:0/2055911321 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f01b40a1e10 0x7f01b40a42d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.612+0000 7f01d2a22640 1 --2- 172.21.15.71:0/2055911321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01cc1020a0 0x7f01cc19ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.612+0000 7f01d2a22640 1 --2- 172.21.15.71:0/2055911321 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f01cc1016e0 0x7f01cc196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.612+0000 7f01d2a22640 1 --2- 172.21.15.71:0/2055911321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01cc1004f0 0x7f01cc196800 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.612+0000 7f01d2a22640 1 -- 172.21.15.71:0/2055911321 >> 172.21.15.71:0/2055911321 conn(0x7f01cc0fba60 msgr2=0x7f01cc104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:18.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.612+0000 7f01d2a22640 1 -- 172.21.15.71:0/2055911321 shutdown_connections 2024-03-04T08:58:18.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.612+0000 7f01d2a22640 1 -- 172.21.15.71:0/2055911321 wait complete. 2024-03-04T08:58:18.620 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:58:18.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.727+0000 7f6a3793a640 1 -- 172.21.15.71:0/265527925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a30101700 msgr2=0x7f6a30101b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:18.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.727+0000 7f6a3793a640 1 --2- 172.21.15.71:0/265527925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a30101700 0x7f6a30101b80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009900 tx=0x7f6a2002bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:18.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.727+0000 7f6a3793a640 1 -- 172.21.15.71:0/265527925 shutdown_connections 2024-03-04T08:58:18.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.727+0000 7f6a3793a640 1 --2- 172.21.15.71:0/265527925 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a301020c0 0x7f6a30108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.727+0000 7f6a3793a640 1 --2- 172.21.15.71:0/265527925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a30101700 0x7f6a30101b80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.727+0000 7f6a3793a640 1 --2- 172.21.15.71:0/265527925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a30100510 0x7f6a30100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.727+0000 7f6a3793a640 1 -- 172.21.15.71:0/265527925 >> 172.21.15.71:0/265527925 conn(0x7f6a300fba80 msgr2=0x7f6a300fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:18.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.727+0000 7f6a3793a640 1 -- 172.21.15.71:0/265527925 shutdown_connections 2024-03-04T08:58:18.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.727+0000 7f6a3793a640 1 -- 172.21.15.71:0/265527925 wait complete. 2024-03-04T08:58:18.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.727+0000 7f6a3793a640 1 Processor -- start 2024-03-04T08:58:18.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.727+0000 7f6a3793a640 1 -- start start 2024-03-04T08:58:18.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.728+0000 7f6a3793a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a30100510 0x7f6a30197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:18.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.728+0000 7f6a3793a640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a30101700 0x7f6a30197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:18.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.728+0000 7f6a3793a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a301020c0 0x7f6a3019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:18.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.728+0000 7f6a3793a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3010d2b0 con 0x7f6a30100510 2024-03-04T08:58:18.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.728+0000 7f6a3793a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3010d130 con 0x7f6a301020c0 2024-03-04T08:58:18.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.728+0000 7f6a3793a640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3010d430 con 0x7f6a30101700 2024-03-04T08:58:18.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.728+0000 7f6a356d6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a30100510 0x7f6a30197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:18.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.728+0000 7f6a356d6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a30100510 0x7f6a30197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36138/0 (socket says 172.21.15.71:36138) 2024-03-04T08:58:18.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.728+0000 7f6a356d6640 1 -- 172.21.15.71:0/3218078197 learned_addr learned my addr 172.21.15.71:0/3218078197 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:18.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.728+0000 7f6a35ed7640 1 --2- 172.21.15.71:0/3218078197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a301020c0 0x7f6a3019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:18.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.728+0000 7f6a34ed5640 1 --2- 172.21.15.71:0/3218078197 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a30101700 0x7f6a30197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:18.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.728+0000 7f6a356d6640 1 -- 172.21.15.71:0/3218078197 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a30101700 msgr2=0x7f6a30197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:18.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.728+0000 7f6a356d6640 1 --2- 172.21.15.71:0/3218078197 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a30101700 0x7f6a30197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.728+0000 7f6a356d6640 1 -- 172.21.15.71:0/3218078197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a301020c0 msgr2=0x7f6a3019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:18.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.728+0000 7f6a356d6640 1 --2- 172.21.15.71:0/3218078197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a301020c0 0x7f6a3019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.728+0000 7f6a356d6640 1 -- 172.21.15.71:0/3218078197 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3019f520 con 0x7f6a30100510 2024-03-04T08:58:18.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.728+0000 7f6a356d6640 1 --2- 172.21.15.71:0/3218078197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a30100510 0x7f6a30197900 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00e9c0 tx=0x7f6a2c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:18.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.728+0000 7f6a267fc640 1 -- 172.21.15.71:0/3218078197 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c00cde0 con 0x7f6a30100510 2024-03-04T08:58:18.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.728+0000 7f6a267fc640 1 -- 172.21.15.71:0/3218078197 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a2c010910 con 0x7f6a30100510 2024-03-04T08:58:18.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.728+0000 7f6a3793a640 1 -- 172.21.15.71:0/3218078197 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a3019f810 con 0x7f6a30100510 2024-03-04T08:58:18.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.728+0000 7f6a267fc640 1 -- 172.21.15.71:0/3218078197 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c018a00 con 0x7f6a30100510 2024-03-04T08:58:18.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.728+0000 7f6a3793a640 1 -- 172.21.15.71:0/3218078197 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a3019fdb0 con 0x7f6a30100510 2024-03-04T08:58:18.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.729+0000 7f6a3793a640 1 -- 172.21.15.71:0/3218078197 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a30107830 con 0x7f6a30100510 2024-03-04T08:58:18.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.730+0000 7f6a267fc640 1 -- 172.21.15.71:0/3218078197 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f6a2c01f020 con 0x7f6a30100510 2024-03-04T08:58:18.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.731+0000 7f6a267fc640 1 --2- 172.21.15.71:0/3218078197 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6a1c0a1e90 0x7f6a1c0a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:18.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.731+0000 7f6a34ed5640 1 --2- 172.21.15.71:0/3218078197 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6a1c0a1e90 0x7f6a1c0a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:18.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.731+0000 7f6a267fc640 1 -- 172.21.15.71:0/3218078197 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f6a2c0ceb30 con 0x7f6a30100510 2024-03-04T08:58:18.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.731+0000 7f6a34ed5640 1 --2- 172.21.15.71:0/3218078197 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6a1c0a1e90 0x7f6a1c0a4350 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6a20036040 tx=0x7f6a20005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:18.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.734+0000 7f6a267fc640 1 -- 172.21.15.71:0/3218078197 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6a2c09baa0 con 0x7f6a30100510 2024-03-04T08:58:18.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:18 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1593945244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T08:58:18.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:18 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2944752809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:58:18.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:18 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2055911321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:58:18.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.950+0000 7f6a3793a640 1 -- 172.21.15.71:0/3218078197 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a3002cff0 con 0x7f6a30100510 2024-03-04T08:58:18.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.950+0000 7f6a267fc640 1 -- 172.21.15.71:0/3218078197 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11339 (secure 0 0 0) 0x7f6a2c09b3a0 con 0x7f6a30100510 2024-03-04T08:58:18.946 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:58:18.947 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3975379807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3975379807},{"type":"v1","addr":"172.21.15.71:6835","nonce":3975379807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1962929347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1962929347},{"type":"v1","addr":"172.21.15.71:6837","nonce":1962929347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/1459252176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1459252176},{"type":"v1","addr":"172.21.15.71:6839","nonce":1459252176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6841/1083546600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":1083546600},{"type":"v1","addr":"172.21.15.71:6841","nonce":1083546600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/178179528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":178179528},{"type":"v1","addr":"172.21.15.158:6833","nonce":178179528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2468877675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2468877675},{"type":"v1","addr":"172.21.15.150:6833","nonce":2468877675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1652094309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1652094309},{"type":"v1","addr":"172.21.15.158:6835","nonce":1652094309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1802416462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1802416462},{"type":"v1","addr":"172.21.15.150:6835","nonce":1802416462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/617058380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":617058380},{"type":"v1","addr":"172.21.15.158:6837","nonce":617058380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/913465600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":913465600},{"type":"v1","addr":"172.21.15.150:6837","nonce":913465600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T08:56:30.343357+0000","modified":"2024-03-04T08:56:54.901214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.158:6839/2921704804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6838","nonce":2921704804},{"type":"v1","addr":"172.21.15.158:6839","nonce":2921704804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/2639278083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2639278083},{"type":"v1","addr":"172.21.15.150:6839","nonce":2639278083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491]},"id":1}]} 2024-03-04T08:58:18.947 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 20 2024-03-04T08:58:18.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.952+0000 7f6a3793a640 1 -- 172.21.15.71:0/3218078197 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6a1c0a1e90 msgr2=0x7f6a1c0a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:18.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.952+0000 7f6a3793a640 1 --2- 172.21.15.71:0/3218078197 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6a1c0a1e90 0x7f6a1c0a4350 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6a20036040 tx=0x7f6a20005fd0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.952+0000 7f6a3793a640 1 -- 172.21.15.71:0/3218078197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a30100510 msgr2=0x7f6a30197900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:18.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.952+0000 7f6a3793a640 1 --2- 172.21.15.71:0/3218078197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a30100510 0x7f6a30197900 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00e9c0 tx=0x7f6a2c00ee80 comp rx=0 tx=0).stop 2024-03-04T08:58:18.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.952+0000 7f6a3793a640 1 -- 172.21.15.71:0/3218078197 shutdown_connections 2024-03-04T08:58:18.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.952+0000 7f6a3793a640 1 --2- 172.21.15.71:0/3218078197 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6a1c0a1e90 0x7f6a1c0a4350 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.952+0000 7f6a3793a640 1 --2- 172.21.15.71:0/3218078197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a301020c0 0x7f6a3019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.952+0000 7f6a3793a640 1 --2- 172.21.15.71:0/3218078197 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6a30101700 0x7f6a30197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.952+0000 7f6a3793a640 1 --2- 172.21.15.71:0/3218078197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a30100510 0x7f6a30197900 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:18.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.952+0000 7f6a3793a640 1 -- 172.21.15.71:0/3218078197 >> 172.21.15.71:0/3218078197 conn(0x7f6a300fba80 msgr2=0x7f6a30104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:18.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.952+0000 7f6a3793a640 1 -- 172.21.15.71:0/3218078197 shutdown_connections 2024-03-04T08:58:18.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:18.953+0000 7f6a3793a640 1 -- 172.21.15.71:0/3218078197 wait complete. 2024-03-04T08:58:18.960 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-04T08:56:30.343357+0000', 'modified': '2024-03-04T08:56:54.901214+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24491}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.158:6839/2921704804', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.158:6838', 'nonce': 2921704804}, {'type': 'v1', 'addr': '172.21.15.158:6839', 'nonce': 2921704804}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24491': {'gid': 24491, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6839/2639278083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6838', 'nonce': 2639278083}, {'type': 'v1', 'addr': '172.21.15.150:6839', 'nonce': 2639278083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24491, 'qdb_cluster': [24491]} 2024-03-04T08:58:18.960 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-04T08:58:18.960 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-04T08:58:19.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.070+0000 7f9134e88640 1 -- 172.21.15.71:0/999021547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91301004d0 msgr2=0x7f91301008f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:19.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.070+0000 7f9134e88640 1 --2- 172.21.15.71:0/999021547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91301004d0 0x7f91301008f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9120009900 tx=0x7f912002bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:19.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.070+0000 7f9134e88640 1 -- 172.21.15.71:0/999021547 shutdown_connections 2024-03-04T08:58:19.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.070+0000 7f9134e88640 1 --2- 172.21.15.71:0/999021547 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9130102080 0x7f9130108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.070+0000 7f9134e88640 1 --2- 172.21.15.71:0/999021547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91301016c0 0x7f9130101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.070+0000 7f9134e88640 1 --2- 172.21.15.71:0/999021547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91301004d0 0x7f91301008f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.070+0000 7f9134e88640 1 -- 172.21.15.71:0/999021547 >> 172.21.15.71:0/999021547 conn(0x7f91300fba60 msgr2=0x7f91300fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:19.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.070+0000 7f9134e88640 1 -- 172.21.15.71:0/999021547 shutdown_connections 2024-03-04T08:58:19.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.070+0000 7f9134e88640 1 -- 172.21.15.71:0/999021547 wait complete. 2024-03-04T08:58:19.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.071+0000 7f9134e88640 1 Processor -- start 2024-03-04T08:58:19.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.071+0000 7f9134e88640 1 -- start start 2024-03-04T08:58:19.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.071+0000 7f9134e88640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91301004d0 0x7f9130104460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:19.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.071+0000 7f9134e88640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f91301016c0 0x7f9130102aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:19.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.071+0000 7f9134e88640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9130102080 0x7f9130102fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:19.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.071+0000 7f9134e88640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913010c160 con 0x7f9130102080 2024-03-04T08:58:19.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.071+0000 7f9134e88640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913010bfe0 con 0x7f91301004d0 2024-03-04T08:58:19.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.071+0000 7f9134e88640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913010c2e0 con 0x7f91301016c0 2024-03-04T08:58:19.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.071+0000 7f912ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9130102080 0x7f9130102fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:19.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.071+0000 7f912e59c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91301004d0 0x7f9130104460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:19.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.071+0000 7f912ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9130102080 0x7f9130102fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36168/0 (socket says 172.21.15.71:36168) 2024-03-04T08:58:19.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.071+0000 7f912ed9d640 1 -- 172.21.15.71:0/2410956097 learned_addr learned my addr 172.21.15.71:0/2410956097 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:19.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.071+0000 7f9127fff640 1 --2- 172.21.15.71:0/2410956097 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f91301016c0 0x7f9130102aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:19.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.071+0000 7f912e59c640 1 -- 172.21.15.71:0/2410956097 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f91301016c0 msgr2=0x7f9130102aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:19.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.071+0000 7f912e59c640 1 --2- 172.21.15.71:0/2410956097 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f91301016c0 0x7f9130102aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.071+0000 7f912e59c640 1 -- 172.21.15.71:0/2410956097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9130102080 msgr2=0x7f9130102fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:19.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.071+0000 7f912e59c640 1 --2- 172.21.15.71:0/2410956097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9130102080 0x7f9130102fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.071+0000 7f912e59c640 1 -- 172.21.15.71:0/2410956097 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9130103870 con 0x7f91301004d0 2024-03-04T08:58:19.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.071+0000 7f912ed9d640 1 --2- 172.21.15.71:0/2410956097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9130102080 0x7f9130102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:58:19.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.072+0000 7f912e59c640 1 --2- 172.21.15.71:0/2410956097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91301004d0 0x7f9130104460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f912002c170 tx=0x7f912002c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:19.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.072+0000 7f91277fe640 1 -- 172.21.15.71:0/2410956097 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9120041070 con 0x7f91301004d0 2024-03-04T08:58:19.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.072+0000 7f91277fe640 1 -- 172.21.15.71:0/2410956097 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9120004450 con 0x7f91301004d0 2024-03-04T08:58:19.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.072+0000 7f9134e88640 1 -- 172.21.15.71:0/2410956097 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91301a19d0 con 0x7f91301004d0 2024-03-04T08:58:19.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.072+0000 7f91277fe640 1 -- 172.21.15.71:0/2410956097 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9120034440 con 0x7f91301004d0 2024-03-04T08:58:19.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.072+0000 7f9134e88640 1 -- 172.21.15.71:0/2410956097 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91301a1ef0 con 0x7f91301004d0 2024-03-04T08:58:19.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.074+0000 7f91277fe640 1 -- 172.21.15.71:0/2410956097 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f9120044070 con 0x7f91301004d0 2024-03-04T08:58:19.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.075+0000 7f91277fe640 1 --2- 172.21.15.71:0/2410956097 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f911c0a1e10 0x7f911c0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:19.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.075+0000 7f91277fe640 1 -- 172.21.15.71:0/2410956097 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f91200efd70 con 0x7f91301004d0 2024-03-04T08:58:19.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.075+0000 7f9127fff640 1 --2- 172.21.15.71:0/2410956097 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f911c0a1e10 0x7f911c0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:19.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.075+0000 7f9137a78740 1 Processor -- start 2024-03-04T08:58:19.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.075+0000 7f9137a78740 1 -- start start 2024-03-04T08:58:19.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.075+0000 7f9127fff640 1 --2- 172.21.15.71:0/2410956097 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f911c0a1e10 0x7f911c0a42d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f91180095f0 tx=0x7f9118005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:19.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.075+0000 7f9137a78740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559a7e1827b0 0x559a7e182bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:19.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.075+0000 7f9137a78740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a7e183980 0x559a7e183e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:19.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.075+0000 7f9137a78740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e184340 0x559a7e18abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:19.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.075+0000 7f9137a78740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7e0e76b0 con 0x559a7e184340 2024-03-04T08:58:19.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.075+0000 7f9137a78740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7e090a60 con 0x559a7e183980 2024-03-04T08:58:19.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.075+0000 7f9137a78740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7e0e3750 con 0x559a7e1827b0 2024-03-04T08:58:19.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.076+0000 7f912ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e184340 0x559a7e18abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:19.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.076+0000 7f9127fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a7e183980 0x559a7e183e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:19.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.076+0000 7f912ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e184340 0x559a7e18abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36178/0 (socket says 172.21.15.71:36178) 2024-03-04T08:58:19.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.076+0000 7f912ed9d640 1 -- 172.21.15.71:0/1548065636 learned_addr learned my addr 172.21.15.71:0/1548065636 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:19.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.076+0000 7f912e59c640 1 --2- 172.21.15.71:0/1548065636 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559a7e1827b0 0x559a7e182bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:19.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.076+0000 7f9127fff640 1 -- 172.21.15.71:0/1548065636 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559a7e1827b0 msgr2=0x559a7e182bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:58:19.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.076+0000 7f9127fff640 1 --2- 172.21.15.71:0/1548065636 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559a7e1827b0 0x559a7e182bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.076+0000 7f9127fff640 1 -- 172.21.15.71:0/1548065636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e184340 msgr2=0x559a7e18abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:58:19.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.076+0000 7f9127fff640 1 --2- 172.21.15.71:0/1548065636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e184340 0x559a7e18abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.076+0000 7f9127fff640 1 -- 172.21.15.71:0/1548065636 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a7e0aef30 con 0x559a7e183980 2024-03-04T08:58:19.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.076+0000 7f9127fff640 1 --2- 172.21.15.71:0/1548065636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a7e183980 0x559a7e183e00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9118011eb0 tx=0x7f9118014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24335e9d404458d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:19.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.076+0000 7f9124ff9640 1 -- 172.21.15.71:0/1548065636 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118012e60 con 0x559a7e183980 2024-03-04T08:58:19.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.076+0000 7f9124ff9640 1 -- 172.21.15.71:0/1548065636 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f911801fb00 con 0x559a7e183980 2024-03-04T08:58:19.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.076+0000 7f9124ff9640 1 -- 172.21.15.71:0/1548065636 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118027c70 con 0x559a7e183980 2024-03-04T08:58:19.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.077+0000 7f9137a78740 1 -- 172.21.15.71:0/1548065636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a7e183980 msgr2=0x559a7e183e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:19.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.077+0000 7f9137a78740 1 --2- 172.21.15.71:0/1548065636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a7e183980 0x559a7e183e00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9118011eb0 tx=0x7f9118014000 comp rx=0 tx=0).stop 2024-03-04T08:58:19.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.077+0000 7f9137a78740 1 -- 172.21.15.71:0/1548065636 shutdown_connections 2024-03-04T08:58:19.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.077+0000 7f9137a78740 1 --2- 172.21.15.71:0/1548065636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e184340 0x559a7e18abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.077+0000 7f9137a78740 1 --2- 172.21.15.71:0/1548065636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a7e183980 0x559a7e183e00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.077+0000 7f9137a78740 1 --2- 172.21.15.71:0/1548065636 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559a7e1827b0 0x559a7e182bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.077+0000 7f9137a78740 1 -- 172.21.15.71:0/1548065636 >> 172.21.15.71:0/1548065636 conn(0x559a7e17dba0 msgr2=0x559a7e17dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:19.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.077+0000 7f9137a78740 1 -- 172.21.15.71:0/1548065636 shutdown_connections 2024-03-04T08:58:19.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.077+0000 7f9137a78740 1 -- 172.21.15.71:0/1548065636 wait complete. 2024-03-04T08:58:19.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.077+0000 7f9137a78740 1 -- start start 2024-03-04T08:58:19.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.078+0000 7f9137a78740 1 Processor -- start 2024-03-04T08:58:19.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.078+0000 7f9137a78740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e1827b0 0x559a7e18cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:19.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.078+0000 7f9137a78740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a7e183980 0x559a7e18d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:19.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.078+0000 7f9137a78740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559a7e184340 0x559a7e1942a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:19.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.078+0000 7f9137a78740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7e18b350 con 0x559a7e1827b0 2024-03-04T08:58:19.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.078+0000 7f9137a78740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7e0a3ea0 con 0x559a7e183980 2024-03-04T08:58:19.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.078+0000 7f9137a78740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7e18b4d0 con 0x559a7e184340 2024-03-04T08:58:19.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.078+0000 7f9137a78740 10 client.0 _ll_register_callbacks cb 0x559a7e0a3870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T08:58:19.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.078+0000 7f912e59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e1827b0 0x559a7e18cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:19.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.078+0000 7f912e59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e1827b0 0x559a7e18cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36182/0 (socket says 172.21.15.71:36182) 2024-03-04T08:58:19.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.078+0000 7f912e59c640 1 -- 172.21.15.71:0/1344606141 learned_addr learned my addr 172.21.15.71:0/1344606141 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:19.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.078+0000 7f9127fff640 1 --2- 172.21.15.71:0/1344606141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a7e183980 0x559a7e18d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:19.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.078+0000 7f912ed9d640 1 --2- 172.21.15.71:0/1344606141 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559a7e184340 0x559a7e1942a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:19.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.078+0000 7f9127fff640 1 -- 172.21.15.71:0/1344606141 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559a7e184340 msgr2=0x559a7e1942a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:58:19.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.078+0000 7f9127fff640 1 --2- 172.21.15.71:0/1344606141 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559a7e184340 0x559a7e1942a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.078+0000 7f9127fff640 1 -- 172.21.15.71:0/1344606141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e1827b0 msgr2=0x559a7e18cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:58:19.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.078+0000 7f9127fff640 1 --2- 172.21.15.71:0/1344606141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e1827b0 0x559a7e18cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.078+0000 7f9127fff640 1 -- 172.21.15.71:0/1344606141 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a7e0a4270 con 0x559a7e183980 2024-03-04T08:58:19.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.078+0000 7f912e59c640 1 --2- 172.21.15.71:0/1344606141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e1827b0 0x559a7e18cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:58:19.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.079+0000 7f9127fff640 1 --2- 172.21.15.71:0/1344606141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a7e183980 0x559a7e18d260 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9118026000 tx=0x7f9118028c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4934b90ea3f13b61 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:19.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.079+0000 7f9115ffb640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-03-04T08:58:19.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.079+0000 7f9115ffb640 1 -- 172.21.15.71:0/1344606141 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118012e60 con 0x559a7e183980 2024-03-04T08:58:19.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.079+0000 7f9115ffb640 1 -- 172.21.15.71:0/1344606141 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f911801f630 con 0x559a7e183980 2024-03-04T08:58:19.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.079+0000 7f9137a78740 1 -- 172.21.15.71:0/1344606141 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f911801fb00 con 0x559a7e183980 2024-03-04T08:58:19.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.079+0000 7f9115ffb640 1 -- 172.21.15.71:0/1344606141 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9118040050 con 0x559a7e183980 2024-03-04T08:58:19.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.079+0000 7f9137a78740 10 client.25055 fetch_fsmap learned FSMap version 20 2024-03-04T08:58:19.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.079+0000 7f9137a78740 1 -- 172.21.15.71:0/1344606141 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x559a7ddfc2d0 con 0x559a7e183980 2024-03-04T08:58:19.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.079+0000 7f9115ffb640 1 -- 172.21.15.71:0/1344606141 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f911801fb00 con 0x559a7e183980 2024-03-04T08:58:19.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.079+0000 7f9115ffb640 1 -- 172.21.15.71:0/1344606141 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 20) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f91180048c0 con 0x559a7e183980 2024-03-04T08:58:19.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.079+0000 7f9137a78740 10 client.25055 fetch_fsmap finished waiting for FSMap version 20 2024-03-04T08:58:19.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.079+0000 7f9137a78740 10 client.25055 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-04T08:58:19.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.079+0000 7f9137a78740 20 client.25055 populate_metadata read hostname 'smithi071' 2024-03-04T08:58:19.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.079+0000 7f9137a78740 1 --2- 172.21.15.71:0/1344606141 >> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] conn(0x559a7e198840 0x559a7e19d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:19.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.079+0000 7f9137a78740 4 client.25055 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-04T08:58:19.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.079+0000 7f9137a78740 1 -- 172.21.15.71:0/1344606141 --> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a7e0f4bf0 con 0x559a7e198840 2024-03-04T08:58:19.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.080+0000 7f912e59c640 1 --2- 172.21.15.71:0/1344606141 >> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] conn(0x559a7e198840 0x559a7e19d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:19.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.080+0000 7f912e59c640 1 --2- 172.21.15.71:0/1344606141 >> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] conn(0x559a7e198840 0x559a7e19d8b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e5648e5ac3f87ba server_cookie=fb1799caa8892960 in_seq=0 out_seq=0 2024-03-04T08:58:19.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.080+0000 7f9115ffb640 10 client.25055 ms_handle_connect on v2:172.21.15.150:6838/2639278083 2024-03-04T08:58:19.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.081+0000 7f9115ffb640 1 -- 172.21.15.71:0/1344606141 <== mds.0 v2:172.21.15.150:6838/2639278083 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559a7e0f4bf0 con 0x559a7e198840 2024-03-04T08:58:19.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.081+0000 7f9115ffb640 10 client.25055 handle_command_reply: tid=0 2024-03-04T08:58:19.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.081+0000 7f9137a78740 1 client.25055 shutdown 2024-03-04T08:58:19.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.081+0000 7f9137a78740 1 -- 172.21.15.71:0/1344606141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a7e183980 msgr2=0x559a7e18d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:19.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.081+0000 7f9137a78740 1 --2- 172.21.15.71:0/1344606141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a7e183980 0x559a7e18d260 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9118026000 tx=0x7f9118028c50 comp rx=0 tx=0).stop 2024-03-04T08:58:19.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.081+0000 7f9137a78740 1 -- 172.21.15.71:0/1344606141 shutdown_connections 2024-03-04T08:58:19.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.081+0000 7f9137a78740 1 --2- 172.21.15.71:0/1344606141 >> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] conn(0x559a7e198840 0x559a7e19d8b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.081+0000 7f9137a78740 1 --2- 172.21.15.71:0/1344606141 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559a7e184340 0x559a7e1942a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.081+0000 7f9115ffb640 0 client.25055 ms_handle_reset on v2:172.21.15.150:6838/2639278083 2024-03-04T08:58:19.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.081+0000 7f9137a78740 1 --2- 172.21.15.71:0/1344606141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a7e183980 0x559a7e18d260 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.081+0000 7f9137a78740 1 --2- 172.21.15.71:0/1344606141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e1827b0 0x559a7e18cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.081+0000 7f9137a78740 1 -- 172.21.15.71:0/1344606141 >> 172.21.15.71:0/1344606141 conn(0x559a7e17dba0 msgr2=0x559a7e17cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:19.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.081+0000 7f9137a78740 1 -- 172.21.15.71:0/1344606141 shutdown_connections 2024-03-04T08:58:19.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.081+0000 7f9137a78740 1 -- 172.21.15.71:0/1344606141 wait complete. 2024-03-04T08:58:19.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.081+0000 7f9137a78740 20 client.25055 trim_cache size 0 max 16384 2024-03-04T08:58:19.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.098+0000 7f9137a78740 1 Processor -- start 2024-03-04T08:58:19.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.098+0000 7f9137a78740 1 -- start start 2024-03-04T08:58:19.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.098+0000 7f9137a78740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a7e1827b0 0x559a7e18d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:19.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.098+0000 7f9137a78740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559a7e183980 0x559a7e194260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:19.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.098+0000 7f9137a78740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e184340 0x559a7e17c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:19.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.098+0000 7f9137a78740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7e198c50 con 0x559a7e184340 2024-03-04T08:58:19.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.098+0000 7f9137a78740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7e1992b0 con 0x559a7e1827b0 2024-03-04T08:58:19.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.098+0000 7f9137a78740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7e199910 con 0x559a7e183980 2024-03-04T08:58:19.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.099+0000 7f912ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e184340 0x559a7e17c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:19.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.099+0000 7f912e59c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a7e1827b0 0x559a7e18d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:19.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.099+0000 7f912ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e184340 0x559a7e17c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36198/0 (socket says 172.21.15.71:36198) 2024-03-04T08:58:19.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.099+0000 7f912ed9d640 1 -- 172.21.15.71:0/1374647138 learned_addr learned my addr 172.21.15.71:0/1374647138 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:19.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.099+0000 7f9127fff640 1 --2- 172.21.15.71:0/1374647138 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559a7e183980 0x559a7e194260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:19.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.099+0000 7f912e59c640 1 -- 172.21.15.71:0/1374647138 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559a7e183980 msgr2=0x559a7e194260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:58:19.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.099+0000 7f912e59c640 1 --2- 172.21.15.71:0/1374647138 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559a7e183980 0x559a7e194260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.099+0000 7f912e59c640 1 -- 172.21.15.71:0/1374647138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e184340 msgr2=0x559a7e17c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:58:19.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.099+0000 7f912e59c640 1 --2- 172.21.15.71:0/1374647138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e184340 0x559a7e17c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.099+0000 7f912e59c640 1 -- 172.21.15.71:0/1374647138 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a7dec66b0 con 0x559a7e1827b0 2024-03-04T08:58:19.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.099+0000 7f912ed9d640 1 --2- 172.21.15.71:0/1374647138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e184340 0x559a7e17c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:58:19.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.099+0000 7f9127fff640 1 --2- 172.21.15.71:0/1374647138 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559a7e183980 0x559a7e194260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:58:19.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.099+0000 7f912e59c640 1 --2- 172.21.15.71:0/1374647138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a7e1827b0 0x559a7e18d040 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9120009900 tx=0x7f91200dc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=348de100cb27467f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:19.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.099+0000 7f9117fff640 1 -- 172.21.15.71:0/1374647138 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91200dae60 con 0x559a7e1827b0 2024-03-04T08:58:19.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.099+0000 7f9117fff640 1 -- 172.21.15.71:0/1374647138 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91200c0840 con 0x559a7e1827b0 2024-03-04T08:58:19.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.099+0000 7f9117fff640 1 -- 172.21.15.71:0/1374647138 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912004b040 con 0x559a7e1827b0 2024-03-04T08:58:19.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.100+0000 7f9137a78740 1 -- 172.21.15.71:0/1374647138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a7e1827b0 msgr2=0x559a7e18d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:19.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.100+0000 7f9137a78740 1 --2- 172.21.15.71:0/1374647138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a7e1827b0 0x559a7e18d040 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9120009900 tx=0x7f91200dc040 comp rx=0 tx=0).stop 2024-03-04T08:58:19.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.100+0000 7f9137a78740 1 -- 172.21.15.71:0/1374647138 shutdown_connections 2024-03-04T08:58:19.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.100+0000 7f9137a78740 1 --2- 172.21.15.71:0/1374647138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e184340 0x559a7e17c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.100+0000 7f9137a78740 1 --2- 172.21.15.71:0/1374647138 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559a7e183980 0x559a7e194260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.100+0000 7f9137a78740 1 --2- 172.21.15.71:0/1374647138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a7e1827b0 0x559a7e18d040 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.100+0000 7f9137a78740 1 -- 172.21.15.71:0/1374647138 >> 172.21.15.71:0/1374647138 conn(0x559a7e17dba0 msgr2=0x559a7e18c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:19.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.100+0000 7f9137a78740 1 -- 172.21.15.71:0/1374647138 shutdown_connections 2024-03-04T08:58:19.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.100+0000 7f9137a78740 1 -- 172.21.15.71:0/1374647138 wait complete. 2024-03-04T08:58:19.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.101+0000 7f9137a78740 1 -- start start 2024-03-04T08:58:19.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.101+0000 7f9137a78740 1 Processor -- start 2024-03-04T08:58:19.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.101+0000 7f9137a78740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e1827b0 0x559a7e18e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:19.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.101+0000 7f9137a78740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559a7e183980 0x559a7e18ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:19.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.101+0000 7f9137a78740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a7e184340 0x559a7e1939a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:19.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.101+0000 7f9137a78740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7e19a5d0 con 0x559a7e1827b0 2024-03-04T08:58:19.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.101+0000 7f9137a78740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7e199f70 con 0x559a7e184340 2024-03-04T08:58:19.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.101+0000 7f9137a78740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7e19ac30 con 0x559a7e183980 2024-03-04T08:58:19.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.101+0000 7f9137a78740 10 client.0 _ll_register_callbacks cb 0x7f9118011db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T08:58:19.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.101+0000 7f912e59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e1827b0 0x559a7e18e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:19.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.101+0000 7f912e59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e1827b0 0x559a7e18e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36202/0 (socket says 172.21.15.71:36202) 2024-03-04T08:58:19.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.101+0000 7f912e59c640 1 -- 172.21.15.71:0/3286730677 learned_addr learned my addr 172.21.15.71:0/3286730677 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:19.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.101+0000 7f912ed9d640 1 --2- 172.21.15.71:0/3286730677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a7e184340 0x559a7e1939a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:19.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.101+0000 7f912e59c640 1 -- 172.21.15.71:0/3286730677 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559a7e183980 msgr2=0x559a7e18ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:58:19.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.101+0000 7f912e59c640 1 --2- 172.21.15.71:0/3286730677 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559a7e183980 0x559a7e18ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.101+0000 7f912e59c640 1 -- 172.21.15.71:0/3286730677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a7e184340 msgr2=0x559a7e1939a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:58:19.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.101+0000 7f912e59c640 1 --2- 172.21.15.71:0/3286730677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a7e184340 0x559a7e1939a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.101+0000 7f912e59c640 1 -- 172.21.15.71:0/3286730677 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a7e193fe0 con 0x559a7e1827b0 2024-03-04T08:58:19.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.102+0000 7f912e59c640 1 --2- 172.21.15.71:0/3286730677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e1827b0 0x559a7e18e500 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f912002e010 tx=0x7f9120003a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a01573fd40a6bd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:19.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.102+0000 7f9115ffb640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-04T08:58:19.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.102+0000 7f9115ffb640 1 -- 172.21.15.71:0/3286730677 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912004b040 con 0x559a7e1827b0 2024-03-04T08:58:19.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.102+0000 7f9115ffb640 1 -- 172.21.15.71:0/3286730677 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f9120003680 con 0x559a7e1827b0 2024-03-04T08:58:19.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.102+0000 7f9137a78740 1 -- 172.21.15.71:0/3286730677 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91200c0840 con 0x559a7e1827b0 2024-03-04T08:58:19.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.102+0000 7f9115ffb640 1 -- 172.21.15.71:0/3286730677 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91200dae60 con 0x559a7e1827b0 2024-03-04T08:58:19.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.102+0000 7f9137a78740 10 client.15231 fetch_fsmap learned FSMap version 20 2024-03-04T08:58:19.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.102+0000 7f9137a78740 1 -- 172.21.15.71:0/3286730677 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x559a7e1f53a0 con 0x559a7e1827b0 2024-03-04T08:58:19.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.102+0000 7f9115ffb640 1 -- 172.21.15.71:0/3286730677 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91200c0840 con 0x559a7e1827b0 2024-03-04T08:58:19.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.102+0000 7f9115ffb640 1 -- 172.21.15.71:0/3286730677 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 20) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f91200c4a00 con 0x559a7e1827b0 2024-03-04T08:58:19.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.102+0000 7f9137a78740 10 client.15231 fetch_fsmap finished waiting for FSMap version 20 2024-03-04T08:58:19.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.102+0000 7f9137a78740 10 client.15231 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-04T08:58:19.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.102+0000 7f9137a78740 20 client.15231 populate_metadata read hostname 'smithi071' 2024-03-04T08:58:19.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.102+0000 7f9137a78740 1 --2- 172.21.15.71:0/3286730677 >> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] conn(0x559a7e1f9840 0x559a7e1f5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:19.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.102+0000 7f9137a78740 4 client.15231 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-04T08:58:19.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.102+0000 7f9137a78740 1 -- 172.21.15.71:0/3286730677 --> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559a7e0eef40 con 0x559a7e1f9840 2024-03-04T08:58:19.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.103+0000 7f9127fff640 1 --2- 172.21.15.71:0/3286730677 >> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] conn(0x559a7e1f9840 0x559a7e1f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:19.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.103+0000 7f9115ffb640 10 client.15231 ms_handle_connect on v2:172.21.15.150:6838/2639278083 2024-03-04T08:58:19.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.103+0000 7f9127fff640 1 --2- 172.21.15.71:0/3286730677 >> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] conn(0x559a7e1f9840 0x559a7e1f5570 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac240bffb491f41e server_cookie=a31df13e791d6c13 in_seq=0 out_seq=0 2024-03-04T08:58:19.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.103+0000 7f9115ffb640 1 -- 172.21.15.71:0/3286730677 <== mds.0 v2:172.21.15.150:6838/2639278083 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x559a7e0eef40 con 0x559a7e1f9840 2024-03-04T08:58:19.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.103+0000 7f9115ffb640 10 client.15231 handle_command_reply: tid=0 2024-03-04T08:58:19.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.103+0000 7f9137a78740 1 client.15231 shutdown 2024-03-04T08:58:19.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.103+0000 7f9137a78740 1 -- 172.21.15.71:0/3286730677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e1827b0 msgr2=0x559a7e18e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:19.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.103+0000 7f9137a78740 1 --2- 172.21.15.71:0/3286730677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e1827b0 0x559a7e18e500 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f912002e010 tx=0x7f9120003a20 comp rx=0 tx=0).stop 2024-03-04T08:58:19.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.103+0000 7f9137a78740 1 -- 172.21.15.71:0/3286730677 shutdown_connections 2024-03-04T08:58:19.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.103+0000 7f9137a78740 1 --2- 172.21.15.71:0/3286730677 >> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] conn(0x559a7e1f9840 0x559a7e1f5570 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.103+0000 7f9137a78740 1 --2- 172.21.15.71:0/3286730677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559a7e184340 0x559a7e1939a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.103+0000 7f9115ffb640 0 client.15231 ms_handle_reset on v2:172.21.15.150:6838/2639278083 2024-03-04T08:58:19.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.103+0000 7f9137a78740 1 --2- 172.21.15.71:0/3286730677 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x559a7e183980 0x559a7e18ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.103+0000 7f9137a78740 1 --2- 172.21.15.71:0/3286730677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559a7e1827b0 0x559a7e18e500 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.103+0000 7f9137a78740 1 -- 172.21.15.71:0/3286730677 >> 172.21.15.71:0/3286730677 conn(0x559a7e17dba0 msgr2=0x559a7e181690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:19.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.103+0000 7f9137a78740 1 -- 172.21.15.71:0/3286730677 shutdown_connections 2024-03-04T08:58:19.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.103+0000 7f9137a78740 1 -- 172.21.15.71:0/3286730677 wait complete. 2024-03-04T08:58:19.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.103+0000 7f9137a78740 20 client.15231 trim_cache size 0 max 16384 2024-03-04T08:58:19.128 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-03-04T08:58:19.128 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "b3c85c40-da03-11ee-95c6-87774f69a715", 2024-03-04T08:58:19.128 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2024-03-04T08:58:19.128 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 24491, 2024-03-04T08:58:19.128 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-03-04T08:58:19.128 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-03-04T08:58:19.128 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-03-04T08:58:19.128 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 108.111238938, 2024-03-04T08:58:19.128 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 20, 2024-03-04T08:58:19.128 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 93, 2024-03-04T08:58:19.128 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 91, 2024-03-04T08:58:19.129 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 113.257444384 2024-03-04T08:58:19.129 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-03-04T08:58:19.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.104+0000 7f9134e88640 1 -- 172.21.15.71:0/2410956097 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f911c0a1e10 msgr2=0x7f911c0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:19.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.104+0000 7f9134e88640 1 --2- 172.21.15.71:0/2410956097 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f911c0a1e10 0x7f911c0a42d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f91180095f0 tx=0x7f9118005f90 comp rx=0 tx=0).stop 2024-03-04T08:58:19.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.104+0000 7f9134e88640 1 -- 172.21.15.71:0/2410956097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91301004d0 msgr2=0x7f9130104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:19.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.104+0000 7f9134e88640 1 --2- 172.21.15.71:0/2410956097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91301004d0 0x7f9130104460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f912002c170 tx=0x7f912002c6c0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.104+0000 7f9134e88640 1 -- 172.21.15.71:0/2410956097 shutdown_connections 2024-03-04T08:58:19.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.104+0000 7f9134e88640 1 --2- 172.21.15.71:0/2410956097 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f911c0a1e10 0x7f911c0a42d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.104+0000 7f9134e88640 1 --2- 172.21.15.71:0/2410956097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9130102080 0x7f9130102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.104+0000 7f9134e88640 1 --2- 172.21.15.71:0/2410956097 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f91301016c0 0x7f9130102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.104+0000 7f9134e88640 1 --2- 172.21.15.71:0/2410956097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91301004d0 0x7f9130104460 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:19.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.104+0000 7f9134e88640 1 -- 172.21.15.71:0/2410956097 >> 172.21.15.71:0/2410956097 conn(0x7f91300fba60 msgr2=0x7f91300fc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:19.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.104+0000 7f9134e88640 1 -- 172.21.15.71:0/2410956097 shutdown_connections 2024-03-04T08:58:19.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:19.104+0000 7f9134e88640 1 -- 172.21.15.71:0/2410956097 wait complete. 2024-03-04T08:58:19.131 INFO:tasks.cephadm:Setup complete, yielding 2024-03-04T08:58:19.132 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-04T08:58:19.142 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi071.front.sepia.ceph.com 2024-03-04T08:58:19.143 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- bash -c 'ceph orch status' 2024-03-04T08:58:19.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:18 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1593945244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T08:58:19.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:18 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2944752809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:58:19.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:18 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2055911321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:58:19.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:18 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1593945244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T08:58:19.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:18 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2944752809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:58:19.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:18 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2055911321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:58:19.381 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:58:20.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:19 smithi071 ceph-mon[32789]: pgmap v381: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:20.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:19 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3218078197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:58:20.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:19 smithi150 ceph-mon[36807]: pgmap v381: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:20.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:19 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3218078197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:58:20.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:19 smithi158 ceph-mon[37241]: pgmap v381: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:20.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:19 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3218078197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:58:21.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.174+0000 7fdcd05e8700 1 -- 172.21.15.71:0/737749918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcc8107e60 msgr2=0x7fdcc810a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:21.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.174+0000 7fdcd05e8700 1 --2- 172.21.15.71:0/737749918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcc8107e60 0x7fdcc810a250 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000b0a0 tx=0x7fdcc000b3b0 comp rx=0 tx=0).stop 2024-03-04T08:58:21.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.174+0000 7fdcd05e8700 1 -- 172.21.15.71:0/737749918 shutdown_connections 2024-03-04T08:58:21.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.174+0000 7fdcd05e8700 1 --2- 172.21.15.71:0/737749918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcc8107e60 0x7fdcc810a250 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:21.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.174+0000 7fdcd05e8700 1 --2- 172.21.15.71:0/737749918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcc8102c00 0x7fdcc8104ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:21.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.174+0000 7fdcd05e8700 1 --2- 172.21.15.71:0/737749918 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdcc8105530 0x7fdcc8107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:21.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.174+0000 7fdcd05e8700 1 -- 172.21.15.71:0/737749918 >> 172.21.15.71:0/737749918 conn(0x7fdcc80fc590 msgr2=0x7fdcc80fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:21.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.174+0000 7fdcd05e8700 1 -- 172.21.15.71:0/737749918 shutdown_connections 2024-03-04T08:58:21.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.174+0000 7fdcd05e8700 1 -- 172.21.15.71:0/737749918 wait complete. 2024-03-04T08:58:21.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.175+0000 7fdcd05e8700 1 Processor -- start 2024-03-04T08:58:21.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.175+0000 7fdcd05e8700 1 -- start start 2024-03-04T08:58:21.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.175+0000 7fdcd05e8700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdcc8102c00 0x7fdcc8102560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:21.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.175+0000 7fdcd05e8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcc8105530 0x7fdcc8100bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:21.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.175+0000 7fdcd05e8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcc8107e60 0x7fdcc81010f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:21.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.175+0000 7fdcd05e8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc810dcc0 con 0x7fdcc8107e60 2024-03-04T08:58:21.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.175+0000 7fdcd05e8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc810db40 con 0x7fdcc8105530 2024-03-04T08:58:21.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.175+0000 7fdcd05e8700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc810de40 con 0x7fdcc8102c00 2024-03-04T08:58:21.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.175+0000 7fdcceb85700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcc8107e60 0x7fdcc81010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:21.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.175+0000 7fdcceb85700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcc8107e60 0x7fdcc81010f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36224/0 (socket says 172.21.15.71:36224) 2024-03-04T08:58:21.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.175+0000 7fdcceb85700 1 -- 172.21.15.71:0/4200730166 learned_addr learned my addr 172.21.15.71:0/4200730166 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:21.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.175+0000 7fdccdb83700 1 --2- 172.21.15.71:0/4200730166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcc8105530 0x7fdcc8100bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:21.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.175+0000 7fdcce384700 1 --2- 172.21.15.71:0/4200730166 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdcc8102c00 0x7fdcc8102560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:21.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.175+0000 7fdcceb85700 1 -- 172.21.15.71:0/4200730166 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdcc8102c00 msgr2=0x7fdcc8102560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:21.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.175+0000 7fdcceb85700 1 --2- 172.21.15.71:0/4200730166 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdcc8102c00 0x7fdcc8102560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:21.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.175+0000 7fdcceb85700 1 -- 172.21.15.71:0/4200730166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcc8105530 msgr2=0x7fdcc8100bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:21.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.175+0000 7fdcceb85700 1 --2- 172.21.15.71:0/4200730166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcc8105530 0x7fdcc8100bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:21.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.175+0000 7fdcceb85700 1 -- 172.21.15.71:0/4200730166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc81019a0 con 0x7fdcc8107e60 2024-03-04T08:58:21.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.176+0000 7fdcceb85700 1 --2- 172.21.15.71:0/4200730166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcc8107e60 0x7fdcc81010f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000b070 tx=0x7fdcc0004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:21.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.176+0000 7fdcbf7fe700 1 -- 172.21.15.71:0/4200730166 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0021070 con 0x7fdcc8107e60 2024-03-04T08:58:21.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.176+0000 7fdcbf7fe700 1 -- 172.21.15.71:0/4200730166 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdcc0007c00 con 0x7fdcc8107e60 2024-03-04T08:58:21.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.176+0000 7fdcd05e8700 1 -- 172.21.15.71:0/4200730166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc81a2400 con 0x7fdcc8107e60 2024-03-04T08:58:21.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.176+0000 7fdcbf7fe700 1 -- 172.21.15.71:0/4200730166 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0005350 con 0x7fdcc8107e60 2024-03-04T08:58:21.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.176+0000 7fdcd05e8700 1 -- 172.21.15.71:0/4200730166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc81a2860 con 0x7fdcc8107e60 2024-03-04T08:58:21.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.177+0000 7fdcd05e8700 1 -- 172.21.15.71:0/4200730166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc8068d80 con 0x7fdcc8107e60 2024-03-04T08:58:21.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.178+0000 7fdcbf7fe700 1 -- 172.21.15.71:0/4200730166 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fdcc0024070 con 0x7fdcc8107e60 2024-03-04T08:58:21.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.178+0000 7fdcbf7fe700 1 --2- 172.21.15.71:0/4200730166 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fdcb40a1ea0 0x7fdcb40a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:21.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.179+0000 7fdcce384700 1 --2- 172.21.15.71:0/4200730166 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fdcb40a1ea0 0x7fdcb40a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:21.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.179+0000 7fdcbf7fe700 1 -- 172.21.15.71:0/4200730166 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fdcc000f020 con 0x7fdcc8107e60 2024-03-04T08:58:21.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.179+0000 7fdcce384700 1 --2- 172.21.15.71:0/4200730166 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fdcb40a1ea0 0x7fdcb40a4360 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009bf0 tx=0x7fdcc4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:21.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.183+0000 7fdcbf7fe700 1 -- 172.21.15.71:0/4200730166 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdcc002b020 con 0x7fdcc8107e60 2024-03-04T08:58:21.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.394+0000 7fdcd05e8700 1 -- 172.21.15.71:0/4200730166 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdcc8062500 con 0x7fdcb40a1ea0 2024-03-04T08:58:21.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.396+0000 7fdcbf7fe700 1 -- 172.21.15.71:0/4200730166 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fdcc8062500 con 0x7fdcb40a1ea0 2024-03-04T08:58:21.391 INFO:teuthology.orchestra.run.smithi071.stdout:Backend: cephadm 2024-03-04T08:58:21.392 INFO:teuthology.orchestra.run.smithi071.stdout:Available: Yes 2024-03-04T08:58:21.392 INFO:teuthology.orchestra.run.smithi071.stdout:Paused: No 2024-03-04T08:58:21.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.397+0000 7fdcd05e8700 1 -- 172.21.15.71:0/4200730166 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fdcb40a1ea0 msgr2=0x7fdcb40a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:21.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.397+0000 7fdcd05e8700 1 --2- 172.21.15.71:0/4200730166 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fdcb40a1ea0 0x7fdcb40a4360 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009bf0 tx=0x7fdcc4009260 comp rx=0 tx=0).stop 2024-03-04T08:58:21.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.397+0000 7fdcd05e8700 1 -- 172.21.15.71:0/4200730166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcc8107e60 msgr2=0x7fdcc81010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:21.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.397+0000 7fdcd05e8700 1 --2- 172.21.15.71:0/4200730166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcc8107e60 0x7fdcc81010f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000b070 tx=0x7fdcc0004d00 comp rx=0 tx=0).stop 2024-03-04T08:58:21.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.398+0000 7fdcd05e8700 1 -- 172.21.15.71:0/4200730166 shutdown_connections 2024-03-04T08:58:21.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.398+0000 7fdcd05e8700 1 --2- 172.21.15.71:0/4200730166 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fdcb40a1ea0 0x7fdcb40a4360 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:21.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.398+0000 7fdcd05e8700 1 --2- 172.21.15.71:0/4200730166 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdcc8102c00 0x7fdcc8102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:21.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.398+0000 7fdcd05e8700 1 --2- 172.21.15.71:0/4200730166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcc8107e60 0x7fdcc81010f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:21.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.398+0000 7fdcd05e8700 1 --2- 172.21.15.71:0/4200730166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcc8105530 0x7fdcc8100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:21.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.398+0000 7fdcd05e8700 1 -- 172.21.15.71:0/4200730166 >> 172.21.15.71:0/4200730166 conn(0x7fdcc80fc590 msgr2=0x7fdcc8103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:21.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.398+0000 7fdcd05e8700 1 -- 172.21.15.71:0/4200730166 shutdown_connections 2024-03-04T08:58:21.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:21.398+0000 7fdcd05e8700 1 -- 172.21.15.71:0/4200730166 wait complete. 2024-03-04T08:58:22.056 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:21 smithi071 ceph-mon[32789]: pgmap v382: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:22.084 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-04T08:58:22.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:21 smithi150 ceph-mon[36807]: pgmap v382: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:22.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:21 smithi158 ceph-mon[37241]: pgmap v382: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:22.327 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:58:23.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:22 smithi071 ceph-mon[32789]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:23.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:22 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-04T08:58:23.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:22 smithi150 ceph-mon[36807]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:23.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:22 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-04T08:58:23.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:22 smithi158 ceph-mon[37241]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:23.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:22 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-04T08:58:24.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.042+0000 7f6ccceef700 1 -- 172.21.15.71:0/2074507406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cc8102b80 msgr2=0x7f6cc8109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:24.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.042+0000 7f6ccceef700 1 --2- 172.21.15.71:0/2074507406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cc8102b80 0x7f6cc8109430 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000b0d0 tx=0x7f6cc000b3e0 comp rx=0 tx=0).stop 2024-03-04T08:58:24.039 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:23 smithi071 ceph-mon[32789]: pgmap v383: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:58:24.039 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:23 smithi071 ceph-mon[32789]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-04T08:58:24.039 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:23 smithi071 ceph-mon[32789]: osdmap e94: 12 total, 12 up, 12 in 2024-03-04T08:58:24.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.042+0000 7f6ccceef700 1 -- 172.21.15.71:0/2074507406 shutdown_connections 2024-03-04T08:58:24.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.042+0000 7f6ccceef700 1 --2- 172.21.15.71:0/2074507406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cc8102b80 0x7f6cc8109430 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:24.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.042+0000 7f6ccceef700 1 --2- 172.21.15.71:0/2074507406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cc8100fe0 0x7f6cc8101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:24.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.042+0000 7f6ccceef700 1 --2- 172.21.15.71:0/2074507406 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6cc81021e0 0x7f6cc8102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:24.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.042+0000 7f6ccceef700 1 -- 172.21.15.71:0/2074507406 >> 172.21.15.71:0/2074507406 conn(0x7f6cc80fc580 msgr2=0x7f6cc80fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:24.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.043+0000 7f6ccceef700 1 -- 172.21.15.71:0/2074507406 shutdown_connections 2024-03-04T08:58:24.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.043+0000 7f6ccceef700 1 -- 172.21.15.71:0/2074507406 wait complete. 2024-03-04T08:58:24.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.043+0000 7f6ccceef700 1 Processor -- start 2024-03-04T08:58:24.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.044+0000 7f6ccceef700 1 -- start start 2024-03-04T08:58:24.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.044+0000 7f6ccceef700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cc8100fe0 0x7f6cc806b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:24.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.044+0000 7f6ccceef700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6cc81021e0 0x7f6cc819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:24.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.044+0000 7f6ccceef700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cc8102b80 0x7f6cc819f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:24.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.044+0000 7f6ccceef700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc810cba0 con 0x7f6cc8100fe0 2024-03-04T08:58:24.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.044+0000 7f6ccceef700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc810ca20 con 0x7f6cc8102b80 2024-03-04T08:58:24.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.044+0000 7f6ccceef700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc810cd20 con 0x7f6cc81021e0 2024-03-04T08:58:24.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.044+0000 7f6cc659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cc8100fe0 0x7f6cc806b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:24.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.044+0000 7f6cc659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cc8100fe0 0x7f6cc806b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36246/0 (socket says 172.21.15.71:36246) 2024-03-04T08:58:24.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.044+0000 7f6cc659c700 1 -- 172.21.15.71:0/704671204 learned_addr learned my addr 172.21.15.71:0/704671204 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:24.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.044+0000 7f6cc6d9d700 1 --2- 172.21.15.71:0/704671204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cc8102b80 0x7f6cc819f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:24.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.044+0000 7f6cc5d9b700 1 --2- 172.21.15.71:0/704671204 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6cc81021e0 0x7f6cc819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:24.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.044+0000 7f6cc659c700 1 -- 172.21.15.71:0/704671204 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6cc81021e0 msgr2=0x7f6cc819cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:24.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.044+0000 7f6cc659c700 1 --2- 172.21.15.71:0/704671204 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6cc81021e0 0x7f6cc819cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:24.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.044+0000 7f6cc659c700 1 -- 172.21.15.71:0/704671204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cc8102b80 msgr2=0x7f6cc819f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:24.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.044+0000 7f6cc659c700 1 --2- 172.21.15.71:0/704671204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cc8102b80 0x7f6cc819f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:24.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.044+0000 7f6cc659c700 1 -- 172.21.15.71:0/704671204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc8196cd0 con 0x7f6cc8100fe0 2024-03-04T08:58:24.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.044+0000 7f6cc5d9b700 1 --2- 172.21.15.71:0/704671204 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6cc81021e0 0x7f6cc819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:58:24.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.045+0000 7f6cc659c700 1 --2- 172.21.15.71:0/704671204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cc8100fe0 0x7f6cc806b180 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800e970 tx=0x7f6cb800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:24.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.045+0000 7f6cb77fe700 1 -- 172.21.15.71:0/704671204 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb800cbd0 con 0x7f6cc8100fe0 2024-03-04T08:58:24.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.045+0000 7f6cb77fe700 1 -- 172.21.15.71:0/704671204 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cb8004500 con 0x7f6cc8100fe0 2024-03-04T08:58:24.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.045+0000 7f6cb77fe700 1 -- 172.21.15.71:0/704671204 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8003e30 con 0x7f6cc8100fe0 2024-03-04T08:58:24.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.045+0000 7f6ccceef700 1 -- 172.21.15.71:0/704671204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc8196fc0 con 0x7f6cc8100fe0 2024-03-04T08:58:24.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.045+0000 7f6ccceef700 1 -- 172.21.15.71:0/704671204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc81974b0 con 0x7f6cc8100fe0 2024-03-04T08:58:24.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.046+0000 7f6ccceef700 1 -- 172.21.15.71:0/704671204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc8068d80 con 0x7f6cc8100fe0 2024-03-04T08:58:24.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.047+0000 7f6cb77fe700 1 -- 172.21.15.71:0/704671204 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f6cb8004000 con 0x7f6cc8100fe0 2024-03-04T08:58:24.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.047+0000 7f6cb77fe700 1 --2- 172.21.15.71:0/704671204 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6cb00a1ea0 0x7f6cb00a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:24.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.048+0000 7f6cc5d9b700 1 --2- 172.21.15.71:0/704671204 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6cb00a1ea0 0x7f6cb00a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:24.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.048+0000 7f6cb77fe700 1 -- 172.21.15.71:0/704671204 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f6cb80ce5d0 con 0x7f6cc8100fe0 2024-03-04T08:58:24.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.048+0000 7f6cc5d9b700 1 --2- 172.21.15.71:0/704671204 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6cb00a1ea0 0x7f6cb00a4360 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc005fd0 tx=0x7f6cbc005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:24.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.052+0000 7f6cb77fe700 1 -- 172.21.15.71:0/704671204 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6cb809b520 con 0x7f6cc8100fe0 2024-03-04T08:58:24.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.254+0000 7f6ccceef700 1 -- 172.21.15.71:0/704671204 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6cc8107ee0 con 0x7f6cb00a1ea0 2024-03-04T08:58:24.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.267+0000 7f6cb77fe700 1 -- 172.21.15.71:0/704671204 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f6cc8107ee0 con 0x7f6cb00a1ea0 2024-03-04T08:58:24.263 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T08:58:24.263 INFO:teuthology.orchestra.run.smithi071.stdout:mds.a smithi071 running (2m) 18s ago 2m 12.9M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a 086e350c76f9 2024-03-04T08:58:24.264 INFO:teuthology.orchestra.run.smithi071.stdout:mds.b smithi150 running (118s) 51s ago 118s 28.2M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a 20c8cbae0940 2024-03-04T08:58:24.264 INFO:teuthology.orchestra.run.smithi071.stdout:mds.c smithi158 running (2m) 51s ago 2m 13.5M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a 78725097cb84 2024-03-04T08:58:24.264 INFO:teuthology.orchestra.run.smithi071.stdout:mds.d smithi071 running (2m) 18s ago 2m 15.3M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a 254998c03e91 2024-03-04T08:58:24.264 INFO:teuthology.orchestra.run.smithi071.stdout:mds.e smithi150 running (2m) 51s ago 2m 14.0M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a 5826b805502d 2024-03-04T08:58:24.264 INFO:teuthology.orchestra.run.smithi071.stdout:mds.f smithi158 running (2m) 51s ago 2m 16.2M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a 7faadf000d22 2024-03-04T08:58:24.264 INFO:teuthology.orchestra.run.smithi071.stdout:mds.g smithi071 running (2m) 18s ago 2m 13.8M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a 0cac76a6e46a 2024-03-04T08:58:24.264 INFO:teuthology.orchestra.run.smithi071.stdout:mds.h smithi150 running (2m) 51s ago 2m 13.9M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a 284af29d0072 2024-03-04T08:58:24.264 INFO:teuthology.orchestra.run.smithi071.stdout:mds.i smithi158 running (2m) 51s ago 2m 16.2M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a c72f4745cb05 2024-03-04T08:58:24.264 INFO:teuthology.orchestra.run.smithi071.stdout:mds.j smithi071 running (2m) 18s ago 2m 16.4M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a 32208bdaccc6 2024-03-04T08:58:24.264 INFO:teuthology.orchestra.run.smithi071.stdout:mds.k smithi150 running (2m) 51s ago 2m 14.0M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a 12678faf940d 2024-03-04T08:58:24.264 INFO:teuthology.orchestra.run.smithi071.stdout:mds.l smithi158 running (2m) 51s ago 2m 13.1M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a d8193eec104a 2024-03-04T08:58:24.264 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.x smithi071 *:9283,8765 running (10m) 18s ago 10m 514M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a ba61387fd7a7 2024-03-04T08:58:24.265 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.y smithi150 *:8443,8765 running (8m) 51s ago 8m 452M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a 39baab92d66c 2024-03-04T08:58:24.265 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.z smithi158 *:8443,8765 running (8m) 51s ago 8m 454M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a 4f4b699bb612 2024-03-04T08:58:24.265 INFO:teuthology.orchestra.run.smithi071.stdout:mon.a smithi071 running (10m) 18s ago 10m 50.0M 2048M 19.0.0-1739-g1bc250c7 f88ffb32aa6a c1582a68dbda 2024-03-04T08:58:24.265 INFO:teuthology.orchestra.run.smithi071.stdout:mon.b smithi150 running (8m) 51s ago 8m 43.1M 2048M 19.0.0-1739-g1bc250c7 f88ffb32aa6a ad4f4fe1bfd0 2024-03-04T08:58:24.265 INFO:teuthology.orchestra.run.smithi071.stdout:mon.c smithi158 running (9m) 51s ago 9m 42.6M 2048M 19.0.0-1739-g1bc250c7 f88ffb32aa6a 78dd30fa6ca7 2024-03-04T08:58:24.265 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (8m) 18s ago 8m 45.5M 4272M 19.0.0-1739-g1bc250c7 f88ffb32aa6a c7c221fecf17 2024-03-04T08:58:24.265 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi150 running (8m) 51s ago 7m 54.2M 4270M 19.0.0-1739-g1bc250c7 f88ffb32aa6a 1b3cc123c3f0 2024-03-04T08:58:24.265 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi158 running (7m) 51s ago 7m 47.2M 4270M 19.0.0-1739-g1bc250c7 f88ffb32aa6a 3f5227346c44 2024-03-04T08:58:24.265 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi071 running (6m) 18s ago 6m 52.9M 4272M 19.0.0-1739-g1bc250c7 f88ffb32aa6a 8b0a4ae2c24f 2024-03-04T08:58:24.265 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi150 running (6m) 51s ago 6m 42.5M 4270M 19.0.0-1739-g1bc250c7 f88ffb32aa6a a0aeed32e054 2024-03-04T08:58:24.265 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi158 running (5m) 51s ago 5m 45.5M 4270M 19.0.0-1739-g1bc250c7 f88ffb32aa6a 85806226e24d 2024-03-04T08:58:24.265 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6 smithi071 running (5m) 18s ago 5m 41.4M 4272M 19.0.0-1739-g1bc250c7 f88ffb32aa6a 2870692d0fc2 2024-03-04T08:58:24.265 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7 smithi150 running (4m) 51s ago 4m 44.7M 4270M 19.0.0-1739-g1bc250c7 f88ffb32aa6a f13f8b97a6c7 2024-03-04T08:58:24.265 INFO:teuthology.orchestra.run.smithi071.stdout:osd.8 smithi158 running (4m) 51s ago 4m 43.0M 4270M 19.0.0-1739-g1bc250c7 f88ffb32aa6a 13828b58057f 2024-03-04T08:58:24.266 INFO:teuthology.orchestra.run.smithi071.stdout:osd.9 smithi071 running (3m) 18s ago 3m 50.0M 4272M 19.0.0-1739-g1bc250c7 f88ffb32aa6a c94f5b300a1c 2024-03-04T08:58:24.266 INFO:teuthology.orchestra.run.smithi071.stdout:osd.10 smithi150 running (3m) 51s ago 3m 46.1M 4270M 19.0.0-1739-g1bc250c7 f88ffb32aa6a 29ea42bb55f3 2024-03-04T08:58:24.266 INFO:teuthology.orchestra.run.smithi071.stdout:osd.11 smithi158 running (3m) 51s ago 2m 42.4M 4270M 19.0.0-1739-g1bc250c7 f88ffb32aa6a 7d43fbedd99c 2024-03-04T08:58:24.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.269+0000 7f6ccceef700 1 -- 172.21.15.71:0/704671204 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6cb00a1ea0 msgr2=0x7f6cb00a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:24.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.269+0000 7f6ccceef700 1 --2- 172.21.15.71:0/704671204 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6cb00a1ea0 0x7f6cb00a4360 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc005fd0 tx=0x7f6cbc005e30 comp rx=0 tx=0).stop 2024-03-04T08:58:24.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.269+0000 7f6ccceef700 1 -- 172.21.15.71:0/704671204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cc8100fe0 msgr2=0x7f6cc806b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:24.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.269+0000 7f6ccceef700 1 --2- 172.21.15.71:0/704671204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cc8100fe0 0x7f6cc806b180 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800e970 tx=0x7f6cb800ec80 comp rx=0 tx=0).stop 2024-03-04T08:58:24.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.269+0000 7f6ccceef700 1 -- 172.21.15.71:0/704671204 shutdown_connections 2024-03-04T08:58:24.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.269+0000 7f6ccceef700 1 --2- 172.21.15.71:0/704671204 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6cb00a1ea0 0x7f6cb00a4360 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:24.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.269+0000 7f6ccceef700 1 --2- 172.21.15.71:0/704671204 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cc8102b80 0x7f6cc819f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:24.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.269+0000 7f6ccceef700 1 --2- 172.21.15.71:0/704671204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cc8100fe0 0x7f6cc806b180 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:24.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.269+0000 7f6ccceef700 1 --2- 172.21.15.71:0/704671204 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6cc81021e0 0x7f6cc819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:24.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.269+0000 7f6ccceef700 1 -- 172.21.15.71:0/704671204 >> 172.21.15.71:0/704671204 conn(0x7f6cc80fc580 msgr2=0x7f6cc8105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:24.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.270+0000 7f6ccceef700 1 -- 172.21.15.71:0/704671204 shutdown_connections 2024-03-04T08:58:24.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:24.270+0000 7f6ccceef700 1 -- 172.21.15.71:0/704671204 wait complete. 2024-03-04T08:58:24.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:23 smithi150 ceph-mon[36807]: pgmap v383: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:58:24.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:23 smithi150 ceph-mon[36807]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-04T08:58:24.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:23 smithi150 ceph-mon[36807]: osdmap e94: 12 total, 12 up, 12 in 2024-03-04T08:58:24.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:23 smithi158 ceph-mon[37241]: pgmap v383: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:58:24.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:23 smithi158 ceph-mon[37241]: from='mgr.14152 172.21.15.71:0/3993077082' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-04T08:58:24.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:23 smithi158 ceph-mon[37241]: osdmap e94: 12 total, 12 up, 12 in 2024-03-04T08:58:24.899 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:24 smithi071 ceph-mon[32789]: osdmap e95: 12 total, 12 up, 12 in 2024-03-04T08:58:24.926 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-04T08:58:25.166 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:58:25.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:24 smithi150 ceph-mon[36807]: osdmap e95: 12 total, 12 up, 12 in 2024-03-04T08:58:25.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:24 smithi158 ceph-mon[37241]: osdmap e95: 12 total, 12 up, 12 in 2024-03-04T08:58:26.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:25 smithi071 ceph-mon[32789]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:26.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:25 smithi071 ceph-mon[32789]: pgmap v386: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-03-04T08:58:26.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:25 smithi071 ceph-mon[32789]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-03-04T08:58:26.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:25 smithi150 ceph-mon[36807]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:26.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:25 smithi150 ceph-mon[36807]: pgmap v386: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-03-04T08:58:26.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:25 smithi150 ceph-mon[36807]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-03-04T08:58:26.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:25 smithi158 ceph-mon[37241]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:26.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:25 smithi158 ceph-mon[37241]: pgmap v386: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-03-04T08:58:26.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:25 smithi158 ceph-mon[37241]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-03-04T08:58:26.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.808+0000 7f35be42e700 1 -- 172.21.15.71:0/2440708206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35b8101260 msgr2=0x7f35b81016e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:26.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.808+0000 7f35be42e700 1 --2- 172.21.15.71:0/2440708206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35b8101260 0x7f35b81016e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009990 tx=0x7f35ac009ca0 comp rx=0 tx=0).stop 2024-03-04T08:58:26.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.808+0000 7f35be42e700 1 -- 172.21.15.71:0/2440708206 shutdown_connections 2024-03-04T08:58:26.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.808+0000 7f35be42e700 1 --2- 172.21.15.71:0/2440708206 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f35b8102690 0x7f35b8102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:26.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.808+0000 7f35be42e700 1 --2- 172.21.15.71:0/2440708206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35b8101c20 0x7f35b81094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:26.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.808+0000 7f35be42e700 1 --2- 172.21.15.71:0/2440708206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35b8101260 0x7f35b81016e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:26.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.808+0000 7f35be42e700 1 -- 172.21.15.71:0/2440708206 >> 172.21.15.71:0/2440708206 conn(0x7f35b80fbd10 msgr2=0x7f35b80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:26.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.808+0000 7f35be42e700 1 -- 172.21.15.71:0/2440708206 shutdown_connections 2024-03-04T08:58:26.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.808+0000 7f35be42e700 1 -- 172.21.15.71:0/2440708206 wait complete. 2024-03-04T08:58:26.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.808+0000 7f35be42e700 1 Processor -- start 2024-03-04T08:58:26.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.808+0000 7f35be42e700 1 -- start start 2024-03-04T08:58:26.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.809+0000 7f35be42e700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f35b8101260 0x7f35b8100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:26.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.809+0000 7f35be42e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35b8101c20 0x7f35b819f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:26.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.809+0000 7f35be42e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35b8102690 0x7f35b81a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:26.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.809+0000 7f35be42e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b810cc30 con 0x7f35b8101c20 2024-03-04T08:58:26.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.809+0000 7f35be42e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b810cab0 con 0x7f35b8102690 2024-03-04T08:58:26.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.809+0000 7f35be42e700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b810cdb0 con 0x7f35b8101260 2024-03-04T08:58:26.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.809+0000 7f35b77fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35b8101c20 0x7f35b819f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:26.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.809+0000 7f35b77fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35b8101c20 0x7f35b819f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43966/0 (socket says 172.21.15.71:43966) 2024-03-04T08:58:26.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.809+0000 7f35b77fe700 1 -- 172.21.15.71:0/2387734593 learned_addr learned my addr 172.21.15.71:0/2387734593 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:26.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.809+0000 7f35bc9cb700 1 --2- 172.21.15.71:0/2387734593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35b8102690 0x7f35b81a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:26.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.809+0000 7f35b7fff700 1 --2- 172.21.15.71:0/2387734593 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f35b8101260 0x7f35b8100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:26.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.809+0000 7f35b77fe700 1 -- 172.21.15.71:0/2387734593 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f35b8101260 msgr2=0x7f35b8100cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:26.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.809+0000 7f35b77fe700 1 --2- 172.21.15.71:0/2387734593 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f35b8101260 0x7f35b8100cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:26.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.809+0000 7f35b77fe700 1 -- 172.21.15.71:0/2387734593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35b8102690 msgr2=0x7f35b81a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:26.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.809+0000 7f35b77fe700 1 --2- 172.21.15.71:0/2387734593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35b8102690 0x7f35b81a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:26.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.809+0000 7f35b77fe700 1 -- 172.21.15.71:0/2387734593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b81a2190 con 0x7f35b8101c20 2024-03-04T08:58:26.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.809+0000 7f35b7fff700 1 --2- 172.21.15.71:0/2387734593 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f35b8101260 0x7f35b8100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:58:26.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.809+0000 7f35b77fe700 1 --2- 172.21.15.71:0/2387734593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35b8101c20 0x7f35b819f460 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f35ac005910 tx=0x7f35ac00bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:26.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.809+0000 7f35b57fa700 1 -- 172.21.15.71:0/2387734593 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac026070 con 0x7f35b8101c20 2024-03-04T08:58:26.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.809+0000 7f35b57fa700 1 -- 172.21.15.71:0/2387734593 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35ac00f460 con 0x7f35b8101c20 2024-03-04T08:58:26.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.809+0000 7f35be42e700 1 -- 172.21.15.71:0/2387734593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b81a2420 con 0x7f35b8101c20 2024-03-04T08:58:26.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.809+0000 7f35b57fa700 1 -- 172.21.15.71:0/2387734593 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac021670 con 0x7f35b8101c20 2024-03-04T08:58:26.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.809+0000 7f35be42e700 1 -- 172.21.15.71:0/2387734593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b81a2870 con 0x7f35b8101c20 2024-03-04T08:58:26.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.810+0000 7f35be42e700 1 -- 172.21.15.71:0/2387734593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b8068d80 con 0x7f35b8101c20 2024-03-04T08:58:26.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.811+0000 7f35b57fa700 1 -- 172.21.15.71:0/2387734593 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f35ac029070 con 0x7f35b8101c20 2024-03-04T08:58:26.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.812+0000 7f35b57fa700 1 --2- 172.21.15.71:0/2387734593 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f35a00a1e10 0x7f35a00a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:26.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.812+0000 7f35b7fff700 1 --2- 172.21.15.71:0/2387734593 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f35a00a1e10 0x7f35a00a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:26.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.812+0000 7f35b57fa700 1 -- 172.21.15.71:0/2387734593 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f35ac0cf060 con 0x7f35b8101c20 2024-03-04T08:58:26.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.812+0000 7f35b7fff700 1 --2- 172.21.15.71:0/2387734593 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f35a00a1e10 0x7f35a00a42d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f35a800bcf0 tx=0x7f35a800b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:26.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:26.816+0000 7f35b57fa700 1 -- 172.21.15.71:0/2387734593 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f35ac05d280 con 0x7f35b8101c20 2024-03-04T08:58:27.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:27.020+0000 7f35be42e700 1 -- 172.21.15.71:0/2387734593 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f35b8062500 con 0x7f35a00a1e10 2024-03-04T08:58:27.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:27.026+0000 7f35b57fa700 1 -- 172.21.15.71:0/2387734593 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f35b8062500 con 0x7f35a00a1e10 2024-03-04T08:58:27.022 INFO:teuthology.orchestra.run.smithi071.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-04T08:58:27.022 INFO:teuthology.orchestra.run.smithi071.stdout:mds.all 12/12 54s ago 2m smithi071=a;smithi071=d;smithi071=g;smithi071=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi158=c;smithi158=f;smithi158=i;smithi158=l;count:12 2024-03-04T08:58:27.023 INFO:teuthology.orchestra.run.smithi071.stdout:mgr 3/3 54s ago 8m smithi071=x;smithi150=y;smithi158=z;count:3 2024-03-04T08:58:27.023 INFO:teuthology.orchestra.run.smithi071.stdout:mon 3/3 54s ago 9m smithi071:172.21.15.71=a;smithi150:172.21.15.150=b;smithi158:172.21.15.158=c;count:3 2024-03-04T08:58:27.023 INFO:teuthology.orchestra.run.smithi071.stdout:osd 12 54s ago - 2024-03-04T08:58:27.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:27.028+0000 7f35be42e700 1 -- 172.21.15.71:0/2387734593 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f35a00a1e10 msgr2=0x7f35a00a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:27.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:27.028+0000 7f35be42e700 1 --2- 172.21.15.71:0/2387734593 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f35a00a1e10 0x7f35a00a42d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f35a800bcf0 tx=0x7f35a800b2c0 comp rx=0 tx=0).stop 2024-03-04T08:58:27.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:27.028+0000 7f35be42e700 1 -- 172.21.15.71:0/2387734593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35b8101c20 msgr2=0x7f35b819f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:27.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:27.028+0000 7f35be42e700 1 --2- 172.21.15.71:0/2387734593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35b8101c20 0x7f35b819f460 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f35ac005910 tx=0x7f35ac00bbb0 comp rx=0 tx=0).stop 2024-03-04T08:58:27.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:27.029+0000 7f35be42e700 1 -- 172.21.15.71:0/2387734593 shutdown_connections 2024-03-04T08:58:27.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:27.029+0000 7f35be42e700 1 --2- 172.21.15.71:0/2387734593 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f35a00a1e10 0x7f35a00a42d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:27.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:27.029+0000 7f35be42e700 1 --2- 172.21.15.71:0/2387734593 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f35b8101260 0x7f35b8100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:27.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:27.029+0000 7f35be42e700 1 --2- 172.21.15.71:0/2387734593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35b8102690 0x7f35b81a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:27.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:27.029+0000 7f35be42e700 1 --2- 172.21.15.71:0/2387734593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35b8101c20 0x7f35b819f460 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:27.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:27.029+0000 7f35be42e700 1 -- 172.21.15.71:0/2387734593 >> 172.21.15.71:0/2387734593 conn(0x7f35b80fbd10 msgr2=0x7f35b8105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:27.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:27.029+0000 7f35be42e700 1 -- 172.21.15.71:0/2387734593 shutdown_connections 2024-03-04T08:58:27.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:27.029+0000 7f35be42e700 1 -- 172.21.15.71:0/2387734593 wait complete. 2024-03-04T08:58:27.679 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-04T08:58:27.920 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:58:27.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:27 smithi071 ceph-mon[32789]: pgmap v387: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 383 B/s wr, 3 op/s 2024-03-04T08:58:28.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:27 smithi150 ceph-mon[36807]: pgmap v387: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 383 B/s wr, 3 op/s 2024-03-04T08:58:28.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:27 smithi158 ceph-mon[37241]: pgmap v387: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 383 B/s wr, 3 op/s 2024-03-04T08:58:29.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:28 smithi071 ceph-mon[32789]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:29.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:28 smithi150 ceph-mon[36807]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:29.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:28 smithi158 ceph-mon[37241]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:29.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.567+0000 7f6da001d700 1 -- 172.21.15.71:0/2172193885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d98103320 msgr2=0x7f6d98103780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:29.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.567+0000 7f6da001d700 1 --2- 172.21.15.71:0/2172193885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d98103320 0x7f6d98103780 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0099c0 tx=0x7f6d8c009cd0 comp rx=0 tx=0).stop 2024-03-04T08:58:29.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.567+0000 7f6da001d700 1 -- 172.21.15.71:0/2172193885 shutdown_connections 2024-03-04T08:58:29.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.567+0000 7f6da001d700 1 --2- 172.21.15.71:0/2172193885 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6d98103cc0 0x7f6d9810a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:29.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.567+0000 7f6da001d700 1 --2- 172.21.15.71:0/2172193885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d98102120 0x7f6d98102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:29.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.567+0000 7f6da001d700 1 --2- 172.21.15.71:0/2172193885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d98103320 0x7f6d98103780 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:29.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.567+0000 7f6da001d700 1 -- 172.21.15.71:0/2172193885 >> 172.21.15.71:0/2172193885 conn(0x7f6d980fd6c0 msgr2=0x7f6d980ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:29.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.567+0000 7f6da001d700 1 -- 172.21.15.71:0/2172193885 shutdown_connections 2024-03-04T08:58:29.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.567+0000 7f6da001d700 1 -- 172.21.15.71:0/2172193885 wait complete. 2024-03-04T08:58:29.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.568+0000 7f6da001d700 1 Processor -- start 2024-03-04T08:58:29.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.568+0000 7f6da001d700 1 -- start start 2024-03-04T08:58:29.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.568+0000 7f6da001d700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6d98102120 0x7f6d98198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:29.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.568+0000 7f6da001d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d98103320 0x7f6d98198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:29.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.568+0000 7f6da001d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d98103cc0 0x7f6d9819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:29.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.568+0000 7f6da001d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9810dce0 con 0x7f6d98103320 2024-03-04T08:58:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.568+0000 7f6da001d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9810db60 con 0x7f6d98103cc0 2024-03-04T08:58:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.568+0000 7f6da001d700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9810de60 con 0x7f6d98102120 2024-03-04T08:58:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.568+0000 7f6d9d5b8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d98103320 0x7f6d98198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.568+0000 7f6d9d5b8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d98103320 0x7f6d98198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43974/0 (socket says 172.21.15.71:43974) 2024-03-04T08:58:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.568+0000 7f6d9d5b8700 1 -- 172.21.15.71:0/2220170415 learned_addr learned my addr 172.21.15.71:0/2220170415 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.568+0000 7f6d9e5ba700 1 --2- 172.21.15.71:0/2220170415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d98103cc0 0x7f6d9819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.568+0000 7f6d9ddb9700 1 --2- 172.21.15.71:0/2220170415 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6d98102120 0x7f6d98198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.568+0000 7f6d9d5b8700 1 -- 172.21.15.71:0/2220170415 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6d98102120 msgr2=0x7f6d98198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.568+0000 7f6d9d5b8700 1 --2- 172.21.15.71:0/2220170415 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6d98102120 0x7f6d98198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.568+0000 7f6d9d5b8700 1 -- 172.21.15.71:0/2220170415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d98103cc0 msgr2=0x7f6d9819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.568+0000 7f6d9d5b8700 1 --2- 172.21.15.71:0/2220170415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d98103cc0 0x7f6d9819f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.568+0000 7f6d9d5b8700 1 -- 172.21.15.71:0/2220170415 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9819fea0 con 0x7f6d98103320 2024-03-04T08:58:29.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.568+0000 7f6d9e5ba700 1 --2- 172.21.15.71:0/2220170415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d98103cc0 0x7f6d9819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:58:29.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.568+0000 7f6d9ddb9700 1 --2- 172.21.15.71:0/2220170415 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6d98102120 0x7f6d98198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:58:29.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.569+0000 7f6d9d5b8700 1 --2- 172.21.15.71:0/2220170415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d98103320 0x7f6d98198780 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c000c00 tx=0x7f6d8c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:29.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.569+0000 7f6d8affd700 1 -- 172.21.15.71:0/2220170415 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c009e60 con 0x7f6d98103320 2024-03-04T08:58:29.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.569+0000 7f6d8affd700 1 -- 172.21.15.71:0/2220170415 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d8c00f460 con 0x7f6d98103320 2024-03-04T08:58:29.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.569+0000 7f6da001d700 1 -- 172.21.15.71:0/2220170415 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d981a0130 con 0x7f6d98103320 2024-03-04T08:58:29.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.569+0000 7f6d8affd700 1 -- 172.21.15.71:0/2220170415 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c017650 con 0x7f6d98103320 2024-03-04T08:58:29.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.569+0000 7f6da001d700 1 -- 172.21.15.71:0/2220170415 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d981a05e0 con 0x7f6d98103320 2024-03-04T08:58:29.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.570+0000 7f6da001d700 1 -- 172.21.15.71:0/2220170415 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d98068d80 con 0x7f6d98103320 2024-03-04T08:58:29.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.571+0000 7f6d8affd700 1 -- 172.21.15.71:0/2220170415 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f6d8c00f9c0 con 0x7f6d98103320 2024-03-04T08:58:29.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.571+0000 7f6d8affd700 1 --2- 172.21.15.71:0/2220170415 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6d840a1e90 0x7f6d840a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:29.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.572+0000 7f6d9ddb9700 1 --2- 172.21.15.71:0/2220170415 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6d840a1e90 0x7f6d840a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:29.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.572+0000 7f6d8affd700 1 -- 172.21.15.71:0/2220170415 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f6d8c0cf110 con 0x7f6d98103320 2024-03-04T08:58:29.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.572+0000 7f6d9ddb9700 1 --2- 172.21.15.71:0/2220170415 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6d840a1e90 0x7f6d840a4350 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6d98103180 tx=0x7f6d94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:29.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.576+0000 7f6d8affd700 1 -- 172.21.15.71:0/2220170415 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6d8c013070 con 0x7f6d98103320 2024-03-04T08:58:29.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.775+0000 7f6da001d700 1 -- 172.21.15.71:0/2220170415 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d98062500 con 0x7f6d840a1e90 2024-03-04T08:58:29.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.777+0000 7f6d8affd700 1 -- 172.21.15.71:0/2220170415 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f6d98062500 con 0x7f6d840a1e90 2024-03-04T08:58:29.774 INFO:teuthology.orchestra.run.smithi071.stdout:HOST ADDR LABELS STATUS 2024-03-04T08:58:29.774 INFO:teuthology.orchestra.run.smithi071.stdout:smithi071 172.21.15.71 2024-03-04T08:58:29.774 INFO:teuthology.orchestra.run.smithi071.stdout:smithi150 172.21.15.150 2024-03-04T08:58:29.774 INFO:teuthology.orchestra.run.smithi071.stdout:smithi158 172.21.15.158 2024-03-04T08:58:29.774 INFO:teuthology.orchestra.run.smithi071.stdout:3 hosts in cluster 2024-03-04T08:58:29.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.779+0000 7f6da001d700 1 -- 172.21.15.71:0/2220170415 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6d840a1e90 msgr2=0x7f6d840a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:29.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.779+0000 7f6da001d700 1 --2- 172.21.15.71:0/2220170415 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6d840a1e90 0x7f6d840a4350 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6d98103180 tx=0x7f6d94009260 comp rx=0 tx=0).stop 2024-03-04T08:58:29.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.779+0000 7f6da001d700 1 -- 172.21.15.71:0/2220170415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d98103320 msgr2=0x7f6d98198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:29.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.779+0000 7f6da001d700 1 --2- 172.21.15.71:0/2220170415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d98103320 0x7f6d98198780 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c000c00 tx=0x7f6d8c00bf90 comp rx=0 tx=0).stop 2024-03-04T08:58:29.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.779+0000 7f6da001d700 1 -- 172.21.15.71:0/2220170415 shutdown_connections 2024-03-04T08:58:29.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.779+0000 7f6da001d700 1 --2- 172.21.15.71:0/2220170415 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f6d840a1e90 0x7f6d840a4350 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:29.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.779+0000 7f6da001d700 1 --2- 172.21.15.71:0/2220170415 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6d98102120 0x7f6d98198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:29.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.779+0000 7f6da001d700 1 --2- 172.21.15.71:0/2220170415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d98103cc0 0x7f6d9819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:29.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.779+0000 7f6da001d700 1 --2- 172.21.15.71:0/2220170415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d98103320 0x7f6d98198780 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:29.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.779+0000 7f6da001d700 1 -- 172.21.15.71:0/2220170415 >> 172.21.15.71:0/2220170415 conn(0x7f6d980fd6c0 msgr2=0x7f6d98106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:29.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.779+0000 7f6da001d700 1 -- 172.21.15.71:0/2220170415 shutdown_connections 2024-03-04T08:58:29.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:29.779+0000 7f6da001d700 1 -- 172.21.15.71:0/2220170415 wait complete. 2024-03-04T08:58:30.083 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:29 smithi071 ceph-mon[32789]: pgmap v388: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 383 B/s wr, 3 op/s 2024-03-04T08:58:30.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:29 smithi150 ceph-mon[36807]: pgmap v388: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 383 B/s wr, 3 op/s 2024-03-04T08:58:30.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:29 smithi158 ceph-mon[37241]: pgmap v388: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 383 B/s wr, 3 op/s 2024-03-04T08:58:30.409 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-04T08:58:30.650 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:58:31.039 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:30 smithi071 ceph-mon[32789]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:31.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:30 smithi150 ceph-mon[36807]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:31.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:30 smithi158 ceph-mon[37241]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:32.116 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:31 smithi071 ceph-mon[32789]: pgmap v389: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 383 B/s wr, 1 op/s 2024-03-04T08:58:32.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:31 smithi150 ceph-mon[36807]: pgmap v389: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 383 B/s wr, 1 op/s 2024-03-04T08:58:32.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:31 smithi158 ceph-mon[37241]: pgmap v389: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 383 B/s wr, 1 op/s 2024-03-04T08:58:32.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.345+0000 7ffbb9366700 1 -- 172.21.15.71:0/1466956021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbb4101c80 msgr2=0x7ffbb4109540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:32.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.345+0000 7ffbb9366700 1 --2- 172.21.15.71:0/1466956021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbb4101c80 0x7ffbb4109540 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00b0d0 tx=0x7ffbac00b3e0 comp rx=0 tx=0).stop 2024-03-04T08:58:32.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.345+0000 7ffbb9366700 1 -- 172.21.15.71:0/1466956021 shutdown_connections 2024-03-04T08:58:32.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.345+0000 7ffbb9366700 1 --2- 172.21.15.71:0/1466956021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbb4101c80 0x7ffbb4109540 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:32.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.345+0000 7ffbb9366700 1 --2- 172.21.15.71:0/1466956021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbb41025a0 0x7ffbb41029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:32.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.345+0000 7ffbb9366700 1 --2- 172.21.15.71:0/1466956021 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffbb41012e0 0x7ffbb4101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:32.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.345+0000 7ffbb9366700 1 -- 172.21.15.71:0/1466956021 >> 172.21.15.71:0/1466956021 conn(0x7ffbb40fbd10 msgr2=0x7ffbb40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:32.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.345+0000 7ffbb9366700 1 -- 172.21.15.71:0/1466956021 shutdown_connections 2024-03-04T08:58:32.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.345+0000 7ffbb9366700 1 -- 172.21.15.71:0/1466956021 wait complete. 2024-03-04T08:58:32.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.346+0000 7ffbb9366700 1 Processor -- start 2024-03-04T08:58:32.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.346+0000 7ffbb9366700 1 -- start start 2024-03-04T08:58:32.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.346+0000 7ffbb9366700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbb41012e0 0x7ffbb4198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:32.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.346+0000 7ffbb9366700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbb4101c80 0x7ffbb41987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:32.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.346+0000 7ffbb9366700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffbb41025a0 0x7ffbb419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:32.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.346+0000 7ffbb9366700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb410cbe0 con 0x7ffbb41012e0 2024-03-04T08:58:32.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.346+0000 7ffbb9366700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb410ca60 con 0x7ffbb4101c80 2024-03-04T08:58:32.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.346+0000 7ffbb9366700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb410cd60 con 0x7ffbb41025a0 2024-03-04T08:58:32.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.346+0000 7ffbb2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbb41012e0 0x7ffbb4198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:32.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.346+0000 7ffbb2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbb41012e0 0x7ffbb4198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43996/0 (socket says 172.21.15.71:43996) 2024-03-04T08:58:32.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.346+0000 7ffbb2ffd700 1 -- 172.21.15.71:0/296446397 learned_addr learned my addr 172.21.15.71:0/296446397 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:32.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.346+0000 7ffbabfff700 1 --2- 172.21.15.71:0/296446397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbb4101c80 0x7ffbb41987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:32.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.346+0000 7ffbb37fe700 1 --2- 172.21.15.71:0/296446397 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffbb41025a0 0x7ffbb419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:32.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.346+0000 7ffbb2ffd700 1 -- 172.21.15.71:0/296446397 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffbb41025a0 msgr2=0x7ffbb419f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:32.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.346+0000 7ffbb2ffd700 1 --2- 172.21.15.71:0/296446397 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffbb41025a0 0x7ffbb419f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:32.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.346+0000 7ffbb2ffd700 1 -- 172.21.15.71:0/296446397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbb4101c80 msgr2=0x7ffbb41987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:32.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.346+0000 7ffbb2ffd700 1 --2- 172.21.15.71:0/296446397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbb4101c80 0x7ffbb41987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:32.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.346+0000 7ffbb2ffd700 1 -- 172.21.15.71:0/296446397 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb419fee0 con 0x7ffbb41012e0 2024-03-04T08:58:32.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.346+0000 7ffbb37fe700 1 --2- 172.21.15.71:0/296446397 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffbb41025a0 0x7ffbb419f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:58:32.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.346+0000 7ffbabfff700 1 --2- 172.21.15.71:0/296446397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbb4101c80 0x7ffbb41987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:58:32.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.347+0000 7ffbb2ffd700 1 --2- 172.21.15.71:0/296446397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbb41012e0 0x7ffbb4198280 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ffba400e9a0 tx=0x7ffba400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:32.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.347+0000 7ffbb0ff9700 1 -- 172.21.15.71:0/296446397 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba400cbd0 con 0x7ffbb41012e0 2024-03-04T08:58:32.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.347+0000 7ffbb9366700 1 -- 172.21.15.71:0/296446397 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb41a01d0 con 0x7ffbb41012e0 2024-03-04T08:58:32.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.347+0000 7ffbb0ff9700 1 -- 172.21.15.71:0/296446397 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffba4004500 con 0x7ffbb41012e0 2024-03-04T08:58:32.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.347+0000 7ffbb0ff9700 1 -- 172.21.15.71:0/296446397 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba4003e00 con 0x7ffbb41012e0 2024-03-04T08:58:32.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.347+0000 7ffbb9366700 1 -- 172.21.15.71:0/296446397 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb41a0760 con 0x7ffbb41012e0 2024-03-04T08:58:32.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.348+0000 7ffbb9366700 1 -- 172.21.15.71:0/296446397 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb4068d80 con 0x7ffbb41012e0 2024-03-04T08:58:32.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.349+0000 7ffbb0ff9700 1 -- 172.21.15.71:0/296446397 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7ffba4003fa0 con 0x7ffbb41012e0 2024-03-04T08:58:32.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.349+0000 7ffbb0ff9700 1 --2- 172.21.15.71:0/296446397 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ffba00a1e10 0x7ffba00a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:32.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.349+0000 7ffbabfff700 1 --2- 172.21.15.71:0/296446397 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ffba00a1e10 0x7ffba00a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:32.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.349+0000 7ffbb0ff9700 1 -- 172.21.15.71:0/296446397 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7ffba40ce500 con 0x7ffbb41012e0 2024-03-04T08:58:32.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.350+0000 7ffbabfff700 1 --2- 172.21.15.71:0/296446397 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ffba00a1e10 0x7ffba00a42d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c005950 tx=0x7ffb9c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:32.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.353+0000 7ffbb0ff9700 1 -- 172.21.15.71:0/296446397 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ffba401f340 con 0x7ffbb41012e0 2024-03-04T08:58:32.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.553+0000 7ffbb9366700 1 -- 172.21.15.71:0/296446397 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ffbb41008e0 con 0x7ffba00a1e10 2024-03-04T08:58:32.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.554+0000 7ffbb0ff9700 1 -- 172.21.15.71:0/296446397 <== mgr.14152 v2:172.21.15.71:6800/2901868399 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7ffbb41008e0 con 0x7ffba00a1e10 2024-03-04T08:58:32.551 INFO:teuthology.orchestra.run.smithi071.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-04T08:58:32.551 INFO:teuthology.orchestra.run.smithi071.stdout:smithi071 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040172400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-04T08:58:32.552 INFO:teuthology.orchestra.run.smithi071.stdout:smithi150 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GX400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-04T08:58:32.552 INFO:teuthology.orchestra.run.smithi071.stdout:smithi158 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7475005A480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-03-04T08:58:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.557+0000 7ffbb9366700 1 -- 172.21.15.71:0/296446397 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ffba00a1e10 msgr2=0x7ffba00a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.557+0000 7ffbb9366700 1 --2- 172.21.15.71:0/296446397 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ffba00a1e10 0x7ffba00a42d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c005950 tx=0x7ffb9c00b410 comp rx=0 tx=0).stop 2024-03-04T08:58:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.557+0000 7ffbb9366700 1 -- 172.21.15.71:0/296446397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbb41012e0 msgr2=0x7ffbb4198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.557+0000 7ffbb9366700 1 --2- 172.21.15.71:0/296446397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbb41012e0 0x7ffbb4198280 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ffba400e9a0 tx=0x7ffba400ecb0 comp rx=0 tx=0).stop 2024-03-04T08:58:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.557+0000 7ffbb9366700 1 -- 172.21.15.71:0/296446397 shutdown_connections 2024-03-04T08:58:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.557+0000 7ffbb9366700 1 --2- 172.21.15.71:0/296446397 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7ffba00a1e10 0x7ffba00a42d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.557+0000 7ffbb9366700 1 --2- 172.21.15.71:0/296446397 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ffbb41025a0 0x7ffbb419f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.557+0000 7ffbb9366700 1 --2- 172.21.15.71:0/296446397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbb41012e0 0x7ffbb4198280 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.557+0000 7ffbb9366700 1 --2- 172.21.15.71:0/296446397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffbb4101c80 0x7ffbb41987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.557+0000 7ffbb9366700 1 -- 172.21.15.71:0/296446397 >> 172.21.15.71:0/296446397 conn(0x7ffbb40fbd10 msgr2=0x7ffbb41001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.557+0000 7ffbb9366700 1 -- 172.21.15.71:0/296446397 shutdown_connections 2024-03-04T08:58:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:32.557+0000 7ffbb9366700 1 -- 172.21.15.71:0/296446397 wait complete. 2024-03-04T08:58:33.403 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-04T08:58:33.414 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi071.front.sepia.ceph.com 2024-03-04T08:58:33.415 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-04T08:58:33.671 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:58:34.017 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:33 smithi071 ceph-mon[32789]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:34.017 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:33 smithi071 ceph-mon[32789]: pgmap v390: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 311 B/s wr, 2 op/s 2024-03-04T08:58:34.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:33 smithi150 ceph-mon[36807]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:34.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:33 smithi150 ceph-mon[36807]: pgmap v390: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 311 B/s wr, 2 op/s 2024-03-04T08:58:34.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:33 smithi158 ceph-mon[37241]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:34.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:33 smithi158 ceph-mon[37241]: pgmap v390: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 311 B/s wr, 2 op/s 2024-03-04T08:58:35.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.298+0000 7f4e0a3f7700 1 -- 172.21.15.71:0/1207614837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e04100ff0 msgr2=0x7f4e04101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:35.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.298+0000 7f4e0a3f7700 1 --2- 172.21.15.71:0/1207614837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e04100ff0 0x7f4e04101410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4df80099c0 tx=0x7f4df8009cd0 comp rx=0 tx=0).stop 2024-03-04T08:58:35.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.298+0000 7f4e0a3f7700 1 -- 172.21.15.71:0/1207614837 shutdown_connections 2024-03-04T08:58:35.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.298+0000 7f4e0a3f7700 1 --2- 172.21.15.71:0/1207614837 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4e04102b90 0x7f4e04109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:35.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.298+0000 7f4e0a3f7700 1 --2- 172.21.15.71:0/1207614837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e04100ff0 0x7f4e04101410 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:35.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.298+0000 7f4e0a3f7700 1 --2- 172.21.15.71:0/1207614837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e041021f0 0x7f4e04102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:35.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.298+0000 7f4e0a3f7700 1 -- 172.21.15.71:0/1207614837 >> 172.21.15.71:0/1207614837 conn(0x7f4e040fc590 msgr2=0x7f4e040fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:35.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.298+0000 7f4e0a3f7700 1 -- 172.21.15.71:0/1207614837 shutdown_connections 2024-03-04T08:58:35.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.298+0000 7f4e0a3f7700 1 -- 172.21.15.71:0/1207614837 wait complete. 2024-03-04T08:58:35.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.299+0000 7f4e0a3f7700 1 Processor -- start 2024-03-04T08:58:35.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.299+0000 7f4e0a3f7700 1 -- start start 2024-03-04T08:58:35.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.299+0000 7f4e0a3f7700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4e04100ff0 0x7f4e04197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:35.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.299+0000 7f4e0a3f7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e041021f0 0x7f4e04197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:35.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.299+0000 7f4e0a3f7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e04102b90 0x7f4e0419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:35.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.299+0000 7f4e0a3f7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0410cbc0 con 0x7f4e041021f0 2024-03-04T08:58:35.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.299+0000 7f4e0a3f7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0410ca40 con 0x7f4e04102b90 2024-03-04T08:58:35.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.299+0000 7f4e0a3f7700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0410cd40 con 0x7f4e04100ff0 2024-03-04T08:58:35.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.299+0000 7f4e037fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e041021f0 0x7f4e04197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:35.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.299+0000 7f4e037fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e041021f0 0x7f4e04197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44024/0 (socket says 172.21.15.71:44024) 2024-03-04T08:58:35.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.299+0000 7f4e037fe700 1 -- 172.21.15.71:0/1475813029 learned_addr learned my addr 172.21.15.71:0/1475813029 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:35.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.300+0000 7f4e08994700 1 --2- 172.21.15.71:0/1475813029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e04102b90 0x7f4e0419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:35.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.300+0000 7f4e03fff700 1 --2- 172.21.15.71:0/1475813029 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4e04100ff0 0x7f4e04197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:35.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.300+0000 7f4e037fe700 1 -- 172.21.15.71:0/1475813029 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4e04100ff0 msgr2=0x7f4e04197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:35.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.300+0000 7f4e037fe700 1 --2- 172.21.15.71:0/1475813029 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4e04100ff0 0x7f4e04197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:35.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.300+0000 7f4e037fe700 1 -- 172.21.15.71:0/1475813029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e04102b90 msgr2=0x7f4e0419e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:35.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.300+0000 7f4e037fe700 1 --2- 172.21.15.71:0/1475813029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e04102b90 0x7f4e0419e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:35.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.300+0000 7f4e037fe700 1 -- 172.21.15.71:0/1475813029 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0419ee70 con 0x7f4e041021f0 2024-03-04T08:58:35.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.300+0000 7f4e08994700 1 --2- 172.21.15.71:0/1475813029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e04102b90 0x7f4e0419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:58:35.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.300+0000 7f4e037fe700 1 --2- 172.21.15.71:0/1475813029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e041021f0 0x7f4e04197750 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4df400e9d0 tx=0x7f4df400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:35.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.300+0000 7f4e017fa700 1 -- 172.21.15.71:0/1475813029 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df400cbd0 con 0x7f4e041021f0 2024-03-04T08:58:35.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.300+0000 7f4e017fa700 1 -- 172.21.15.71:0/1475813029 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4df400ee70 con 0x7f4e041021f0 2024-03-04T08:58:35.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.300+0000 7f4e0a3f7700 1 -- 172.21.15.71:0/1475813029 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e0419f160 con 0x7f4e041021f0 2024-03-04T08:58:35.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.300+0000 7f4e017fa700 1 -- 172.21.15.71:0/1475813029 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df4018660 con 0x7f4e041021f0 2024-03-04T08:58:35.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.300+0000 7f4e0a3f7700 1 -- 172.21.15.71:0/1475813029 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e0419f6a0 con 0x7f4e041021f0 2024-03-04T08:58:35.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.301+0000 7f4e0a3f7700 1 -- 172.21.15.71:0/1475813029 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e04068d80 con 0x7f4e041021f0 2024-03-04T08:58:35.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.302+0000 7f4e017fa700 1 -- 172.21.15.71:0/1475813029 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f4df4010430 con 0x7f4e041021f0 2024-03-04T08:58:35.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.303+0000 7f4e017fa700 1 --2- 172.21.15.71:0/1475813029 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f4dec0a1e10 0x7f4dec0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:35.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.303+0000 7f4e03fff700 1 --2- 172.21.15.71:0/1475813029 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f4dec0a1e10 0x7f4dec0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:35.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.303+0000 7f4e017fa700 1 -- 172.21.15.71:0/1475813029 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f4df40ce560 con 0x7f4e041021f0 2024-03-04T08:58:35.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.303+0000 7f4e03fff700 1 --2- 172.21.15.71:0/1475813029 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f4dec0a1e10 0x7f4dec0a42d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4df8000c00 tx=0x7f4df8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:35.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.306+0000 7f4e017fa700 1 -- 172.21.15.71:0/1475813029 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4df409b610 con 0x7f4e041021f0 2024-03-04T08:58:35.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.546+0000 7f4e0a3f7700 1 -- 172.21.15.71:0/1475813029 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4e0402d9e0 con 0x7f4e041021f0 2024-03-04T08:58:35.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.546+0000 7f4e017fa700 1 -- 172.21.15.71:0/1475813029 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+2958 (secure 0 0 0) 0x7f4df400cd70 con 0x7f4e041021f0 2024-03-04T08:58:35.543 INFO:teuthology.orchestra.run.smithi071.stdout:e20 2024-03-04T08:58:35.543 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T08:58:35.543 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-04T08:58:35.543 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-03-04T08:58:35.543 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:58:35.543 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-03-04T08:58:35.543 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-03-04T08:58:35.543 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 20 2024-03-04T08:58:35.543 INFO:teuthology.orchestra.run.smithi071.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-04T08:58:35.543 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-03-04T08:56:30.343357+0000 2024-03-04T08:58:35.544 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-03-04T08:56:54.901214+0000 2024-03-04T08:58:35.544 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-03-04T08:58:35.544 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-03-04T08:58:35.544 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 300 2024-03-04T08:58:35.544 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-03-04T08:58:35.544 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-03-04T08:58:35.544 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-03-04T08:58:35.544 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-03-04T08:58:35.544 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-03-04T08:58:35.544 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 0 2024-03-04T08:58:35.544 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-04T08:58:35.544 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 1 2024-03-04T08:58:35.544 INFO:teuthology.orchestra.run.smithi071.stdout:in 0 2024-03-04T08:58:35.545 INFO:teuthology.orchestra.run.smithi071.stdout:up {0=24491} 2024-03-04T08:58:35.545 INFO:teuthology.orchestra.run.smithi071.stdout:failed 2024-03-04T08:58:35.545 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-03-04T08:58:35.545 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-03-04T08:58:35.545 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3,4] 2024-03-04T08:58:35.545 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-03-04T08:58:35.545 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data disabled 2024-03-04T08:58:35.545 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-03-04T08:58:35.545 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-03-04T08:58:35.545 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 0 2024-03-04T08:58:35.545 INFO:teuthology.orchestra.run.smithi071.stdout:qdb_cluster leader: 24491 members: 24491 2024-03-04T08:58:35.545 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.b{0:24491} state up:active seq 3 addr [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T08:58:35.545 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.i{0:24472} state up:standby-replay seq 1 addr [v2:172.21.15.158:6838/2921704804,v1:172.21.15.158:6839/2921704804] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T08:58:35.545 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:58:35.545 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:58:35.546 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-03-04T08:58:35.546 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:58:35.546 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.j{-1:14550} state up:standby seq 1 addr [v2:172.21.15.71:6834/3975379807,v1:172.21.15.71:6835/3975379807] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T08:58:35.546 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.d{-1:14568} state up:standby seq 1 addr [v2:172.21.15.71:6836/1962929347,v1:172.21.15.71:6837/1962929347] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T08:58:35.546 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.g{-1:14634} state up:standby seq 1 addr [v2:172.21.15.71:6838/1459252176,v1:172.21.15.71:6839/1459252176] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T08:58:35.546 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.a{-1:14646} state up:standby seq 1 addr [v2:172.21.15.71:6840/1083546600,v1:172.21.15.71:6841/1083546600] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T08:58:35.546 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.l{-1:24391} state up:standby seq 1 addr [v2:172.21.15.158:6832/178179528,v1:172.21.15.158:6833/178179528] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T08:58:35.546 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.k{-1:24413} state up:standby seq 1 addr [v2:172.21.15.150:6832/2468877675,v1:172.21.15.150:6833/2468877675] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T08:58:35.546 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.f{-1:24430} state up:standby seq 1 addr [v2:172.21.15.158:6834/1652094309,v1:172.21.15.158:6835/1652094309] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T08:58:35.546 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.e{-1:24437} state up:standby seq 1 addr [v2:172.21.15.150:6834/1802416462,v1:172.21.15.150:6835/1802416462] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T08:58:35.546 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.c{-1:24442} state up:standby seq 1 addr [v2:172.21.15.158:6836/617058380,v1:172.21.15.158:6837/617058380] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T08:58:35.546 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.h{-1:24461} state up:standby seq 1 addr [v2:172.21.15.150:6836/913465600,v1:172.21.15.150:6837/913465600] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T08:58:35.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.549+0000 7f4e0a3f7700 1 -- 172.21.15.71:0/1475813029 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f4dec0a1e10 msgr2=0x7f4dec0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:35.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.549+0000 7f4e0a3f7700 1 --2- 172.21.15.71:0/1475813029 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f4dec0a1e10 0x7f4dec0a42d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4df8000c00 tx=0x7f4df8005fb0 comp rx=0 tx=0).stop 2024-03-04T08:58:35.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.549+0000 7f4e0a3f7700 1 -- 172.21.15.71:0/1475813029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e041021f0 msgr2=0x7f4e04197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:35.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.549+0000 7f4e0a3f7700 1 --2- 172.21.15.71:0/1475813029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e041021f0 0x7f4e04197750 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4df400e9d0 tx=0x7f4df400ece0 comp rx=0 tx=0).stop 2024-03-04T08:58:35.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.550+0000 7f4e0a3f7700 1 -- 172.21.15.71:0/1475813029 shutdown_connections 2024-03-04T08:58:35.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.550+0000 7f4e0a3f7700 1 --2- 172.21.15.71:0/1475813029 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f4dec0a1e10 0x7f4dec0a42d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:35.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.550+0000 7f4e0a3f7700 1 --2- 172.21.15.71:0/1475813029 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4e04100ff0 0x7f4e04197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:35.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.550+0000 7f4e0a3f7700 1 --2- 172.21.15.71:0/1475813029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e04102b90 0x7f4e0419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:35.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.550+0000 7f4e0a3f7700 1 --2- 172.21.15.71:0/1475813029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e041021f0 0x7f4e04197750 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:35.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.550+0000 7f4e0a3f7700 1 -- 172.21.15.71:0/1475813029 >> 172.21.15.71:0/1475813029 conn(0x7f4e040fc590 msgr2=0x7f4e04105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:35.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.550+0000 7f4e0a3f7700 1 -- 172.21.15.71:0/1475813029 shutdown_connections 2024-03-04T08:58:35.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:35.550+0000 7f4e0a3f7700 1 -- 172.21.15.71:0/1475813029 wait complete. 2024-03-04T08:58:35.548 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 20 2024-03-04T08:58:36.116 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:35 smithi071 ceph-mon[32789]: pgmap v391: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 282 B/s wr, 2 op/s 2024-03-04T08:58:36.116 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:35 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1475813029' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T08:58:36.145 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-04T08:58:36.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:35 smithi150 ceph-mon[36807]: pgmap v391: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 282 B/s wr, 2 op/s 2024-03-04T08:58:36.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:35 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1475813029' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T08:58:36.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:35 smithi158 ceph-mon[37241]: pgmap v391: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 282 B/s wr, 2 op/s 2024-03-04T08:58:36.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:35 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1475813029' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T08:58:36.390 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:58:38.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.113+0000 7f8d2e11c700 1 -- 172.21.15.71:0/3974852351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d281032f0 msgr2=0x7f8d28103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:38.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.113+0000 7f8d2e11c700 1 --2- 172.21.15.71:0/3974852351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d281032f0 0x7f8d28103750 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0099c0 tx=0x7f8d1c009cd0 comp rx=0 tx=0).stop 2024-03-04T08:58:38.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.113+0000 7f8d2e11c700 1 -- 172.21.15.71:0/3974852351 shutdown_connections 2024-03-04T08:58:38.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.113+0000 7f8d2e11c700 1 --2- 172.21.15.71:0/3974852351 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d281020f0 0x7f8d28102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:38.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.113+0000 7f8d2e11c700 1 --2- 172.21.15.71:0/3974852351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d28103c90 0x7f8d2810a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:38.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.113+0000 7f8d2e11c700 1 --2- 172.21.15.71:0/3974852351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d281032f0 0x7f8d28103750 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:38.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.113+0000 7f8d2e11c700 1 -- 172.21.15.71:0/3974852351 >> 172.21.15.71:0/3974852351 conn(0x7f8d280fd6d0 msgr2=0x7f8d280ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:38.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.113+0000 7f8d2e11c700 1 -- 172.21.15.71:0/3974852351 shutdown_connections 2024-03-04T08:58:38.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.113+0000 7f8d2e11c700 1 -- 172.21.15.71:0/3974852351 wait complete. 2024-03-04T08:58:38.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.113+0000 7f8d2e11c700 1 Processor -- start 2024-03-04T08:58:38.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.114+0000 7f8d2e11c700 1 -- start start 2024-03-04T08:58:38.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.114+0000 7f8d2e11c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d281020f0 0x7f8d28075e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:38.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.114+0000 7f8d2e11c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d281032f0 0x7f8d280744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:38.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.114+0000 7f8d2e11c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d28103c90 0x7f8d280749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:38.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.114+0000 7f8d2e11c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2810dcd0 con 0x7f8d28103c90 2024-03-04T08:58:38.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.114+0000 7f8d2e11c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2810db50 con 0x7f8d281020f0 2024-03-04T08:58:38.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.114+0000 7f8d2e11c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2810de50 con 0x7f8d281032f0 2024-03-04T08:58:38.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.114+0000 7f8d27fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d28103c90 0x7f8d280749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:38.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.114+0000 7f8d27fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d28103c90 0x7f8d280749f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60642/0 (socket says 172.21.15.71:60642) 2024-03-04T08:58:38.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.114+0000 7f8d27fff700 1 -- 172.21.15.71:0/617995203 learned_addr learned my addr 172.21.15.71:0/617995203 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:38.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.114+0000 7f8d277fe700 1 --2- 172.21.15.71:0/617995203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d281020f0 0x7f8d28075e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:38.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.114+0000 7f8d26ffd700 1 --2- 172.21.15.71:0/617995203 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d281032f0 0x7f8d280744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:38.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.114+0000 7f8d27fff700 1 -- 172.21.15.71:0/617995203 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d281032f0 msgr2=0x7f8d280744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:38.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.114+0000 7f8d27fff700 1 --2- 172.21.15.71:0/617995203 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d281032f0 0x7f8d280744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:38.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.114+0000 7f8d27fff700 1 -- 172.21.15.71:0/617995203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d281020f0 msgr2=0x7f8d28075e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:38.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.114+0000 7f8d27fff700 1 --2- 172.21.15.71:0/617995203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d281020f0 0x7f8d28075e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:38.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.114+0000 7f8d27fff700 1 -- 172.21.15.71:0/617995203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d280752a0 con 0x7f8d28103c90 2024-03-04T08:58:38.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.114+0000 7f8d26ffd700 1 --2- 172.21.15.71:0/617995203 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d281032f0 0x7f8d280744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:58:38.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.114+0000 7f8d27fff700 1 --2- 172.21.15.71:0/617995203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d28103c90 0x7f8d280749f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8d2000e940 tx=0x7f8d2000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:38.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.115+0000 7f8d24ff9700 1 -- 172.21.15.71:0/617995203 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2000cbd0 con 0x7f8d28103c90 2024-03-04T08:58:38.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.115+0000 7f8d24ff9700 1 -- 172.21.15.71:0/617995203 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d2000ee90 con 0x7f8d28103c90 2024-03-04T08:58:38.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.115+0000 7f8d2e11c700 1 -- 172.21.15.71:0/617995203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d281a24a0 con 0x7f8d28103c90 2024-03-04T08:58:38.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.115+0000 7f8d24ff9700 1 -- 172.21.15.71:0/617995203 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d200056c0 con 0x7f8d28103c90 2024-03-04T08:58:38.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.115+0000 7f8d2e11c700 1 -- 172.21.15.71:0/617995203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d281a2960 con 0x7f8d28103c90 2024-03-04T08:58:38.117 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.116+0000 7f8d2e11c700 1 -- 172.21.15.71:0/617995203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d28068d80 con 0x7f8d28103c90 2024-03-04T08:58:38.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.117+0000 7f8d24ff9700 1 -- 172.21.15.71:0/617995203 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f8d20004750 con 0x7f8d28103c90 2024-03-04T08:58:38.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.117+0000 7f8d24ff9700 1 --2- 172.21.15.71:0/617995203 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f8d100a1e10 0x7f8d100a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:38.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.117+0000 7f8d277fe700 1 --2- 172.21.15.71:0/617995203 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f8d100a1e10 0x7f8d100a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:38.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.117+0000 7f8d24ff9700 1 -- 172.21.15.71:0/617995203 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f8d200ce340 con 0x7f8d28103c90 2024-03-04T08:58:38.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.118+0000 7f8d277fe700 1 --2- 172.21.15.71:0/617995203 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f8d100a1e10 0x7f8d100a42d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8d28103150 tx=0x7f8d18009180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:38.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.122+0000 7f8d24ff9700 1 -- 172.21.15.71:0/617995203 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8d2009b490 con 0x7f8d28103c90 2024-03-04T08:58:38.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:37 smithi071 ceph-mon[32789]: pgmap v392: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T08:58:38.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:37 smithi150 ceph-mon[36807]: pgmap v392: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T08:58:38.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:37 smithi158 ceph-mon[37241]: pgmap v392: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T08:58:38.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.317+0000 7f8d2e11c700 1 -- 172.21.15.71:0/617995203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f8d2802d9b0 con 0x7f8d28103c90 2024-03-04T08:58:38.314 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 95 2024-03-04T08:58:38.314 INFO:teuthology.orchestra.run.smithi071.stdout:fsid b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T08:58:38.314 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-03-04T08:47:28.546102+0000 2024-03-04T08:58:38.314 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-03-04T08:58:23.853934+0000 2024-03-04T08:58:38.314 INFO:teuthology.orchestra.run.smithi071.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-04T08:58:38.314 INFO:teuthology.orchestra.run.smithi071.stdout:crush_version 27 2024-03-04T08:58:38.315 INFO:teuthology.orchestra.run.smithi071.stdout:full_ratio 0.95 2024-03-04T08:58:38.315 INFO:teuthology.orchestra.run.smithi071.stdout:backfillfull_ratio 0.9 2024-03-04T08:58:38.315 INFO:teuthology.orchestra.run.smithi071.stdout:nearfull_ratio 0.85 2024-03-04T08:58:38.315 INFO:teuthology.orchestra.run.smithi071.stdout:require_min_compat_client luminous 2024-03-04T08:58:38.315 INFO:teuthology.orchestra.run.smithi071.stdout:min_compat_client luminous 2024-03-04T08:58:38.315 INFO:teuthology.orchestra.run.smithi071.stdout:require_osd_release squid 2024-03-04T08:58:38.315 INFO:teuthology.orchestra.run.smithi071.stdout:stretch_mode_enabled false 2024-03-04T08:58:38.315 INFO:teuthology.orchestra.run.smithi071.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-04T08:58:38.315 INFO:teuthology.orchestra.run.smithi071.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-04T08:58:38.315 INFO:teuthology.orchestra.run.smithi071.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-03-04T08:58:38.315 INFO:teuthology.orchestra.run.smithi071.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 91 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-03-04T08:58:38.315 INFO:teuthology.orchestra.run.smithi071.stdout:max_osd 12 2024-03-04T08:58:38.316 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 up in weight 1 up_from 8 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6802/1173641530,v1:172.21.15.71:6803/1173641530] [v2:172.21.15.71:6804/1173641530,v1:172.21.15.71:6805/1173641530] exists,up 7e43b47d-8888-4631-bc96-8a96fcf8cd46 2024-03-04T08:58:38.316 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 up in weight 1 up_from 13 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6800/3383948110,v1:172.21.15.150:6801/3383948110] [v2:172.21.15.150:6802/3383948110,v1:172.21.15.150:6803/3383948110] exists,up 1209f214-f2e2-4209-bf69-4528a43a2123 2024-03-04T08:58:38.316 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 up in weight 1 up_from 18 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.158:6800/586944144,v1:172.21.15.158:6801/586944144] [v2:172.21.15.158:6802/586944144,v1:172.21.15.158:6803/586944144] exists,up b88c3082-93a9-4c9b-9221-a7105a44c91a 2024-03-04T08:58:38.316 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 up in weight 1 up_from 26 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6810/1311903262,v1:172.21.15.71:6811/1311903262] [v2:172.21.15.71:6812/1311903262,v1:172.21.15.71:6813/1311903262] exists,up e4fc7723-fd73-40b2-b9d0-7f9bda282ece 2024-03-04T08:58:38.316 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 up in weight 1 up_from 34 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6808/2300837573,v1:172.21.15.150:6809/2300837573] [v2:172.21.15.150:6810/2300837573,v1:172.21.15.150:6811/2300837573] exists,up e7eb72ac-f066-4ccd-b2bf-22a6a26daf41 2024-03-04T08:58:38.316 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 up in weight 1 up_from 39 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.158:6808/1922694045,v1:172.21.15.158:6809/1922694045] [v2:172.21.15.158:6810/1922694045,v1:172.21.15.158:6811/1922694045] exists,up 9b295840-e104-4a66-9cc4-7020627c1765 2024-03-04T08:58:38.319 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6 up in weight 1 up_from 47 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6818/3848842707,v1:172.21.15.71:6819/3848842707] [v2:172.21.15.71:6820/3848842707,v1:172.21.15.71:6821/3848842707] exists,up 6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2 2024-03-04T08:58:38.319 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7 up in weight 1 up_from 52 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6816/1328330305,v1:172.21.15.150:6817/1328330305] [v2:172.21.15.150:6818/1328330305,v1:172.21.15.150:6819/1328330305] exists,up 6160a40a-d30f-4569-949c-0afb1bb3d133 2024-03-04T08:58:38.319 INFO:teuthology.orchestra.run.smithi071.stdout:osd.8 up in weight 1 up_from 61 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.158:6816/2195304577,v1:172.21.15.158:6817/2195304577] [v2:172.21.15.158:6818/2195304577,v1:172.21.15.158:6819/2195304577] exists,up 95ed2341-84a4-465c-9948-ac667680ce34 2024-03-04T08:58:38.319 INFO:teuthology.orchestra.run.smithi071.stdout:osd.9 up in weight 1 up_from 69 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6826/1913779347,v1:172.21.15.71:6827/1913779347] [v2:172.21.15.71:6828/1913779347,v1:172.21.15.71:6829/1913779347] exists,up 088f6419-63a0-4768-afae-dbb8a5f5a565 2024-03-04T08:58:38.319 INFO:teuthology.orchestra.run.smithi071.stdout:osd.10 up in weight 1 up_from 74 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6824/3295420996,v1:172.21.15.150:6825/3295420996] [v2:172.21.15.150:6826/3295420996,v1:172.21.15.150:6827/3295420996] exists,up 18c85e24-068d-4f1e-a3b8-f7915b363234 2024-03-04T08:58:38.319 INFO:teuthology.orchestra.run.smithi071.stdout:osd.11 up in weight 1 up_from 82 up_thru 94 down_at 0 last_clean_interval [0,0) [v2:172.21.15.158:6824/3332083865,v1:172.21.15.158:6825/3332083865] [v2:172.21.15.158:6826/3332083865,v1:172.21.15.158:6827/3332083865] exists,up 4f6b3ddf-c482-4740-b1fd-de836e047c2d 2024-03-04T08:58:38.320 INFO:teuthology.orchestra.run.smithi071.stdout:pg_upmap_items 2.5 [8,11] 2024-03-04T08:58:38.320 INFO:teuthology.orchestra.run.smithi071.stdout:pg_upmap_items 2.a [2,11] 2024-03-04T08:58:38.320 INFO:teuthology.orchestra.run.smithi071.stdout:pg_upmap_items 2.14 [2,4] 2024-03-04T08:58:38.320 INFO:teuthology.orchestra.run.smithi071.stdout:pg_upmap_items 2.1a [8,1] 2024-03-04T08:58:38.320 INFO:teuthology.orchestra.run.smithi071.stdout:pg_upmap_items 3.13 [4,9] 2024-03-04T08:58:38.320 INFO:teuthology.orchestra.run.smithi071.stdout:pg_upmap_items 3.25 [5,9] 2024-03-04T08:58:38.320 INFO:teuthology.orchestra.run.smithi071.stdout:pg_upmap_items 3.3c [4,9] 2024-03-04T08:58:38.320 INFO:teuthology.orchestra.run.smithi071.stdout:pg_upmap_items 4.9 [5,8] 2024-03-04T08:58:38.320 INFO:teuthology.orchestra.run.smithi071.stdout:pg_upmap_items 4.b [9,10] 2024-03-04T08:58:38.320 INFO:teuthology.orchestra.run.smithi071.stdout:pg_upmap_items 4.14 [9,8] 2024-03-04T08:58:38.320 INFO:teuthology.orchestra.run.smithi071.stdout:pg_upmap_items 4.16 [9,8] 2024-03-04T08:58:38.320 INFO:teuthology.orchestra.run.smithi071.stdout:blocklist 172.21.15.71:0/1287712358 expires 2024-03-05T08:48:22.593931+0000 2024-03-04T08:58:38.320 INFO:teuthology.orchestra.run.smithi071.stdout:blocklist 172.21.15.71:0/3770452860 expires 2024-03-05T08:48:22.593931+0000 2024-03-04T08:58:38.321 INFO:teuthology.orchestra.run.smithi071.stdout:blocklist 172.21.15.71:0/718743094 expires 2024-03-05T08:47:55.502914+0000 2024-03-04T08:58:38.321 INFO:teuthology.orchestra.run.smithi071.stdout:blocklist 172.21.15.71:6801/1350134536 expires 2024-03-05T08:47:55.502914+0000 2024-03-04T08:58:38.321 INFO:teuthology.orchestra.run.smithi071.stdout:blocklist 172.21.15.71:0/1904717532 expires 2024-03-05T08:47:55.502914+0000 2024-03-04T08:58:38.321 INFO:teuthology.orchestra.run.smithi071.stdout:blocklist 172.21.15.71:6801/2808156336 expires 2024-03-05T08:48:22.593931+0000 2024-03-04T08:58:38.321 INFO:teuthology.orchestra.run.smithi071.stdout:blocklist 172.21.15.71:6800/1350134536 expires 2024-03-05T08:47:55.502914+0000 2024-03-04T08:58:38.321 INFO:teuthology.orchestra.run.smithi071.stdout:blocklist 172.21.15.71:6800/2808156336 expires 2024-03-05T08:48:22.593931+0000 2024-03-04T08:58:38.321 INFO:teuthology.orchestra.run.smithi071.stdout:blocklist 172.21.15.71:0/2862184313 expires 2024-03-05T08:47:55.502914+0000 2024-03-04T08:58:38.321 INFO:teuthology.orchestra.run.smithi071.stdout:blocklist 172.21.15.71:0/3621754630 expires 2024-03-05T08:48:22.593931+0000 2024-03-04T08:58:38.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.318+0000 7f8d24ff9700 1 -- 172.21.15.71:0/617995203 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v95) v1 ==== 56+0+5637 (secure 0 0 0) 0x7f8d2009ad90 con 0x7f8d28103c90 2024-03-04T08:58:38.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.320+0000 7f8d2e11c700 1 -- 172.21.15.71:0/617995203 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f8d100a1e10 msgr2=0x7f8d100a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:38.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.320+0000 7f8d2e11c700 1 --2- 172.21.15.71:0/617995203 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f8d100a1e10 0x7f8d100a42d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8d28103150 tx=0x7f8d18009180 comp rx=0 tx=0).stop 2024-03-04T08:58:38.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.320+0000 7f8d2e11c700 1 -- 172.21.15.71:0/617995203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d28103c90 msgr2=0x7f8d280749f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:38.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.320+0000 7f8d2e11c700 1 --2- 172.21.15.71:0/617995203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d28103c90 0x7f8d280749f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8d2000e940 tx=0x7f8d2000ec50 comp rx=0 tx=0).stop 2024-03-04T08:58:38.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.320+0000 7f8d2e11c700 1 -- 172.21.15.71:0/617995203 shutdown_connections 2024-03-04T08:58:38.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.320+0000 7f8d2e11c700 1 --2- 172.21.15.71:0/617995203 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f8d100a1e10 0x7f8d100a42d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:38.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.320+0000 7f8d2e11c700 1 --2- 172.21.15.71:0/617995203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d28103c90 0x7f8d280749f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:38.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.320+0000 7f8d2e11c700 1 --2- 172.21.15.71:0/617995203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d281020f0 0x7f8d28075e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:38.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.320+0000 7f8d2e11c700 1 --2- 172.21.15.71:0/617995203 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d281032f0 0x7f8d280744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:38.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.320+0000 7f8d2e11c700 1 -- 172.21.15.71:0/617995203 >> 172.21.15.71:0/617995203 conn(0x7f8d280fd6d0 msgr2=0x7f8d281065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:38.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.320+0000 7f8d2e11c700 1 -- 172.21.15.71:0/617995203 shutdown_connections 2024-03-04T08:58:38.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:38.320+0000 7f8d2e11c700 1 -- 172.21.15.71:0/617995203 wait complete. 2024-03-04T08:58:38.988 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:38 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/617995203' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-04T08:58:39.014 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-04T08:58:39.025 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T08:58:39.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.135+0000 7f00b7b5c640 1 -- 172.21.15.71:0/580670473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00b01016e0 msgr2=0x7f00b0101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:39.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.135+0000 7f00b7b5c640 1 --2- 172.21.15.71:0/580670473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00b01016e0 0x7f00b0101b60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009860 tx=0x7f00a002bb70 comp rx=0 tx=0).stop 2024-03-04T08:58:39.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.135+0000 7f00b7b5c640 1 -- 172.21.15.71:0/580670473 shutdown_connections 2024-03-04T08:58:39.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.135+0000 7f00b7b5c640 1 --2- 172.21.15.71:0/580670473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b01020a0 0x7f00b0108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.135+0000 7f00b7b5c640 1 --2- 172.21.15.71:0/580670473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00b01016e0 0x7f00b0101b60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.135+0000 7f00b7b5c640 1 --2- 172.21.15.71:0/580670473 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f00b01004f0 0x7f00b0100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.135+0000 7f00b7b5c640 1 -- 172.21.15.71:0/580670473 >> 172.21.15.71:0/580670473 conn(0x7f00b00fba60 msgr2=0x7f00b00fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:39.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.135+0000 7f00b7b5c640 1 -- 172.21.15.71:0/580670473 shutdown_connections 2024-03-04T08:58:39.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.135+0000 7f00b7b5c640 1 -- 172.21.15.71:0/580670473 wait complete. 2024-03-04T08:58:39.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.135+0000 7f00b7b5c640 1 Processor -- start 2024-03-04T08:58:39.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.135+0000 7f00b7b5c640 1 -- start start 2024-03-04T08:58:39.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.135+0000 7f00b7b5c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b01004f0 0x7f00b006a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:39.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.135+0000 7f00b7b5c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00b01016e0 0x7f00b019c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:39.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.135+0000 7f00b7b5c640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f00b01020a0 0x7f00b019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:39.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.136+0000 7f00b7b5c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b010c180 con 0x7f00b01016e0 2024-03-04T08:58:39.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.136+0000 7f00b7b5c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b010c000 con 0x7f00b01004f0 2024-03-04T08:58:39.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.136+0000 7f00b7b5c640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b010c300 con 0x7f00b01020a0 2024-03-04T08:58:39.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.136+0000 7f00b50f7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00b01016e0 0x7f00b019c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:39.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.136+0000 7f00b50f7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00b01016e0 0x7f00b019c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60664/0 (socket says 172.21.15.71:60664) 2024-03-04T08:58:39.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.136+0000 7f00b50f7640 1 -- 172.21.15.71:0/2184931800 learned_addr learned my addr 172.21.15.71:0/2184931800 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:39.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.136+0000 7f00b58f8640 1 --2- 172.21.15.71:0/2184931800 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b01004f0 0x7f00b006a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:39.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.136+0000 7f00b60f9640 1 --2- 172.21.15.71:0/2184931800 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f00b01020a0 0x7f00b019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:39.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.136+0000 7f00b50f7640 1 -- 172.21.15.71:0/2184931800 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f00b01020a0 msgr2=0x7f00b019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:39.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.136+0000 7f00b50f7640 1 --2- 172.21.15.71:0/2184931800 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f00b01020a0 0x7f00b019ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.136+0000 7f00b50f7640 1 -- 172.21.15.71:0/2184931800 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b01004f0 msgr2=0x7f00b006a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:39.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.136+0000 7f00b50f7640 1 --2- 172.21.15.71:0/2184931800 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b01004f0 0x7f00b006a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.136+0000 7f00b50f7640 1 -- 172.21.15.71:0/2184931800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b019f560 con 0x7f00b01016e0 2024-03-04T08:58:39.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.136+0000 7f00b50f7640 1 --2- 172.21.15.71:0/2184931800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00b01016e0 0x7f00b019c630 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f00a0005940 tx=0x7f00a00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:39.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.136+0000 7f00a6ffd640 1 -- 172.21.15.71:0/2184931800 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0041020 con 0x7f00b01016e0 2024-03-04T08:58:39.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.136+0000 7f00b7b5c640 1 -- 172.21.15.71:0/2184931800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b019f790 con 0x7f00b01016e0 2024-03-04T08:58:39.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.136+0000 7f00a6ffd640 1 -- 172.21.15.71:0/2184931800 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00a00323f0 con 0x7f00b01016e0 2024-03-04T08:58:39.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.136+0000 7f00a6ffd640 1 -- 172.21.15.71:0/2184931800 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0039040 con 0x7f00b01016e0 2024-03-04T08:58:39.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.136+0000 7f00b7b5c640 1 -- 172.21.15.71:0/2184931800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b019fc80 con 0x7f00b01016e0 2024-03-04T08:58:39.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.137+0000 7f00b7b5c640 1 -- 172.21.15.71:0/2184931800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b0107810 con 0x7f00b01016e0 2024-03-04T08:58:39.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.138+0000 7f00a6ffd640 1 -- 172.21.15.71:0/2184931800 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f00a0044020 con 0x7f00b01016e0 2024-03-04T08:58:39.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.139+0000 7f00a6ffd640 1 --2- 172.21.15.71:0/2184931800 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f009c0a1d80 0x7f009c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:39.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.139+0000 7f00b58f8640 1 --2- 172.21.15.71:0/2184931800 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f009c0a1d80 0x7f009c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:39.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.139+0000 7f00a6ffd640 1 -- 172.21.15.71:0/2184931800 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f00a00efc40 con 0x7f00b01016e0 2024-03-04T08:58:39.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.139+0000 7f00b58f8640 1 --2- 172.21.15.71:0/2184931800 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f009c0a1d80 0x7f009c0a4240 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f00ac005fd0 tx=0x7f00ac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:39.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.143+0000 7f00a6ffd640 1 -- 172.21.15.71:0/2184931800 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f00a004b020 con 0x7f00b01016e0 2024-03-04T08:58:39.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:38 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/617995203' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-04T08:58:39.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:38 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/617995203' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-04T08:58:39.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.317+0000 7f00b7b5c640 1 -- 172.21.15.71:0/2184931800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00b00621d0 con 0x7f00b01016e0 2024-03-04T08:58:39.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.318+0000 7f00a6ffd640 1 -- 172.21.15.71:0/2184931800 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24727 (secure 0 0 0) 0x7f00a00bccf0 con 0x7f00b01016e0 2024-03-04T08:58:39.315 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:58:39.315 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":95,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","created":"2024-03-04T08:47:28.546102+0000","modified":"2024-03-04T08:58:23.853934+0000","last_up_change":"2024-03-04T08:55:30.744961+0000","last_in_change":"2024-03-04T08:55:11.570990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T08:51:14.933487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T08:56:28.253074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T08:56:29.543622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T08:56:32.884220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7e43b47d-8888-4631-bc96-8a96fcf8cd46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6803","nonce":1173641530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6805","nonce":1173641530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6809","nonce":1173641530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6807","nonce":1173641530}]},"public_addr":"172.21.15.71:6803/1173641530","cluster_addr":"172.21.15.71:6805/1173641530","heartbeat_back_addr":"172.21.15.71:6809/1173641530","heartbeat_front_addr":"172.21.15.71:6807/1173641530","state":["exists","up"]},{"osd":1,"uuid":"1209f214-f2e2-4209-bf69-4528a43a2123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6801","nonce":3383948110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6803","nonce":3383948110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6807","nonce":3383948110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6805","nonce":3383948110}]},"public_addr":"172.21.15.150:6801/3383948110","cluster_addr":"172.21.15.150:6803/3383948110","heartbeat_back_addr":"172.21.15.150:6807/3383948110","heartbeat_front_addr":"172.21.15.150:6805/3383948110","state":["exists","up"]},{"osd":2,"uuid":"b88c3082-93a9-4c9b-9221-a7105a44c91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6801","nonce":586944144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6803","nonce":586944144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6807","nonce":586944144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6805","nonce":586944144}]},"public_addr":"172.21.15.158:6801/586944144","cluster_addr":"172.21.15.158:6803/586944144","heartbeat_back_addr":"172.21.15.158:6807/586944144","heartbeat_front_addr":"172.21.15.158:6805/586944144","state":["exists","up"]},{"osd":3,"uuid":"e4fc7723-fd73-40b2-b9d0-7f9bda282ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6811","nonce":1311903262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6813","nonce":1311903262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6817","nonce":1311903262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6815","nonce":1311903262}]},"public_addr":"172.21.15.71:6811/1311903262","cluster_addr":"172.21.15.71:6813/1311903262","heartbeat_back_addr":"172.21.15.71:6817/1311903262","heartbeat_front_addr":"172.21.15.71:6815/1311903262","state":["exists","up"]},{"osd":4,"uuid":"e7eb72ac-f066-4ccd-b2bf-22a6a26daf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6809","nonce":2300837573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6811","nonce":2300837573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6815","nonce":2300837573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6813","nonce":2300837573}]},"public_addr":"172.21.15.150:6809/2300837573","cluster_addr":"172.21.15.150:6811/2300837573","heartbeat_back_addr":"172.21.15.150:6815/2300837573","heartbeat_front_addr":"172.21.15.150:6813/2300837573","state":["exists","up"]},{"osd":5,"uuid":"9b295840-e104-4a66-9cc4-7020627c1765","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6809","nonce":1922694045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6811","nonce":1922694045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6815","nonce":1922694045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6813","nonce":1922694045}]},"public_addr":"172.21.15.158:6809/1922694045","cluster_addr":"172.21.15.158:6811/1922694045","heartbeat_back_addr":"172.21.15.158:6815/1922694045","heartbeat_front_addr":"172.21.15.158:6813/1922694045","state":["exists","up"]},{"osd":6,"uuid":"6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6819","nonce":3848842707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6821","nonce":3848842707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6825","nonce":3848842707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6823","nonce":3848842707}]},"public_addr":"172.21.15.71:6819/3848842707","cluster_addr":"172.21.15.71:6821/3848842707","heartbeat_back_addr":"172.21.15.71:6825/3848842707","heartbeat_front_addr":"172.21.15.71:6823/3848842707","state":["exists","up"]},{"osd":7,"uuid":"6160a40a-d30f-4569-949c-0afb1bb3d133","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6817","nonce":1328330305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6819","nonce":1328330305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6823","nonce":1328330305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6821","nonce":1328330305}]},"public_addr":"172.21.15.150:6817/1328330305","cluster_addr":"172.21.15.150:6819/1328330305","heartbeat_back_addr":"172.21.15.150:6823/1328330305","heartbeat_front_addr":"172.21.15.150:6821/1328330305","state":["exists","up"]},{"osd":8,"uuid":"95ed2341-84a4-465c-9948-ac667680ce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6817","nonce":2195304577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6819","nonce":2195304577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6823","nonce":2195304577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6821","nonce":2195304577}]},"public_addr":"172.21.15.158:6817/2195304577","cluster_addr":"172.21.15.158:6819/2195304577","heartbeat_back_addr":"172.21.15.158:6823/2195304577","heartbeat_front_addr":"172.21.15.158:6821/2195304577","state":["exists","up"]},{"osd":9,"uuid":"088f6419-63a0-4768-afae-dbb8a5f5a565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6827","nonce":1913779347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6829","nonce":1913779347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6833","nonce":1913779347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6831","nonce":1913779347}]},"public_addr":"172.21.15.71:6827/1913779347","cluster_addr":"172.21.15.71:6829/1913779347","heartbeat_back_addr":"172.21.15.71:6833/1913779347","heartbeat_front_addr":"172.21.15.71:6831/1913779347","state":["exists","up"]},{"osd":10,"uuid":"18c85e24-068d-4f1e-a3b8-f7915b363234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6825","nonce":3295420996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6827","nonce":3295420996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6831","nonce":3295420996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6829","nonce":3295420996}]},"public_addr":"172.21.15.150:6825/3295420996","cluster_addr":"172.21.15.150:6827/3295420996","heartbeat_back_addr":"172.21.15.150:6831/3295420996","heartbeat_front_addr":"172.21.15.150:6829/3295420996","state":["exists","up"]},{"osd":11,"uuid":"4f6b3ddf-c482-4740-b1fd-de836e047c2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6825","nonce":3332083865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6827","nonce":3332083865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6831","nonce":3332083865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6829","nonce":3332083865}]},"public_addr":"172.21.15.158:6825/3332083865","cluster_addr":"172.21.15.158:6827/3332083865","heartbeat_back_addr":"172.21.15.158:6831/3332083865","heartbeat_front_addr":"172.21.15.158:6829/3332083865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:05.997832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:29.314235+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:08.819465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:41.194275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:06.032479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:38.814498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:13.067960+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:38.568479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:03.416713+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:38.349479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:04.096945+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:29.425250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1287712358":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/3770452860":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/718743094":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/1904717532":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:6800/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6800/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/2862184313":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/3621754630":"2024-03-05T08:48:22.593931+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T08:58:39.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.320+0000 7f00b7b5c640 1 -- 172.21.15.71:0/2184931800 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f009c0a1d80 msgr2=0x7f009c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:39.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.320+0000 7f00b7b5c640 1 --2- 172.21.15.71:0/2184931800 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f009c0a1d80 0x7f009c0a4240 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f00ac005fd0 tx=0x7f00ac00b410 comp rx=0 tx=0).stop 2024-03-04T08:58:39.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.320+0000 7f00b7b5c640 1 -- 172.21.15.71:0/2184931800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00b01016e0 msgr2=0x7f00b019c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:39.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.320+0000 7f00b7b5c640 1 --2- 172.21.15.71:0/2184931800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00b01016e0 0x7f00b019c630 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f00a0005940 tx=0x7f00a00042f0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.320+0000 7f00b7b5c640 1 -- 172.21.15.71:0/2184931800 shutdown_connections 2024-03-04T08:58:39.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.320+0000 7f00b7b5c640 1 --2- 172.21.15.71:0/2184931800 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f009c0a1d80 0x7f009c0a4240 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.320+0000 7f00b7b5c640 1 --2- 172.21.15.71:0/2184931800 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f00b01020a0 0x7f00b019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.320+0000 7f00b7b5c640 1 --2- 172.21.15.71:0/2184931800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00b01016e0 0x7f00b019c630 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.320+0000 7f00b7b5c640 1 --2- 172.21.15.71:0/2184931800 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00b01004f0 0x7f00b006a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.320+0000 7f00b7b5c640 1 -- 172.21.15.71:0/2184931800 >> 172.21.15.71:0/2184931800 conn(0x7f00b00fba60 msgr2=0x7f00b01049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:39.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.321+0000 7f00b7b5c640 1 -- 172.21.15.71:0/2184931800 shutdown_connections 2024-03-04T08:58:39.317 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.321+0000 7f00b7b5c640 1 -- 172.21.15.71:0/2184931800 wait complete. 2024-03-04T08:58:39.328 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T08:51:14.933487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T08:56:28.253074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T08:56:29.543622+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T08:56:32.884220+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T08:58:39.329 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T08:58:39.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.437+0000 7f2907484640 1 -- 172.21.15.71:0/409274176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2900075610 msgr2=0x7f2900073a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:39.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.437+0000 7f2907484640 1 --2- 172.21.15.71:0/409274176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2900075610 0x7f2900073a60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009930 tx=0x7f28fc02bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:39.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.437+0000 7f2907484640 1 -- 172.21.15.71:0/409274176 shutdown_connections 2024-03-04T08:58:39.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.437+0000 7f2907484640 1 --2- 172.21.15.71:0/409274176 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f290010dbc0 0x7f290010ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.437+0000 7f2907484640 1 --2- 172.21.15.71:0/409274176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2900074030 0x7f29000744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.437+0000 7f2907484640 1 --2- 172.21.15.71:0/409274176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2900075610 0x7f2900073a60 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.437+0000 7f2907484640 1 -- 172.21.15.71:0/409274176 >> 172.21.15.71:0/409274176 conn(0x7f29000fcb60 msgr2=0x7f29000fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:39.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.437+0000 7f2907484640 1 -- 172.21.15.71:0/409274176 shutdown_connections 2024-03-04T08:58:39.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.437+0000 7f2907484640 1 -- 172.21.15.71:0/409274176 wait complete. 2024-03-04T08:58:39.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.438+0000 7f2907484640 1 Processor -- start 2024-03-04T08:58:39.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.438+0000 7f2907484640 1 -- start start 2024-03-04T08:58:39.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.438+0000 7f2907484640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2900074030 0x7f290019df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:39.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.438+0000 7f2907484640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2900075610 0x7f290019e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:39.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.438+0000 7f2907484640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f290010dbc0 0x7f29001a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:39.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.438+0000 7f2907484640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29001138b0 con 0x7f2900074030 2024-03-04T08:58:39.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.438+0000 7f2907484640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2900113730 con 0x7f290010dbc0 2024-03-04T08:58:39.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.438+0000 7f2907484640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2900113a30 con 0x7f2900075610 2024-03-04T08:58:39.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.438+0000 7f2905220640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2900074030 0x7f290019df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:39.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.438+0000 7f2905220640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2900074030 0x7f290019df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60688/0 (socket says 172.21.15.71:60688) 2024-03-04T08:58:39.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.438+0000 7f2905220640 1 -- 172.21.15.71:0/2767047509 learned_addr learned my addr 172.21.15.71:0/2767047509 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:39.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.438+0000 7f2905a21640 1 --2- 172.21.15.71:0/2767047509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f290010dbc0 0x7f29001a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:39.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.438+0000 7f2905220640 1 -- 172.21.15.71:0/2767047509 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2900075610 msgr2=0x7f290019e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:39.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.438+0000 7f2905220640 1 --2- 172.21.15.71:0/2767047509 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2900075610 0x7f290019e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.438+0000 7f2905220640 1 -- 172.21.15.71:0/2767047509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f290010dbc0 msgr2=0x7f29001a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:39.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.438+0000 7f2905220640 1 --2- 172.21.15.71:0/2767047509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f290010dbc0 0x7f29001a54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.438+0000 7f2905220640 1 -- 172.21.15.71:0/2767047509 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29001a5b40 con 0x7f2900074030 2024-03-04T08:58:39.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.439+0000 7f2905220640 1 --2- 172.21.15.71:0/2767047509 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2900074030 0x7f290019df20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f28fc000c00 tx=0x7f28fc02c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:39.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.439+0000 7f28f67fc640 1 -- 172.21.15.71:0/2767047509 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc043020 con 0x7f2900074030 2024-03-04T08:58:39.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.439+0000 7f2907484640 1 -- 172.21.15.71:0/2767047509 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29001a5dd0 con 0x7f2900074030 2024-03-04T08:58:39.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.439+0000 7f28f67fc640 1 -- 172.21.15.71:0/2767047509 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28fc0054e0 con 0x7f2900074030 2024-03-04T08:58:39.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.439+0000 7f28f67fc640 1 -- 172.21.15.71:0/2767047509 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc003ce0 con 0x7f2900074030 2024-03-04T08:58:39.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.439+0000 7f2907484640 1 -- 172.21.15.71:0/2767047509 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29001a62e0 con 0x7f2900074030 2024-03-04T08:58:39.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.440+0000 7f2907484640 1 -- 172.21.15.71:0/2767047509 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2900103d90 con 0x7f2900074030 2024-03-04T08:58:39.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.441+0000 7f28f67fc640 1 -- 172.21.15.71:0/2767047509 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f28fc046020 con 0x7f2900074030 2024-03-04T08:58:39.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.441+0000 7f28f67fc640 1 --2- 172.21.15.71:0/2767047509 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f28ec0a1d80 0x7f28ec0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:39.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.442+0000 7f2904a1f640 1 --2- 172.21.15.71:0/2767047509 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f28ec0a1d80 0x7f28ec0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:39.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.442+0000 7f28f67fc640 1 -- 172.21.15.71:0/2767047509 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f28fc0f0630 con 0x7f2900074030 2024-03-04T08:58:39.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.442+0000 7f2904a1f640 1 --2- 172.21.15.71:0/2767047509 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f28ec0a1d80 0x7f28ec0a4240 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f28f00058c0 tx=0x7f28f0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:39.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.445+0000 7f28f67fc640 1 -- 172.21.15.71:0/2767047509 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f28fc04d020 con 0x7f2900074030 2024-03-04T08:58:39.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.618+0000 7f2907484640 1 -- 172.21.15.71:0/2767047509 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f290002d180 con 0x7f2900074030 2024-03-04T08:58:39.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.619+0000 7f28f67fc640 1 -- 172.21.15.71:0/2767047509 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f28fc0bd6e0 con 0x7f2900074030 2024-03-04T08:58:39.615 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-03-04T08:58:39.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.621+0000 7f2907484640 1 -- 172.21.15.71:0/2767047509 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f28ec0a1d80 msgr2=0x7f28ec0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:39.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.621+0000 7f2907484640 1 --2- 172.21.15.71:0/2767047509 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f28ec0a1d80 0x7f28ec0a4240 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f28f00058c0 tx=0x7f28f0005850 comp rx=0 tx=0).stop 2024-03-04T08:58:39.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.621+0000 7f2907484640 1 -- 172.21.15.71:0/2767047509 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2900074030 msgr2=0x7f290019df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:39.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.621+0000 7f2907484640 1 --2- 172.21.15.71:0/2767047509 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2900074030 0x7f290019df20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f28fc000c00 tx=0x7f28fc02c6c0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.621+0000 7f2907484640 1 -- 172.21.15.71:0/2767047509 shutdown_connections 2024-03-04T08:58:39.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.621+0000 7f2907484640 1 --2- 172.21.15.71:0/2767047509 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f28ec0a1d80 0x7f28ec0a4240 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.621+0000 7f2907484640 1 --2- 172.21.15.71:0/2767047509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f290010dbc0 0x7f29001a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.621+0000 7f2907484640 1 --2- 172.21.15.71:0/2767047509 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2900075610 0x7f290019e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.621+0000 7f2907484640 1 --2- 172.21.15.71:0/2767047509 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2900074030 0x7f290019df20 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.621+0000 7f2907484640 1 -- 172.21.15.71:0/2767047509 >> 172.21.15.71:0/2767047509 conn(0x7f29000fcb60 msgr2=0x7f2900100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:39.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.621+0000 7f2907484640 1 -- 172.21.15.71:0/2767047509 shutdown_connections 2024-03-04T08:58:39.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.621+0000 7f2907484640 1 -- 172.21.15.71:0/2767047509 wait complete. 2024-03-04T08:58:39.628 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T08:58:39.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.737+0000 7f423bb56640 1 -- 172.21.15.71:0/2180857592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4234104980 msgr2=0x7f4234106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:39.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.737+0000 7f423bb56640 1 --2- 172.21.15.71:0/2180857592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4234104980 0x7f4234106d70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4224009900 tx=0x7f422402bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:39.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.737+0000 7f423bb56640 1 -- 172.21.15.71:0/2180857592 shutdown_connections 2024-03-04T08:58:39.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.737+0000 7f423bb56640 1 --2- 172.21.15.71:0/2180857592 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f42341072b0 0x7f4234109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.737+0000 7f423bb56640 1 --2- 172.21.15.71:0/2180857592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4234104980 0x7f4234106d70 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.737+0000 7f423bb56640 1 --2- 172.21.15.71:0/2180857592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4234102050 0x7f4234104440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.737+0000 7f423bb56640 1 -- 172.21.15.71:0/2180857592 >> 172.21.15.71:0/2180857592 conn(0x7f42340fba60 msgr2=0x7f42340fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:39.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.738+0000 7f423bb56640 1 -- 172.21.15.71:0/2180857592 shutdown_connections 2024-03-04T08:58:39.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.738+0000 7f423bb56640 1 -- 172.21.15.71:0/2180857592 wait complete. 2024-03-04T08:58:39.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.738+0000 7f423bb56640 1 Processor -- start 2024-03-04T08:58:39.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.738+0000 7f423bb56640 1 -- start start 2024-03-04T08:58:39.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.738+0000 7f423bb56640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4234102050 0x7f42341019c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:39.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.738+0000 7f423bb56640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4234104980 0x7f4234100000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:39.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.738+0000 7f423bb56640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f42341072b0 0x7f4234100540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:39.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.738+0000 7f423bb56640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423410d260 con 0x7f4234104980 2024-03-04T08:58:39.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.738+0000 7f423bb56640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423410d0e0 con 0x7f4234102050 2024-03-04T08:58:39.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.738+0000 7f423bb56640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423410d3e0 con 0x7f42341072b0 2024-03-04T08:58:39.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.738+0000 7f42390f1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4234104980 0x7f4234100000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:39.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.738+0000 7f42390f1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4234104980 0x7f4234100000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60712/0 (socket says 172.21.15.71:60712) 2024-03-04T08:58:39.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.738+0000 7f42390f1640 1 -- 172.21.15.71:0/2524167448 learned_addr learned my addr 172.21.15.71:0/2524167448 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:39.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.738+0000 7f42398f2640 1 --2- 172.21.15.71:0/2524167448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4234102050 0x7f42341019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:39.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.738+0000 7f423a0f3640 1 --2- 172.21.15.71:0/2524167448 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f42341072b0 0x7f4234100540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:39.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.739+0000 7f42390f1640 1 -- 172.21.15.71:0/2524167448 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f42341072b0 msgr2=0x7f4234100540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:39.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.739+0000 7f42390f1640 1 --2- 172.21.15.71:0/2524167448 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f42341072b0 0x7f4234100540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.739+0000 7f42390f1640 1 -- 172.21.15.71:0/2524167448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4234102050 msgr2=0x7f42341019c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:39.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.739+0000 7f42390f1640 1 --2- 172.21.15.71:0/2524167448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4234102050 0x7f42341019c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.739+0000 7f42390f1640 1 -- 172.21.15.71:0/2524167448 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4234100dd0 con 0x7f4234104980 2024-03-04T08:58:39.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.739+0000 7f423a0f3640 1 --2- 172.21.15.71:0/2524167448 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f42341072b0 0x7f4234100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:58:39.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.739+0000 7f42390f1640 1 --2- 172.21.15.71:0/2524167448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4234104980 0x7f4234100000 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4224000c00 tx=0x7f4224005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:39.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.739+0000 7f422affd640 1 -- 172.21.15.71:0/2524167448 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4224041070 con 0x7f4234104980 2024-03-04T08:58:39.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.739+0000 7f423bb56640 1 -- 172.21.15.71:0/2524167448 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42341a1a00 con 0x7f4234104980 2024-03-04T08:58:39.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.739+0000 7f422affd640 1 -- 172.21.15.71:0/2524167448 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42240044d0 con 0x7f4234104980 2024-03-04T08:58:39.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.739+0000 7f422affd640 1 -- 172.21.15.71:0/2524167448 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4224034440 con 0x7f4234104980 2024-03-04T08:58:39.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.739+0000 7f423bb56640 1 -- 172.21.15.71:0/2524167448 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42341a1f50 con 0x7f4234104980 2024-03-04T08:58:39.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.740+0000 7f423bb56640 1 -- 172.21.15.71:0/2524167448 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42340fd550 con 0x7f4234104980 2024-03-04T08:58:39.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.741+0000 7f422affd640 1 -- 172.21.15.71:0/2524167448 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f4224044070 con 0x7f4234104980 2024-03-04T08:58:39.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.742+0000 7f422affd640 1 --2- 172.21.15.71:0/2524167448 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f42200a1d80 0x7f42200a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:39.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.742+0000 7f42398f2640 1 --2- 172.21.15.71:0/2524167448 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f42200a1d80 0x7f42200a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:39.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.742+0000 7f422affd640 1 -- 172.21.15.71:0/2524167448 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f42240efeb0 con 0x7f4234104980 2024-03-04T08:58:39.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.742+0000 7f42398f2640 1 --2- 172.21.15.71:0/2524167448 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f42200a1d80 0x7f42200a4240 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f42300099a0 tx=0x7f4230009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:39.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.745+0000 7f422affd640 1 -- 172.21.15.71:0/2524167448 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f42240bcf60 con 0x7f4234104980 2024-03-04T08:58:39.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.919+0000 7f423bb56640 1 -- 172.21.15.71:0/2524167448 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f423402d180 con 0x7f4234104980 2024-03-04T08:58:39.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.919+0000 7f422affd640 1 -- 172.21.15.71:0/2524167448 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f42240bc860 con 0x7f4234104980 2024-03-04T08:58:39.915 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-03-04T08:58:39.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.921+0000 7f423bb56640 1 -- 172.21.15.71:0/2524167448 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f42200a1d80 msgr2=0x7f42200a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:39.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.921+0000 7f423bb56640 1 --2- 172.21.15.71:0/2524167448 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f42200a1d80 0x7f42200a4240 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f42300099a0 tx=0x7f4230009500 comp rx=0 tx=0).stop 2024-03-04T08:58:39.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.921+0000 7f423bb56640 1 -- 172.21.15.71:0/2524167448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4234104980 msgr2=0x7f4234100000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:39.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.921+0000 7f423bb56640 1 --2- 172.21.15.71:0/2524167448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4234104980 0x7f4234100000 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4224000c00 tx=0x7f4224005d50 comp rx=0 tx=0).stop 2024-03-04T08:58:39.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.921+0000 7f423bb56640 1 -- 172.21.15.71:0/2524167448 shutdown_connections 2024-03-04T08:58:39.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.921+0000 7f423bb56640 1 --2- 172.21.15.71:0/2524167448 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f42200a1d80 0x7f42200a4240 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.921+0000 7f423bb56640 1 --2- 172.21.15.71:0/2524167448 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f42341072b0 0x7f4234100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.921+0000 7f423bb56640 1 --2- 172.21.15.71:0/2524167448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4234104980 0x7f4234100000 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.921+0000 7f423bb56640 1 --2- 172.21.15.71:0/2524167448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4234102050 0x7f42341019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:39.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.921+0000 7f423bb56640 1 -- 172.21.15.71:0/2524167448 >> 172.21.15.71:0/2524167448 conn(0x7f42340fba60 msgr2=0x7f4234102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:39.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.921+0000 7f423bb56640 1 -- 172.21.15.71:0/2524167448 shutdown_connections 2024-03-04T08:58:39.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:39.921+0000 7f423bb56640 1 -- 172.21.15.71:0/2524167448 wait complete. 2024-03-04T08:58:39.928 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T08:58:40.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.036+0000 7f31e56ed640 1 -- 172.21.15.71:0/2918071953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31e0075590 msgr2=0x7f31e00739e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:40.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.036+0000 7f31e56ed640 1 --2- 172.21.15.71:0/2918071953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31e0075590 0x7f31e00739e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f31d0009900 tx=0x7f31d002bcd0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.036+0000 7f31e56ed640 1 -- 172.21.15.71:0/2918071953 shutdown_connections 2024-03-04T08:58:40.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.036+0000 7f31e56ed640 1 --2- 172.21.15.71:0/2918071953 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f31e0109490 0x7f31e010b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.036+0000 7f31e56ed640 1 --2- 172.21.15.71:0/2918071953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31e0073fb0 0x7f31e0074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.036+0000 7f31e56ed640 1 --2- 172.21.15.71:0/2918071953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31e0075590 0x7f31e00739e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.036+0000 7f31e56ed640 1 -- 172.21.15.71:0/2918071953 >> 172.21.15.71:0/2918071953 conn(0x7f31e00fcb70 msgr2=0x7f31e00fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:40.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.036+0000 7f31e56ed640 1 -- 172.21.15.71:0/2918071953 shutdown_connections 2024-03-04T08:58:40.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.036+0000 7f31e56ed640 1 -- 172.21.15.71:0/2918071953 wait complete. 2024-03-04T08:58:40.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.036+0000 7f31e56ed640 1 Processor -- start 2024-03-04T08:58:40.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.037+0000 7f31e56ed640 1 -- start start 2024-03-04T08:58:40.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.037+0000 7f31e56ed640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f31e0073fb0 0x7f31e0199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:40.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.037+0000 7f31e56ed640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31e0075590 0x7f31e019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:40.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.037+0000 7f31e56ed640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31e0109490 0x7f31e01a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:40.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.037+0000 7f31e56ed640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e010f460 con 0x7f31e0109490 2024-03-04T08:58:40.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.037+0000 7f31e56ed640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e010f2e0 con 0x7f31e0075590 2024-03-04T08:58:40.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.037+0000 7f31e56ed640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e010f5e0 con 0x7f31e0073fb0 2024-03-04T08:58:40.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.037+0000 7f31df7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31e0109490 0x7f31e01a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:40.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.037+0000 7f31df7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31e0109490 0x7f31e01a1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60736/0 (socket says 172.21.15.71:60736) 2024-03-04T08:58:40.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.037+0000 7f31df7fe640 1 -- 172.21.15.71:0/1238102623 learned_addr learned my addr 172.21.15.71:0/1238102623 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:40.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.037+0000 7f31de7fc640 1 --2- 172.21.15.71:0/1238102623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31e0075590 0x7f31e019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:40.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.037+0000 7f31deffd640 1 --2- 172.21.15.71:0/1238102623 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f31e0073fb0 0x7f31e0199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:40.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.037+0000 7f31df7fe640 1 -- 172.21.15.71:0/1238102623 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f31e0073fb0 msgr2=0x7f31e0199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:40.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.037+0000 7f31df7fe640 1 --2- 172.21.15.71:0/1238102623 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f31e0073fb0 0x7f31e0199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.037+0000 7f31df7fe640 1 -- 172.21.15.71:0/1238102623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31e0075590 msgr2=0x7f31e019a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:40.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.037+0000 7f31df7fe640 1 --2- 172.21.15.71:0/1238102623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31e0075590 0x7f31e019a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.037+0000 7f31df7fe640 1 -- 172.21.15.71:0/1238102623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e01a1770 con 0x7f31e0109490 2024-03-04T08:58:40.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.037+0000 7f31deffd640 1 --2- 172.21.15.71:0/1238102623 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f31e0073fb0 0x7f31e0199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:58:40.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.037+0000 7f31df7fe640 1 --2- 172.21.15.71:0/1238102623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31e0109490 0x7f31e01a1100 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f31d800eaa0 tx=0x7f31d800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:40.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.037+0000 7f31c7fff640 1 -- 172.21.15.71:0/1238102623 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d800cdb0 con 0x7f31e0109490 2024-03-04T08:58:40.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.038+0000 7f31c7fff640 1 -- 172.21.15.71:0/1238102623 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31d8005020 con 0x7f31e0109490 2024-03-04T08:58:40.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.038+0000 7f31e56ed640 1 -- 172.21.15.71:0/1238102623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e01a1a60 con 0x7f31e0109490 2024-03-04T08:58:40.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.038+0000 7f31c7fff640 1 -- 172.21.15.71:0/1238102623 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d80116b0 con 0x7f31e0109490 2024-03-04T08:58:40.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.038+0000 7f31e56ed640 1 -- 172.21.15.71:0/1238102623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e01a1fb0 con 0x7f31e0109490 2024-03-04T08:58:40.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.038+0000 7f31e56ed640 1 -- 172.21.15.71:0/1238102623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e0108250 con 0x7f31e0109490 2024-03-04T08:58:40.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.040+0000 7f31c7fff640 1 -- 172.21.15.71:0/1238102623 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f31d8005560 con 0x7f31e0109490 2024-03-04T08:58:40.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.040+0000 7f31c7fff640 1 --2- 172.21.15.71:0/1238102623 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f31c80a1d80 0x7f31c80a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:40.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.040+0000 7f31deffd640 1 --2- 172.21.15.71:0/1238102623 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f31c80a1d80 0x7f31c80a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:40.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.040+0000 7f31c7fff640 1 -- 172.21.15.71:0/1238102623 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f31d80ce6d0 con 0x7f31e0109490 2024-03-04T08:58:40.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.041+0000 7f31deffd640 1 --2- 172.21.15.71:0/1238102623 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f31c80a1d80 0x7f31c80a4240 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f31d00096c0 tx=0x7f31d0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:40.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.044+0000 7f31c7fff640 1 -- 172.21.15.71:0/1238102623 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f31d8016030 con 0x7f31e0109490 2024-03-04T08:58:40.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:39 smithi071 ceph-mon[32789]: pgmap v393: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:40.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:39 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2184931800' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:58:40.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:39 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2767047509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:58:40.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.216+0000 7f31e56ed640 1 -- 172.21.15.71:0/1238102623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f31e002d240 con 0x7f31e0109490 2024-03-04T08:58:40.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.216+0000 7f31c7fff640 1 -- 172.21.15.71:0/1238102623 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f31d809b780 con 0x7f31e0109490 2024-03-04T08:58:40.212 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2024-03-04T08:58:40.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.218+0000 7f31e56ed640 1 -- 172.21.15.71:0/1238102623 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f31c80a1d80 msgr2=0x7f31c80a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:40.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.218+0000 7f31e56ed640 1 --2- 172.21.15.71:0/1238102623 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f31c80a1d80 0x7f31c80a4240 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f31d00096c0 tx=0x7f31d0000bc0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.218+0000 7f31e56ed640 1 -- 172.21.15.71:0/1238102623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31e0109490 msgr2=0x7f31e01a1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:40.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.218+0000 7f31e56ed640 1 --2- 172.21.15.71:0/1238102623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31e0109490 0x7f31e01a1100 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f31d800eaa0 tx=0x7f31d800ef60 comp rx=0 tx=0).stop 2024-03-04T08:58:40.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.218+0000 7f31e56ed640 1 -- 172.21.15.71:0/1238102623 shutdown_connections 2024-03-04T08:58:40.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.218+0000 7f31e56ed640 1 --2- 172.21.15.71:0/1238102623 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f31c80a1d80 0x7f31c80a4240 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.218+0000 7f31e56ed640 1 --2- 172.21.15.71:0/1238102623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31e0109490 0x7f31e01a1100 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.218+0000 7f31e56ed640 1 --2- 172.21.15.71:0/1238102623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31e0075590 0x7f31e019a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.218+0000 7f31e56ed640 1 --2- 172.21.15.71:0/1238102623 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f31e0073fb0 0x7f31e0199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.218+0000 7f31e56ed640 1 -- 172.21.15.71:0/1238102623 >> 172.21.15.71:0/1238102623 conn(0x7f31e00fcb70 msgr2=0x7f31e0105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:40.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.218+0000 7f31e56ed640 1 -- 172.21.15.71:0/1238102623 shutdown_connections 2024-03-04T08:58:40.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.218+0000 7f31e56ed640 1 -- 172.21.15.71:0/1238102623 wait complete. 2024-03-04T08:58:40.226 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T08:58:40.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:39 smithi150 ceph-mon[36807]: pgmap v393: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:40.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:39 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2184931800' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:58:40.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:39 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2767047509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:58:40.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:39 smithi158 ceph-mon[37241]: pgmap v393: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:40.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:39 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2184931800' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:58:40.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:39 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2767047509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:58:40.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.334+0000 7f29d208f640 1 -- 172.21.15.71:0/4134841202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29cc102070 msgr2=0x7f29cc104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:40.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.334+0000 7f29d208f640 1 --2- 172.21.15.71:0/4134841202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29cc102070 0x7f29cc104460 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f29c4009be0 tx=0x7f29c402bef0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.334+0000 7f29d208f640 1 -- 172.21.15.71:0/4134841202 shutdown_connections 2024-03-04T08:58:40.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.334+0000 7f29d208f640 1 --2- 172.21.15.71:0/4134841202 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f29cc1072d0 0x7f29cc109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.334+0000 7f29d208f640 1 --2- 172.21.15.71:0/4134841202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29cc1049a0 0x7f29cc106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.334+0000 7f29d208f640 1 --2- 172.21.15.71:0/4134841202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29cc102070 0x7f29cc104460 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.334+0000 7f29d208f640 1 -- 172.21.15.71:0/4134841202 >> 172.21.15.71:0/4134841202 conn(0x7f29cc0fba80 msgr2=0x7f29cc0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:40.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.334+0000 7f29d208f640 1 -- 172.21.15.71:0/4134841202 shutdown_connections 2024-03-04T08:58:40.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.334+0000 7f29d208f640 1 -- 172.21.15.71:0/4134841202 wait complete. 2024-03-04T08:58:40.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.335+0000 7f29d208f640 1 Processor -- start 2024-03-04T08:58:40.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.335+0000 7f29d208f640 1 -- start start 2024-03-04T08:58:40.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.335+0000 7f29d208f640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f29cc102070 0x7f29cc197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:40.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.335+0000 7f29d208f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29cc1049a0 0x7f29cc197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:40.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.335+0000 7f29d208f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29cc1072d0 0x7f29cc19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:40.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.335+0000 7f29d208f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc10d280 con 0x7f29cc1072d0 2024-03-04T08:58:40.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.335+0000 7f29d208f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc10d100 con 0x7f29cc1049a0 2024-03-04T08:58:40.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.335+0000 7f29d208f640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc10d400 con 0x7f29cc102070 2024-03-04T08:58:40.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.335+0000 7f29cbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29cc1072d0 0x7f29cc19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:40.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.335+0000 7f29caffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29cc1049a0 0x7f29cc197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:40.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.335+0000 7f29cbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29cc1072d0 0x7f29cc19eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60754/0 (socket says 172.21.15.71:60754) 2024-03-04T08:58:40.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.335+0000 7f29caffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29cc1049a0 0x7f29cc197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.71:56900/0 (socket says 172.21.15.71:56900) 2024-03-04T08:58:40.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.335+0000 7f29cbfff640 1 -- 172.21.15.71:0/2949077483 learned_addr learned my addr 172.21.15.71:0/2949077483 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:40.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.335+0000 7f29cb7fe640 1 --2- 172.21.15.71:0/2949077483 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f29cc102070 0x7f29cc197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:40.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.335+0000 7f29cbfff640 1 -- 172.21.15.71:0/2949077483 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f29cc102070 msgr2=0x7f29cc197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:40.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.335+0000 7f29cbfff640 1 --2- 172.21.15.71:0/2949077483 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f29cc102070 0x7f29cc197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.335+0000 7f29cbfff640 1 -- 172.21.15.71:0/2949077483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29cc1049a0 msgr2=0x7f29cc197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:40.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.335+0000 7f29cbfff640 1 --2- 172.21.15.71:0/2949077483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29cc1049a0 0x7f29cc197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.336+0000 7f29cbfff640 1 -- 172.21.15.71:0/2949077483 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc19f550 con 0x7f29cc1072d0 2024-03-04T08:58:40.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.336+0000 7f29cbfff640 1 --2- 172.21.15.71:0/2949077483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29cc1072d0 0x7f29cc19eee0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00e970 tx=0x7f29bc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:40.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.336+0000 7f29c8ff9640 1 -- 172.21.15.71:0/2949077483 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc00ccc0 con 0x7f29cc1072d0 2024-03-04T08:58:40.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.336+0000 7f29c8ff9640 1 -- 172.21.15.71:0/2949077483 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29bc00ce60 con 0x7f29cc1072d0 2024-03-04T08:58:40.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.336+0000 7f29d208f640 1 -- 172.21.15.71:0/2949077483 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29cc19f840 con 0x7f29cc1072d0 2024-03-04T08:58:40.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.336+0000 7f29c8ff9640 1 -- 172.21.15.71:0/2949077483 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc010490 con 0x7f29cc1072d0 2024-03-04T08:58:40.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.336+0000 7f29d208f640 1 -- 172.21.15.71:0/2949077483 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29cc19fd90 con 0x7f29cc1072d0 2024-03-04T08:58:40.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.337+0000 7f29d208f640 1 -- 172.21.15.71:0/2949077483 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29cc0fd570 con 0x7f29cc1072d0 2024-03-04T08:58:40.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.338+0000 7f29c8ff9640 1 -- 172.21.15.71:0/2949077483 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f29bc005020 con 0x7f29cc1072d0 2024-03-04T08:58:40.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.338+0000 7f29c8ff9640 1 --2- 172.21.15.71:0/2949077483 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f29b80a1d80 0x7f29b80a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:40.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.339+0000 7f29cb7fe640 1 --2- 172.21.15.71:0/2949077483 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f29b80a1d80 0x7f29b80a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:40.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.339+0000 7f29c8ff9640 1 -- 172.21.15.71:0/2949077483 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f29bc0ce550 con 0x7f29cc1072d0 2024-03-04T08:58:40.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.339+0000 7f29cb7fe640 1 --2- 172.21.15.71:0/2949077483 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f29b80a1d80 0x7f29b80a4240 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f29c4006010 tx=0x7f29c4000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:40.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.343+0000 7f29c8ff9640 1 -- 172.21.15.71:0/2949077483 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f29bc09b600 con 0x7f29cc1072d0 2024-03-04T08:58:40.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.516+0000 7f29d208f640 1 -- 172.21.15.71:0/2949077483 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f29cc0621d0 con 0x7f29cc1072d0 2024-03-04T08:58:40.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.517+0000 7f29c8ff9640 1 -- 172.21.15.71:0/2949077483 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7f29bc09af00 con 0x7f29cc1072d0 2024-03-04T08:58:40.513 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-03-04T08:58:40.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.519+0000 7f29d208f640 1 -- 172.21.15.71:0/2949077483 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f29b80a1d80 msgr2=0x7f29b80a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:40.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.519+0000 7f29d208f640 1 --2- 172.21.15.71:0/2949077483 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f29b80a1d80 0x7f29b80a4240 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f29c4006010 tx=0x7f29c4000f40 comp rx=0 tx=0).stop 2024-03-04T08:58:40.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.519+0000 7f29d208f640 1 -- 172.21.15.71:0/2949077483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29cc1072d0 msgr2=0x7f29cc19eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:40.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.519+0000 7f29d208f640 1 --2- 172.21.15.71:0/2949077483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29cc1072d0 0x7f29cc19eee0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00e970 tx=0x7f29bc00ed70 comp rx=0 tx=0).stop 2024-03-04T08:58:40.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.519+0000 7f29d208f640 1 -- 172.21.15.71:0/2949077483 shutdown_connections 2024-03-04T08:58:40.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.519+0000 7f29d208f640 1 --2- 172.21.15.71:0/2949077483 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f29b80a1d80 0x7f29b80a4240 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.519+0000 7f29d208f640 1 --2- 172.21.15.71:0/2949077483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29cc1072d0 0x7f29cc19eee0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.519+0000 7f29d208f640 1 --2- 172.21.15.71:0/2949077483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29cc1049a0 0x7f29cc197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.519+0000 7f29d208f640 1 --2- 172.21.15.71:0/2949077483 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f29cc102070 0x7f29cc197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.519+0000 7f29d208f640 1 -- 172.21.15.71:0/2949077483 >> 172.21.15.71:0/2949077483 conn(0x7f29cc0fba80 msgr2=0x7f29cc102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:40.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.519+0000 7f29d208f640 1 -- 172.21.15.71:0/2949077483 shutdown_connections 2024-03-04T08:58:40.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.519+0000 7f29d208f640 1 -- 172.21.15.71:0/2949077483 wait complete. 2024-03-04T08:58:40.526 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:58:40.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.631+0000 7f64b0f13640 1 -- 172.21.15.71:0/660871808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64ac1020a0 msgr2=0x7f64ac108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:40.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.631+0000 7f64b0f13640 1 --2- 172.21.15.71:0/660871808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64ac1020a0 0x7f64ac108930 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f649c009ca0 tx=0x7f649c02c0f0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.631+0000 7f64b0f13640 1 -- 172.21.15.71:0/660871808 shutdown_connections 2024-03-04T08:58:40.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.631+0000 7f64b0f13640 1 --2- 172.21.15.71:0/660871808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64ac1020a0 0x7f64ac108930 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.631+0000 7f64b0f13640 1 --2- 172.21.15.71:0/660871808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64ac1016e0 0x7f64ac101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.631+0000 7f64b0f13640 1 --2- 172.21.15.71:0/660871808 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f64ac1004f0 0x7f64ac100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.631+0000 7f64b0f13640 1 -- 172.21.15.71:0/660871808 >> 172.21.15.71:0/660871808 conn(0x7f64ac0fba60 msgr2=0x7f64ac0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:40.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.631+0000 7f64b0f13640 1 -- 172.21.15.71:0/660871808 shutdown_connections 2024-03-04T08:58:40.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.631+0000 7f64b0f13640 1 -- 172.21.15.71:0/660871808 wait complete. 2024-03-04T08:58:40.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.632+0000 7f64b0f13640 1 Processor -- start 2024-03-04T08:58:40.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.632+0000 7f64b0f13640 1 -- start start 2024-03-04T08:58:40.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.632+0000 7f64b0f13640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64ac1004f0 0x7f64ac069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:40.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.632+0000 7f64b0f13640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f64ac1016e0 0x7f64ac19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:40.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.632+0000 7f64b0f13640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64ac1020a0 0x7f64ac19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:40.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.632+0000 7f64b0f13640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac10c180 con 0x7f64ac1004f0 2024-03-04T08:58:40.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.632+0000 7f64b0f13640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac10c000 con 0x7f64ac1020a0 2024-03-04T08:58:40.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.632+0000 7f64b0f13640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac10c300 con 0x7f64ac1016e0 2024-03-04T08:58:40.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.632+0000 7f64aa59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64ac1004f0 0x7f64ac069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:40.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.633+0000 7f64aa59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64ac1004f0 0x7f64ac069300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60780/0 (socket says 172.21.15.71:60780) 2024-03-04T08:58:40.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.633+0000 7f64aa59c640 1 -- 172.21.15.71:0/218604386 learned_addr learned my addr 172.21.15.71:0/218604386 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:40.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.633+0000 7f64aad9d640 1 --2- 172.21.15.71:0/218604386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64ac1020a0 0x7f64ac19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:40.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.633+0000 7f64a9d9b640 1 --2- 172.21.15.71:0/218604386 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f64ac1016e0 0x7f64ac19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:40.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.633+0000 7f64aa59c640 1 -- 172.21.15.71:0/218604386 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f64ac1016e0 msgr2=0x7f64ac19c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:40.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.633+0000 7f64aa59c640 1 --2- 172.21.15.71:0/218604386 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f64ac1016e0 0x7f64ac19c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.633+0000 7f64aa59c640 1 -- 172.21.15.71:0/218604386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64ac1020a0 msgr2=0x7f64ac19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:40.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.633+0000 7f64aa59c640 1 --2- 172.21.15.71:0/218604386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64ac1020a0 0x7f64ac19ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.633+0000 7f64aa59c640 1 -- 172.21.15.71:0/218604386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ac19f470 con 0x7f64ac1004f0 2024-03-04T08:58:40.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.633+0000 7f64aad9d640 1 --2- 172.21.15.71:0/218604386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64ac1020a0 0x7f64ac19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:58:40.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.633+0000 7f64aa59c640 1 --2- 172.21.15.71:0/218604386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64ac1004f0 0x7f64ac069300 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f64a400d770 tx=0x7f64a400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:40.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.633+0000 7f64a37fe640 1 -- 172.21.15.71:0/218604386 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a4017070 con 0x7f64ac1004f0 2024-03-04T08:58:40.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.633+0000 7f64a37fe640 1 -- 172.21.15.71:0/218604386 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64a4009c30 con 0x7f64ac1004f0 2024-03-04T08:58:40.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.633+0000 7f64b0f13640 1 -- 172.21.15.71:0/218604386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ac19f760 con 0x7f64ac1004f0 2024-03-04T08:58:40.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.633+0000 7f64a37fe640 1 -- 172.21.15.71:0/218604386 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a4003da0 con 0x7f64ac1004f0 2024-03-04T08:58:40.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.633+0000 7f64b0f13640 1 -- 172.21.15.71:0/218604386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ac19fcb0 con 0x7f64ac1004f0 2024-03-04T08:58:40.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.634+0000 7f64b0f13640 1 -- 172.21.15.71:0/218604386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ac107810 con 0x7f64ac1004f0 2024-03-04T08:58:40.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.635+0000 7f64a37fe640 1 -- 172.21.15.71:0/218604386 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f64a4003f40 con 0x7f64ac1004f0 2024-03-04T08:58:40.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.636+0000 7f64a37fe640 1 --2- 172.21.15.71:0/218604386 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f64980a1d80 0x7f64980a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:40.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.636+0000 7f64a9d9b640 1 --2- 172.21.15.71:0/218604386 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f64980a1d80 0x7f64980a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:40.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.636+0000 7f64a37fe640 1 -- 172.21.15.71:0/218604386 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f64a40cd9c0 con 0x7f64ac1004f0 2024-03-04T08:58:40.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.636+0000 7f64a9d9b640 1 --2- 172.21.15.71:0/218604386 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f64980a1d80 0x7f64980a4240 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f64ac069070 tx=0x7f6494005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:40.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.640+0000 7f64a37fe640 1 -- 172.21.15.71:0/218604386 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f64a4009de0 con 0x7f64ac1004f0 2024-03-04T08:58:40.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.853+0000 7f64b0f13640 1 -- 172.21.15.71:0/218604386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64ac02cff0 con 0x7f64ac1004f0 2024-03-04T08:58:40.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.853+0000 7f64a37fe640 1 -- 172.21.15.71:0/218604386 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11339 (secure 0 0 0) 0x7f64a409a420 con 0x7f64ac1004f0 2024-03-04T08:58:40.850 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:58:40.850 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3975379807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3975379807},{"type":"v1","addr":"172.21.15.71:6835","nonce":3975379807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1962929347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1962929347},{"type":"v1","addr":"172.21.15.71:6837","nonce":1962929347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/1459252176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1459252176},{"type":"v1","addr":"172.21.15.71:6839","nonce":1459252176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6841/1083546600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":1083546600},{"type":"v1","addr":"172.21.15.71:6841","nonce":1083546600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/178179528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":178179528},{"type":"v1","addr":"172.21.15.158:6833","nonce":178179528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2468877675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2468877675},{"type":"v1","addr":"172.21.15.150:6833","nonce":2468877675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1652094309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1652094309},{"type":"v1","addr":"172.21.15.158:6835","nonce":1652094309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1802416462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1802416462},{"type":"v1","addr":"172.21.15.150:6835","nonce":1802416462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/617058380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":617058380},{"type":"v1","addr":"172.21.15.158:6837","nonce":617058380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/913465600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":913465600},{"type":"v1","addr":"172.21.15.150:6837","nonce":913465600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T08:56:30.343357+0000","modified":"2024-03-04T08:56:54.901214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.158:6839/2921704804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6838","nonce":2921704804},{"type":"v1","addr":"172.21.15.158:6839","nonce":2921704804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/2639278083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2639278083},{"type":"v1","addr":"172.21.15.150:6839","nonce":2639278083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491]},"id":1}]} 2024-03-04T08:58:40.850 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 20 2024-03-04T08:58:40.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.855+0000 7f64b0f13640 1 -- 172.21.15.71:0/218604386 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f64980a1d80 msgr2=0x7f64980a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:40.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.855+0000 7f64b0f13640 1 --2- 172.21.15.71:0/218604386 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f64980a1d80 0x7f64980a4240 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f64ac069070 tx=0x7f6494005f90 comp rx=0 tx=0).stop 2024-03-04T08:58:40.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.855+0000 7f64b0f13640 1 -- 172.21.15.71:0/218604386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64ac1004f0 msgr2=0x7f64ac069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:40.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.855+0000 7f64b0f13640 1 --2- 172.21.15.71:0/218604386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64ac1004f0 0x7f64ac069300 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f64a400d770 tx=0x7f64a400dc30 comp rx=0 tx=0).stop 2024-03-04T08:58:40.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.855+0000 7f64b0f13640 1 -- 172.21.15.71:0/218604386 shutdown_connections 2024-03-04T08:58:40.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.855+0000 7f64b0f13640 1 --2- 172.21.15.71:0/218604386 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f64980a1d80 0x7f64980a4240 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.855+0000 7f64b0f13640 1 --2- 172.21.15.71:0/218604386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64ac1020a0 0x7f64ac19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.855+0000 7f64b0f13640 1 --2- 172.21.15.71:0/218604386 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f64ac1016e0 0x7f64ac19c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.855+0000 7f64b0f13640 1 --2- 172.21.15.71:0/218604386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64ac1004f0 0x7f64ac069300 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.855+0000 7f64b0f13640 1 -- 172.21.15.71:0/218604386 >> 172.21.15.71:0/218604386 conn(0x7f64ac0fba60 msgr2=0x7f64ac1049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:40.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.856+0000 7f64b0f13640 1 -- 172.21.15.71:0/218604386 shutdown_connections 2024-03-04T08:58:40.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.856+0000 7f64b0f13640 1 -- 172.21.15.71:0/218604386 wait complete. 2024-03-04T08:58:40.863 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T08:58:40.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.969+0000 7f2b4973a640 1 -- 172.21.15.71:0/4251581493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b441004f0 msgr2=0x7f2b44100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:40.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.969+0000 7f2b4973a640 1 --2- 172.21.15.71:0/4251581493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b441004f0 0x7f2b44100910 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2b34009ca0 tx=0x7f2b3402c0f0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.969+0000 7f2b4973a640 1 -- 172.21.15.71:0/4251581493 shutdown_connections 2024-03-04T08:58:40.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.969+0000 7f2b4973a640 1 --2- 172.21.15.71:0/4251581493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b441020a0 0x7f2b44108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.969+0000 7f2b4973a640 1 --2- 172.21.15.71:0/4251581493 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2b441016e0 0x7f2b44101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.969+0000 7f2b4973a640 1 --2- 172.21.15.71:0/4251581493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b441004f0 0x7f2b44100910 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.969+0000 7f2b4973a640 1 -- 172.21.15.71:0/4251581493 >> 172.21.15.71:0/4251581493 conn(0x7f2b440fba60 msgr2=0x7f2b440fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:40.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.969+0000 7f2b4973a640 1 -- 172.21.15.71:0/4251581493 shutdown_connections 2024-03-04T08:58:40.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.969+0000 7f2b4973a640 1 -- 172.21.15.71:0/4251581493 wait complete. 2024-03-04T08:58:40.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.970+0000 7f2b4973a640 1 Processor -- start 2024-03-04T08:58:40.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.970+0000 7f2b4973a640 1 -- start start 2024-03-04T08:58:40.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.970+0000 7f2b4973a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b441004f0 0x7f2b440692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:40.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.970+0000 7f2b4973a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b441016e0 0x7f2b4419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:40.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.970+0000 7f2b4973a640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2b441020a0 0x7f2b4419ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:40.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.970+0000 7f2b4973a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4410c180 con 0x7f2b441016e0 2024-03-04T08:58:40.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.970+0000 7f2b4973a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4410c000 con 0x7f2b441004f0 2024-03-04T08:58:40.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.970+0000 7f2b4973a640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4410c300 con 0x7f2b441020a0 2024-03-04T08:58:40.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.970+0000 7f2b427fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b441016e0 0x7f2b4419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:40.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.970+0000 7f2b427fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b441016e0 0x7f2b4419c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60792/0 (socket says 172.21.15.71:60792) 2024-03-04T08:58:40.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.970+0000 7f2b427fc640 1 -- 172.21.15.71:0/2435332518 learned_addr learned my addr 172.21.15.71:0/2435332518 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:40.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.970+0000 7f2b42ffd640 1 --2- 172.21.15.71:0/2435332518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b441004f0 0x7f2b440692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:40.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.971+0000 7f2b437fe640 1 --2- 172.21.15.71:0/2435332518 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2b441020a0 0x7f2b4419ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:40.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.971+0000 7f2b427fc640 1 -- 172.21.15.71:0/2435332518 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2b441020a0 msgr2=0x7f2b4419ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:40.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.971+0000 7f2b427fc640 1 --2- 172.21.15.71:0/2435332518 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2b441020a0 0x7f2b4419ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.971+0000 7f2b427fc640 1 -- 172.21.15.71:0/2435332518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b441004f0 msgr2=0x7f2b440692c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:40.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.971+0000 7f2b427fc640 1 --2- 172.21.15.71:0/2435332518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b441004f0 0x7f2b440692c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:40.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.971+0000 7f2b427fc640 1 -- 172.21.15.71:0/2435332518 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b4419f540 con 0x7f2b441016e0 2024-03-04T08:58:40.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.971+0000 7f2b427fc640 1 --2- 172.21.15.71:0/2435332518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b441016e0 0x7f2b4419c640 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800d950 tx=0x7f2b3800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:40.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.971+0000 7f2b2bfff640 1 -- 172.21.15.71:0/2435332518 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b38017030 con 0x7f2b441016e0 2024-03-04T08:58:40.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.971+0000 7f2b4973a640 1 -- 172.21.15.71:0/2435332518 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b4419f830 con 0x7f2b441016e0 2024-03-04T08:58:40.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.971+0000 7f2b2bfff640 1 -- 172.21.15.71:0/2435332518 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b38004020 con 0x7f2b441016e0 2024-03-04T08:58:40.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.971+0000 7f2b2bfff640 1 -- 172.21.15.71:0/2435332518 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b380047a0 con 0x7f2b441016e0 2024-03-04T08:58:40.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.971+0000 7f2b4973a640 1 -- 172.21.15.71:0/2435332518 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b4419fd80 con 0x7f2b441016e0 2024-03-04T08:58:40.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.972+0000 7f2b4973a640 1 -- 172.21.15.71:0/2435332518 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b44107890 con 0x7f2b441016e0 2024-03-04T08:58:40.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.973+0000 7f2b2bfff640 1 -- 172.21.15.71:0/2435332518 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f2b380041d0 con 0x7f2b441016e0 2024-03-04T08:58:40.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.974+0000 7f2b2bfff640 1 --2- 172.21.15.71:0/2435332518 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f2b2c0a1d80 0x7f2b2c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:40.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.974+0000 7f2b42ffd640 1 --2- 172.21.15.71:0/2435332518 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f2b2c0a1d80 0x7f2b2c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:40.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.974+0000 7f2b2bfff640 1 -- 172.21.15.71:0/2435332518 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f2b380ce250 con 0x7f2b441016e0 2024-03-04T08:58:40.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.974+0000 7f2b42ffd640 1 --2- 172.21.15.71:0/2435332518 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f2b2c0a1d80 0x7f2b2c0a4240 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2b3402c630 tx=0x7f2b34009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:40.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:40.977+0000 7f2b2bfff640 1 -- 172.21.15.71:0/2435332518 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2b3809b300 con 0x7f2b441016e0 2024-03-04T08:58:41.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.150+0000 7f2b4973a640 1 -- 172.21.15.71:0/2435332518 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b4402d240 con 0x7f2b441016e0 2024-03-04T08:58:41.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.152+0000 7f2b2bfff640 1 -- 172.21.15.71:0/2435332518 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24727 (secure 0 0 0) 0x7f2b3809ac00 con 0x7f2b441016e0 2024-03-04T08:58:41.148 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:58:41.148 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":95,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","created":"2024-03-04T08:47:28.546102+0000","modified":"2024-03-04T08:58:23.853934+0000","last_up_change":"2024-03-04T08:55:30.744961+0000","last_in_change":"2024-03-04T08:55:11.570990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T08:51:14.933487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T08:56:28.253074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T08:56:29.543622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T08:56:32.884220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7e43b47d-8888-4631-bc96-8a96fcf8cd46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6803","nonce":1173641530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6805","nonce":1173641530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6809","nonce":1173641530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6807","nonce":1173641530}]},"public_addr":"172.21.15.71:6803/1173641530","cluster_addr":"172.21.15.71:6805/1173641530","heartbeat_back_addr":"172.21.15.71:6809/1173641530","heartbeat_front_addr":"172.21.15.71:6807/1173641530","state":["exists","up"]},{"osd":1,"uuid":"1209f214-f2e2-4209-bf69-4528a43a2123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6801","nonce":3383948110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6803","nonce":3383948110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6807","nonce":3383948110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6805","nonce":3383948110}]},"public_addr":"172.21.15.150:6801/3383948110","cluster_addr":"172.21.15.150:6803/3383948110","heartbeat_back_addr":"172.21.15.150:6807/3383948110","heartbeat_front_addr":"172.21.15.150:6805/3383948110","state":["exists","up"]},{"osd":2,"uuid":"b88c3082-93a9-4c9b-9221-a7105a44c91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6801","nonce":586944144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6803","nonce":586944144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6807","nonce":586944144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6805","nonce":586944144}]},"public_addr":"172.21.15.158:6801/586944144","cluster_addr":"172.21.15.158:6803/586944144","heartbeat_back_addr":"172.21.15.158:6807/586944144","heartbeat_front_addr":"172.21.15.158:6805/586944144","state":["exists","up"]},{"osd":3,"uuid":"e4fc7723-fd73-40b2-b9d0-7f9bda282ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6811","nonce":1311903262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6813","nonce":1311903262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6817","nonce":1311903262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6815","nonce":1311903262}]},"public_addr":"172.21.15.71:6811/1311903262","cluster_addr":"172.21.15.71:6813/1311903262","heartbeat_back_addr":"172.21.15.71:6817/1311903262","heartbeat_front_addr":"172.21.15.71:6815/1311903262","state":["exists","up"]},{"osd":4,"uuid":"e7eb72ac-f066-4ccd-b2bf-22a6a26daf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6809","nonce":2300837573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6811","nonce":2300837573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6815","nonce":2300837573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6813","nonce":2300837573}]},"public_addr":"172.21.15.150:6809/2300837573","cluster_addr":"172.21.15.150:6811/2300837573","heartbeat_back_addr":"172.21.15.150:6815/2300837573","heartbeat_front_addr":"172.21.15.150:6813/2300837573","state":["exists","up"]},{"osd":5,"uuid":"9b295840-e104-4a66-9cc4-7020627c1765","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6809","nonce":1922694045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6811","nonce":1922694045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6815","nonce":1922694045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6813","nonce":1922694045}]},"public_addr":"172.21.15.158:6809/1922694045","cluster_addr":"172.21.15.158:6811/1922694045","heartbeat_back_addr":"172.21.15.158:6815/1922694045","heartbeat_front_addr":"172.21.15.158:6813/1922694045","state":["exists","up"]},{"osd":6,"uuid":"6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6819","nonce":3848842707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6821","nonce":3848842707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6825","nonce":3848842707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6823","nonce":3848842707}]},"public_addr":"172.21.15.71:6819/3848842707","cluster_addr":"172.21.15.71:6821/3848842707","heartbeat_back_addr":"172.21.15.71:6825/3848842707","heartbeat_front_addr":"172.21.15.71:6823/3848842707","state":["exists","up"]},{"osd":7,"uuid":"6160a40a-d30f-4569-949c-0afb1bb3d133","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6817","nonce":1328330305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6819","nonce":1328330305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6823","nonce":1328330305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6821","nonce":1328330305}]},"public_addr":"172.21.15.150:6817/1328330305","cluster_addr":"172.21.15.150:6819/1328330305","heartbeat_back_addr":"172.21.15.150:6823/1328330305","heartbeat_front_addr":"172.21.15.150:6821/1328330305","state":["exists","up"]},{"osd":8,"uuid":"95ed2341-84a4-465c-9948-ac667680ce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6817","nonce":2195304577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6819","nonce":2195304577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6823","nonce":2195304577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6821","nonce":2195304577}]},"public_addr":"172.21.15.158:6817/2195304577","cluster_addr":"172.21.15.158:6819/2195304577","heartbeat_back_addr":"172.21.15.158:6823/2195304577","heartbeat_front_addr":"172.21.15.158:6821/2195304577","state":["exists","up"]},{"osd":9,"uuid":"088f6419-63a0-4768-afae-dbb8a5f5a565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6827","nonce":1913779347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6829","nonce":1913779347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6833","nonce":1913779347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6831","nonce":1913779347}]},"public_addr":"172.21.15.71:6827/1913779347","cluster_addr":"172.21.15.71:6829/1913779347","heartbeat_back_addr":"172.21.15.71:6833/1913779347","heartbeat_front_addr":"172.21.15.71:6831/1913779347","state":["exists","up"]},{"osd":10,"uuid":"18c85e24-068d-4f1e-a3b8-f7915b363234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6825","nonce":3295420996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6827","nonce":3295420996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6831","nonce":3295420996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6829","nonce":3295420996}]},"public_addr":"172.21.15.150:6825/3295420996","cluster_addr":"172.21.15.150:6827/3295420996","heartbeat_back_addr":"172.21.15.150:6831/3295420996","heartbeat_front_addr":"172.21.15.150:6829/3295420996","state":["exists","up"]},{"osd":11,"uuid":"4f6b3ddf-c482-4740-b1fd-de836e047c2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6825","nonce":3332083865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6827","nonce":3332083865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6831","nonce":3332083865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6829","nonce":3332083865}]},"public_addr":"172.21.15.158:6825/3332083865","cluster_addr":"172.21.15.158:6827/3332083865","heartbeat_back_addr":"172.21.15.158:6831/3332083865","heartbeat_front_addr":"172.21.15.158:6829/3332083865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:05.997832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:29.314235+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:08.819465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:41.194275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:06.032479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:38.814498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:13.067960+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:38.568479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:03.416713+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:38.349479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:04.096945+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:29.425250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1287712358":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/3770452860":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/718743094":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/1904717532":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:6800/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6800/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/2862184313":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/3621754630":"2024-03-05T08:48:22.593931+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T08:58:41.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.154+0000 7f2b4973a640 1 -- 172.21.15.71:0/2435332518 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f2b2c0a1d80 msgr2=0x7f2b2c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:41.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.154+0000 7f2b4973a640 1 --2- 172.21.15.71:0/2435332518 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f2b2c0a1d80 0x7f2b2c0a4240 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2b3402c630 tx=0x7f2b34009360 comp rx=0 tx=0).stop 2024-03-04T08:58:41.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.154+0000 7f2b4973a640 1 -- 172.21.15.71:0/2435332518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b441016e0 msgr2=0x7f2b4419c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:41.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.154+0000 7f2b4973a640 1 --2- 172.21.15.71:0/2435332518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b441016e0 0x7f2b4419c640 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800d950 tx=0x7f2b3800de10 comp rx=0 tx=0).stop 2024-03-04T08:58:41.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.154+0000 7f2b4973a640 1 -- 172.21.15.71:0/2435332518 shutdown_connections 2024-03-04T08:58:41.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.154+0000 7f2b4973a640 1 --2- 172.21.15.71:0/2435332518 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f2b2c0a1d80 0x7f2b2c0a4240 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.154+0000 7f2b4973a640 1 --2- 172.21.15.71:0/2435332518 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f2b441020a0 0x7f2b4419ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.154+0000 7f2b4973a640 1 --2- 172.21.15.71:0/2435332518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b441016e0 0x7f2b4419c640 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.154+0000 7f2b4973a640 1 --2- 172.21.15.71:0/2435332518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b441004f0 0x7f2b440692c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.154+0000 7f2b4973a640 1 -- 172.21.15.71:0/2435332518 >> 172.21.15.71:0/2435332518 conn(0x7f2b440fba60 msgr2=0x7f2b44104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:41.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.154+0000 7f2b4973a640 1 -- 172.21.15.71:0/2435332518 shutdown_connections 2024-03-04T08:58:41.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.154+0000 7f2b4973a640 1 -- 172.21.15.71:0/2435332518 wait complete. 2024-03-04T08:58:41.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:40 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2524167448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T08:58:41.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:40 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1238102623' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T08:58:41.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:40 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2949077483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T08:58:41.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:40 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/218604386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:58:41.162 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T08:51:14.933487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T08:56:28.253074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T08:56:29.543622+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T08:56:32.884220+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T08:58:41.162 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T08:58:41.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.271+0000 7f51db66b640 1 -- 172.21.15.71:0/2493568744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d4102090 msgr2=0x7f51d4108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:41.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.271+0000 7f51db66b640 1 --2- 172.21.15.71:0/2493568744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d4102090 0x7f51d4108920 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009ca0 tx=0x7f51cc02c0f0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.271+0000 7f51db66b640 1 -- 172.21.15.71:0/2493568744 shutdown_connections 2024-03-04T08:58:41.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.271+0000 7f51db66b640 1 --2- 172.21.15.71:0/2493568744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d4102090 0x7f51d4108920 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.271+0000 7f51db66b640 1 --2- 172.21.15.71:0/2493568744 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51d41016d0 0x7f51d4101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.271+0000 7f51db66b640 1 --2- 172.21.15.71:0/2493568744 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51d41004e0 0x7f51d4100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.271+0000 7f51db66b640 1 -- 172.21.15.71:0/2493568744 >> 172.21.15.71:0/2493568744 conn(0x7f51d40fba70 msgr2=0x7f51d40fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:41.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.271+0000 7f51db66b640 1 -- 172.21.15.71:0/2493568744 shutdown_connections 2024-03-04T08:58:41.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.271+0000 7f51db66b640 1 -- 172.21.15.71:0/2493568744 wait complete. 2024-03-04T08:58:41.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.271+0000 7f51db66b640 1 Processor -- start 2024-03-04T08:58:41.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.271+0000 7f51db66b640 1 -- start start 2024-03-04T08:58:41.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.271+0000 7f51db66b640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51d41004e0 0x7f51d406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:41.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.271+0000 7f51db66b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51d41016d0 0x7f51d419c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:41.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.271+0000 7f51db66b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d4102090 0x7f51d419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:41.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.271+0000 7f51db66b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d410c170 con 0x7f51d4102090 2024-03-04T08:58:41.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.271+0000 7f51db66b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d410bff0 con 0x7f51d41016d0 2024-03-04T08:58:41.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.271+0000 7f51db66b640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d410c2f0 con 0x7f51d41004e0 2024-03-04T08:58:41.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.272+0000 7f51d9c08640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d4102090 0x7f51d419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:41.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.272+0000 7f51d9c08640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d4102090 0x7f51d419ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60806/0 (socket says 172.21.15.71:60806) 2024-03-04T08:58:41.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.272+0000 7f51d9c08640 1 -- 172.21.15.71:0/291841264 learned_addr learned my addr 172.21.15.71:0/291841264 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:41.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.272+0000 7f51d8c06640 1 --2- 172.21.15.71:0/291841264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51d41016d0 0x7f51d419c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:41.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.272+0000 7f51d9407640 1 --2- 172.21.15.71:0/291841264 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51d41004e0 0x7f51d406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:41.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.272+0000 7f51d9c08640 1 -- 172.21.15.71:0/291841264 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51d41004e0 msgr2=0x7f51d406a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:41.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.272+0000 7f51d9c08640 1 --2- 172.21.15.71:0/291841264 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51d41004e0 0x7f51d406a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.272+0000 7f51d9c08640 1 -- 172.21.15.71:0/291841264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51d41016d0 msgr2=0x7f51d419c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:41.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.272+0000 7f51d9c08640 1 --2- 172.21.15.71:0/291841264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51d41016d0 0x7f51d419c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.272+0000 7f51d9c08640 1 -- 172.21.15.71:0/291841264 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d419f4f0 con 0x7f51d4102090 2024-03-04T08:58:41.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.272+0000 7f51d9c08640 1 --2- 172.21.15.71:0/291841264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d4102090 0x7f51d419ebe0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f51cc037040 tx=0x7f51cc032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:41.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.272+0000 7f51c67fc640 1 -- 172.21.15.71:0/291841264 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc041020 con 0x7f51d4102090 2024-03-04T08:58:41.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.272+0000 7f51c67fc640 1 -- 172.21.15.71:0/291841264 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51cc034d90 con 0x7f51d4102090 2024-03-04T08:58:41.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.272+0000 7f51db66b640 1 -- 172.21.15.71:0/291841264 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d419f720 con 0x7f51d4102090 2024-03-04T08:58:41.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.272+0000 7f51c67fc640 1 -- 172.21.15.71:0/291841264 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc039040 con 0x7f51d4102090 2024-03-04T08:58:41.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.272+0000 7f51db66b640 1 -- 172.21.15.71:0/291841264 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d419fc60 con 0x7f51d4102090 2024-03-04T08:58:41.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.273+0000 7f51db66b640 1 -- 172.21.15.71:0/291841264 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d4107800 con 0x7f51d4102090 2024-03-04T08:58:41.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.274+0000 7f51c67fc640 1 -- 172.21.15.71:0/291841264 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f51cc044020 con 0x7f51d4102090 2024-03-04T08:58:41.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.275+0000 7f51c67fc640 1 --2- 172.21.15.71:0/291841264 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f51c00a1e00 0x7f51c00a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:41.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.275+0000 7f51d9407640 1 --2- 172.21.15.71:0/291841264 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f51c00a1e00 0x7f51c00a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:41.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.275+0000 7f51c67fc640 1 -- 172.21.15.71:0/291841264 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f51cc0f0030 con 0x7f51d4102090 2024-03-04T08:58:41.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.275+0000 7f51d9407640 1 --2- 172.21.15.71:0/291841264 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f51c00a1e00 0x7f51c00a42c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f51c8005fd0 tx=0x7f51c8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:41.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.279+0000 7f51c67fc640 1 -- 172.21.15.71:0/291841264 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f51cc04b020 con 0x7f51d4102090 2024-03-04T08:58:41.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:40 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2524167448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T08:58:41.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:40 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1238102623' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T08:58:41.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:40 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2949077483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T08:58:41.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:40 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/218604386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:58:41.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:40 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2524167448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T08:58:41.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:40 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1238102623' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T08:58:41.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:40 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2949077483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T08:58:41.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:40 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/218604386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:58:41.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.454+0000 7f51db66b640 1 -- 172.21.15.71:0/291841264 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f51d40621d0 con 0x7f51d4102090 2024-03-04T08:58:41.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.455+0000 7f51c67fc640 1 -- 172.21.15.71:0/291841264 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f51cc0bd0e0 con 0x7f51d4102090 2024-03-04T08:58:41.451 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-03-04T08:58:41.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.457+0000 7f51db66b640 1 -- 172.21.15.71:0/291841264 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f51c00a1e00 msgr2=0x7f51c00a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:41.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.457+0000 7f51db66b640 1 --2- 172.21.15.71:0/291841264 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f51c00a1e00 0x7f51c00a42c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f51c8005fd0 tx=0x7f51c8005850 comp rx=0 tx=0).stop 2024-03-04T08:58:41.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.457+0000 7f51db66b640 1 -- 172.21.15.71:0/291841264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d4102090 msgr2=0x7f51d419ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:41.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.457+0000 7f51db66b640 1 --2- 172.21.15.71:0/291841264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d4102090 0x7f51d419ebe0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f51cc037040 tx=0x7f51cc032f20 comp rx=0 tx=0).stop 2024-03-04T08:58:41.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.457+0000 7f51db66b640 1 -- 172.21.15.71:0/291841264 shutdown_connections 2024-03-04T08:58:41.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.457+0000 7f51db66b640 1 --2- 172.21.15.71:0/291841264 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f51c00a1e00 0x7f51c00a42c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.457+0000 7f51db66b640 1 --2- 172.21.15.71:0/291841264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51d4102090 0x7f51d419ebe0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.457+0000 7f51db66b640 1 --2- 172.21.15.71:0/291841264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51d41016d0 0x7f51d419c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.457+0000 7f51db66b640 1 --2- 172.21.15.71:0/291841264 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f51d41004e0 0x7f51d406a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.457+0000 7f51db66b640 1 -- 172.21.15.71:0/291841264 >> 172.21.15.71:0/291841264 conn(0x7f51d40fba70 msgr2=0x7f51d41049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:41.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.457+0000 7f51db66b640 1 -- 172.21.15.71:0/291841264 shutdown_connections 2024-03-04T08:58:41.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.457+0000 7f51db66b640 1 -- 172.21.15.71:0/291841264 wait complete. 2024-03-04T08:58:41.464 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T08:58:41.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.572+0000 7f9070c46640 1 -- 172.21.15.71:0/170452371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f906c1016c0 msgr2=0x7f906c101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:41.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.572+0000 7f9070c46640 1 --2- 172.21.15.71:0/170452371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f906c1016c0 0x7f906c101b40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9054009900 tx=0x7f905402bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:41.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.572+0000 7f9070c46640 1 -- 172.21.15.71:0/170452371 shutdown_connections 2024-03-04T08:58:41.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.572+0000 7f9070c46640 1 --2- 172.21.15.71:0/170452371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f906c102080 0x7f906c108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.572+0000 7f9070c46640 1 --2- 172.21.15.71:0/170452371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f906c1016c0 0x7f906c101b40 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.572+0000 7f9070c46640 1 --2- 172.21.15.71:0/170452371 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f906c1004d0 0x7f906c1008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.572+0000 7f9070c46640 1 -- 172.21.15.71:0/170452371 >> 172.21.15.71:0/170452371 conn(0x7f906c0fba60 msgr2=0x7f906c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:41.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.572+0000 7f9070c46640 1 -- 172.21.15.71:0/170452371 shutdown_connections 2024-03-04T08:58:41.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.572+0000 7f9070c46640 1 -- 172.21.15.71:0/170452371 wait complete. 2024-03-04T08:58:41.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.573+0000 7f9070c46640 1 Processor -- start 2024-03-04T08:58:41.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.573+0000 7f9070c46640 1 -- start start 2024-03-04T08:58:41.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.573+0000 7f9070c46640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f906c1004d0 0x7f906c1978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:41.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.573+0000 7f9070c46640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f906c1016c0 0x7f906c197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:41.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.573+0000 7f9070c46640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f906c102080 0x7f906c19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:41.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.573+0000 7f9070c46640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c10c160 con 0x7f906c102080 2024-03-04T08:58:41.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.573+0000 7f9070c46640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c10bfe0 con 0x7f906c1016c0 2024-03-04T08:58:41.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.573+0000 7f9070c46640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c10c2e0 con 0x7f906c1004d0 2024-03-04T08:58:41.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.573+0000 7f906ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f906c102080 0x7f906c19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:41.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.573+0000 7f9063fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f906c1016c0 0x7f906c197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:41.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.573+0000 7f906ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f906c102080 0x7f906c19eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60822/0 (socket says 172.21.15.71:60822) 2024-03-04T08:58:41.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.573+0000 7f906ad9d640 1 -- 172.21.15.71:0/423380043 learned_addr learned my addr 172.21.15.71:0/423380043 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:41.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.574+0000 7f906a59c640 1 --2- 172.21.15.71:0/423380043 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f906c1004d0 0x7f906c1978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:41.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.574+0000 7f906ad9d640 1 -- 172.21.15.71:0/423380043 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f906c1004d0 msgr2=0x7f906c1978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:41.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.574+0000 7f906ad9d640 1 --2- 172.21.15.71:0/423380043 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f906c1004d0 0x7f906c1978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.574+0000 7f906ad9d640 1 -- 172.21.15.71:0/423380043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f906c1016c0 msgr2=0x7f906c197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:41.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.574+0000 7f906ad9d640 1 --2- 172.21.15.71:0/423380043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f906c1016c0 0x7f906c197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.574+0000 7f906ad9d640 1 -- 172.21.15.71:0/423380043 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906c19f510 con 0x7f906c102080 2024-03-04T08:58:41.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.574+0000 7f9063fff640 1 --2- 172.21.15.71:0/423380043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f906c1016c0 0x7f906c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:58:41.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.574+0000 7f906ad9d640 1 --2- 172.21.15.71:0/423380043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f906c102080 0x7f906c19eea0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f906400eb10 tx=0x7f906400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:41.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.574+0000 7f90637fe640 1 -- 172.21.15.71:0/423380043 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9064004b20 con 0x7f906c102080 2024-03-04T08:58:41.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.574+0000 7f90637fe640 1 -- 172.21.15.71:0/423380043 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9064004cc0 con 0x7f906c102080 2024-03-04T08:58:41.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.574+0000 7f9070c46640 1 -- 172.21.15.71:0/423380043 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f906c19f800 con 0x7f906c102080 2024-03-04T08:58:41.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.574+0000 7f90637fe640 1 -- 172.21.15.71:0/423380043 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9064010760 con 0x7f906c102080 2024-03-04T08:58:41.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.574+0000 7f9070c46640 1 -- 172.21.15.71:0/423380043 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906c19fcd0 con 0x7f906c102080 2024-03-04T08:58:41.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.575+0000 7f9070c46640 1 -- 172.21.15.71:0/423380043 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f906c19ff90 con 0x7f906c102080 2024-03-04T08:58:41.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.576+0000 7f90637fe640 1 -- 172.21.15.71:0/423380043 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f9064005020 con 0x7f906c102080 2024-03-04T08:58:41.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.577+0000 7f90637fe640 1 --2- 172.21.15.71:0/423380043 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f90580a1d80 0x7f90580a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:41.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.577+0000 7f906a59c640 1 --2- 172.21.15.71:0/423380043 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f90580a1d80 0x7f90580a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:41.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.577+0000 7f90637fe640 1 -- 172.21.15.71:0/423380043 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f90640ce6e0 con 0x7f906c102080 2024-03-04T08:58:41.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.577+0000 7f906a59c640 1 --2- 172.21.15.71:0/423380043 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f90580a1d80 0x7f90580a4240 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f906c101520 tx=0x7f905c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:41.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.581+0000 7f90637fe640 1 -- 172.21.15.71:0/423380043 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f906409b790 con 0x7f906c102080 2024-03-04T08:58:41.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.755+0000 7f9070c46640 1 -- 172.21.15.71:0/423380043 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f906c1a0160 con 0x7f906c102080 2024-03-04T08:58:41.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.756+0000 7f90637fe640 1 -- 172.21.15.71:0/423380043 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f906409b090 con 0x7f906c102080 2024-03-04T08:58:41.752 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-03-04T08:58:41.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.758+0000 7f9070c46640 1 -- 172.21.15.71:0/423380043 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f90580a1d80 msgr2=0x7f90580a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:41.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.758+0000 7f9070c46640 1 --2- 172.21.15.71:0/423380043 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f90580a1d80 0x7f90580a4240 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f906c101520 tx=0x7f905c005f90 comp rx=0 tx=0).stop 2024-03-04T08:58:41.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.758+0000 7f9070c46640 1 -- 172.21.15.71:0/423380043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f906c102080 msgr2=0x7f906c19eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:41.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.758+0000 7f9070c46640 1 --2- 172.21.15.71:0/423380043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f906c102080 0x7f906c19eea0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f906400eb10 tx=0x7f906400c2d0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.758+0000 7f9070c46640 1 -- 172.21.15.71:0/423380043 shutdown_connections 2024-03-04T08:58:41.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.758+0000 7f9070c46640 1 --2- 172.21.15.71:0/423380043 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f90580a1d80 0x7f90580a4240 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.758+0000 7f9070c46640 1 --2- 172.21.15.71:0/423380043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f906c102080 0x7f906c19eea0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.758+0000 7f9070c46640 1 --2- 172.21.15.71:0/423380043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f906c1016c0 0x7f906c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.758+0000 7f9070c46640 1 --2- 172.21.15.71:0/423380043 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f906c1004d0 0x7f906c1978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.758+0000 7f9070c46640 1 -- 172.21.15.71:0/423380043 >> 172.21.15.71:0/423380043 conn(0x7f906c0fba60 msgr2=0x7f906c068730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:41.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.758+0000 7f9070c46640 1 -- 172.21.15.71:0/423380043 shutdown_connections 2024-03-04T08:58:41.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.758+0000 7f9070c46640 1 -- 172.21.15.71:0/423380043 wait complete. 2024-03-04T08:58:41.765 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T08:58:41.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.872+0000 7fda1de9a640 1 -- 172.21.15.71:0/2368941466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda18102070 msgr2=0x7fda18104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:41.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.872+0000 7fda1de9a640 1 --2- 172.21.15.71:0/2368941466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda18102070 0x7fda18104460 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fda08009860 tx=0x7fda0802bb70 comp rx=0 tx=0).stop 2024-03-04T08:58:41.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.872+0000 7fda1de9a640 1 -- 172.21.15.71:0/2368941466 shutdown_connections 2024-03-04T08:58:41.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.872+0000 7fda1de9a640 1 --2- 172.21.15.71:0/2368941466 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fda181072d0 0x7fda18109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.872+0000 7fda1de9a640 1 --2- 172.21.15.71:0/2368941466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda181049a0 0x7fda18106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.872+0000 7fda1de9a640 1 --2- 172.21.15.71:0/2368941466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda18102070 0x7fda18104460 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.872+0000 7fda1de9a640 1 -- 172.21.15.71:0/2368941466 >> 172.21.15.71:0/2368941466 conn(0x7fda180fba80 msgr2=0x7fda180fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:41.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.872+0000 7fda1de9a640 1 -- 172.21.15.71:0/2368941466 shutdown_connections 2024-03-04T08:58:41.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.872+0000 7fda1de9a640 1 -- 172.21.15.71:0/2368941466 wait complete. 2024-03-04T08:58:41.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.873+0000 7fda1de9a640 1 Processor -- start 2024-03-04T08:58:41.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.873+0000 7fda1de9a640 1 -- start start 2024-03-04T08:58:41.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.873+0000 7fda1de9a640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fda18102070 0x7fda18197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:41.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.873+0000 7fda1de9a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda181049a0 0x7fda18197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:41.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.873+0000 7fda1de9a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda181072d0 0x7fda1819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:41.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.873+0000 7fda1de9a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1810d260 con 0x7fda181049a0 2024-03-04T08:58:41.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.873+0000 7fda1de9a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1810d0e0 con 0x7fda181072d0 2024-03-04T08:58:41.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.873+0000 7fda1de9a640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1810d3e0 con 0x7fda18102070 2024-03-04T08:58:41.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.873+0000 7fda16ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda181049a0 0x7fda18197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:41.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.873+0000 7fda16ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda181049a0 0x7fda18197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60842/0 (socket says 172.21.15.71:60842) 2024-03-04T08:58:41.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.873+0000 7fda16ffd640 1 -- 172.21.15.71:0/2934420036 learned_addr learned my addr 172.21.15.71:0/2934420036 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:41.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.873+0000 7fda17fff640 1 --2- 172.21.15.71:0/2934420036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda181072d0 0x7fda1819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:41.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.874+0000 7fda177fe640 1 --2- 172.21.15.71:0/2934420036 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fda18102070 0x7fda18197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:41.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.874+0000 7fda16ffd640 1 -- 172.21.15.71:0/2934420036 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fda18102070 msgr2=0x7fda18197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:41.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.874+0000 7fda16ffd640 1 --2- 172.21.15.71:0/2934420036 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fda18102070 0x7fda18197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.874+0000 7fda16ffd640 1 -- 172.21.15.71:0/2934420036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda181072d0 msgr2=0x7fda1819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:41.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.874+0000 7fda16ffd640 1 --2- 172.21.15.71:0/2934420036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda181072d0 0x7fda1819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:41.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.874+0000 7fda16ffd640 1 -- 172.21.15.71:0/2934420036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1819f550 con 0x7fda181049a0 2024-03-04T08:58:41.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.874+0000 7fda177fe640 1 --2- 172.21.15.71:0/2934420036 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fda18102070 0x7fda18197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:58:41.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.874+0000 7fda16ffd640 1 --2- 172.21.15.71:0/2934420036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda181049a0 0x7fda18197e70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fda0c00e9a0 tx=0x7fda0c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:41.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.874+0000 7fda14ff9640 1 -- 172.21.15.71:0/2934420036 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c00cd80 con 0x7fda181049a0 2024-03-04T08:58:41.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.874+0000 7fda1de9a640 1 -- 172.21.15.71:0/2934420036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda1819f840 con 0x7fda181049a0 2024-03-04T08:58:41.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.874+0000 7fda14ff9640 1 -- 172.21.15.71:0/2934420036 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda0c0035f0 con 0x7fda181049a0 2024-03-04T08:58:41.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.874+0000 7fda1de9a640 1 -- 172.21.15.71:0/2934420036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda1819fde0 con 0x7fda181049a0 2024-03-04T08:58:41.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.874+0000 7fda14ff9640 1 -- 172.21.15.71:0/2934420036 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c018630 con 0x7fda181049a0 2024-03-04T08:58:41.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.875+0000 7fda1de9a640 1 -- 172.21.15.71:0/2934420036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda180fd570 con 0x7fda181049a0 2024-03-04T08:58:41.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.876+0000 7fda14ff9640 1 -- 172.21.15.71:0/2934420036 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fda0c0037a0 con 0x7fda181049a0 2024-03-04T08:58:41.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.876+0000 7fda14ff9640 1 --2- 172.21.15.71:0/2934420036 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fda000a1d80 0x7fda000a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:41.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.877+0000 7fda177fe640 1 --2- 172.21.15.71:0/2934420036 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fda000a1d80 0x7fda000a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:41.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.877+0000 7fda14ff9640 1 -- 172.21.15.71:0/2934420036 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7fda0c0ce3a0 con 0x7fda181049a0 2024-03-04T08:58:41.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.877+0000 7fda177fe640 1 --2- 172.21.15.71:0/2934420036 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fda000a1d80 0x7fda000a4240 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fda08005940 tx=0x7fda080058b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:41.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:41.881+0000 7fda14ff9640 1 -- 172.21.15.71:0/2934420036 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fda0c09b450 con 0x7fda181049a0 2024-03-04T08:58:42.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.055+0000 7fda1de9a640 1 -- 172.21.15.71:0/2934420036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fda180621d0 con 0x7fda181049a0 2024-03-04T08:58:42.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.055+0000 7fda14ff9640 1 -- 172.21.15.71:0/2934420036 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7fda0c09ad50 con 0x7fda181049a0 2024-03-04T08:58:42.051 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2024-03-04T08:58:42.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.057+0000 7fda1de9a640 1 -- 172.21.15.71:0/2934420036 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fda000a1d80 msgr2=0x7fda000a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:42.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.057+0000 7fda1de9a640 1 --2- 172.21.15.71:0/2934420036 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fda000a1d80 0x7fda000a4240 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fda08005940 tx=0x7fda080058b0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.057+0000 7fda1de9a640 1 -- 172.21.15.71:0/2934420036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda181049a0 msgr2=0x7fda18197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:42.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.057+0000 7fda1de9a640 1 --2- 172.21.15.71:0/2934420036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda181049a0 0x7fda18197e70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fda0c00e9a0 tx=0x7fda0c00ee60 comp rx=0 tx=0).stop 2024-03-04T08:58:42.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.057+0000 7fda1de9a640 1 -- 172.21.15.71:0/2934420036 shutdown_connections 2024-03-04T08:58:42.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.057+0000 7fda1de9a640 1 --2- 172.21.15.71:0/2934420036 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fda000a1d80 0x7fda000a4240 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.057+0000 7fda1de9a640 1 --2- 172.21.15.71:0/2934420036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda181072d0 0x7fda1819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.057+0000 7fda1de9a640 1 --2- 172.21.15.71:0/2934420036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda181049a0 0x7fda18197e70 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.057+0000 7fda1de9a640 1 --2- 172.21.15.71:0/2934420036 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fda18102070 0x7fda18197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.057+0000 7fda1de9a640 1 -- 172.21.15.71:0/2934420036 >> 172.21.15.71:0/2934420036 conn(0x7fda180fba80 msgr2=0x7fda181029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:42.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.057+0000 7fda1de9a640 1 -- 172.21.15.71:0/2934420036 shutdown_connections 2024-03-04T08:58:42.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.057+0000 7fda1de9a640 1 -- 172.21.15.71:0/2934420036 wait complete. 2024-03-04T08:58:42.063 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:41 smithi071 ceph-mon[32789]: pgmap v394: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:42.063 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:41 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2435332518' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:58:42.063 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:41 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/291841264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:58:42.063 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:41 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/423380043' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T08:58:42.064 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T08:58:42.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.172+0000 7f08762fb640 1 -- 172.21.15.71:0/1660032137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0870103190 msgr2=0x7f0870109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:42.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.172+0000 7f08762fb640 1 --2- 172.21.15.71:0/1660032137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0870103190 0x7f0870109a20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0868009ca0 tx=0x7f086802c0f0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.172+0000 7f08762fb640 1 -- 172.21.15.71:0/1660032137 shutdown_connections 2024-03-04T08:58:42.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.172+0000 7f08762fb640 1 --2- 172.21.15.71:0/1660032137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0870103190 0x7f0870109a20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.172+0000 7f08762fb640 1 --2- 172.21.15.71:0/1660032137 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f08701027d0 0x7f0870102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.172+0000 7f08762fb640 1 --2- 172.21.15.71:0/1660032137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08701015e0 0x7f0870101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.172+0000 7f08762fb640 1 -- 172.21.15.71:0/1660032137 >> 172.21.15.71:0/1660032137 conn(0x7f08700fcb70 msgr2=0x7f08700fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:42.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.172+0000 7f08762fb640 1 -- 172.21.15.71:0/1660032137 shutdown_connections 2024-03-04T08:58:42.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.172+0000 7f08762fb640 1 -- 172.21.15.71:0/1660032137 wait complete. 2024-03-04T08:58:42.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.173+0000 7f08762fb640 1 Processor -- start 2024-03-04T08:58:42.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.173+0000 7f08762fb640 1 -- start start 2024-03-04T08:58:42.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.173+0000 7f08762fb640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f08701015e0 0x7f0870197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:42.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.173+0000 7f08762fb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08701027d0 0x7f0870197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:42.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.173+0000 7f08762fb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0870103190 0x7f087019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:42.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.173+0000 7f08762fb640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087010d240 con 0x7f08701027d0 2024-03-04T08:58:42.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.173+0000 7f08762fb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087010d0c0 con 0x7f0870103190 2024-03-04T08:58:42.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.173+0000 7f08762fb640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087010d3c0 con 0x7f08701015e0 2024-03-04T08:58:42.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.173+0000 7f086f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08701027d0 0x7f0870197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:42.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.173+0000 7f086f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08701027d0 0x7f0870197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60852/0 (socket says 172.21.15.71:60852) 2024-03-04T08:58:42.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.173+0000 7f086f7fe640 1 -- 172.21.15.71:0/1244198665 learned_addr learned my addr 172.21.15.71:0/1244198665 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:42.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.173+0000 7f0874898640 1 --2- 172.21.15.71:0/1244198665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0870103190 0x7f087019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:42.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.173+0000 7f086ffff640 1 --2- 172.21.15.71:0/1244198665 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f08701015e0 0x7f0870197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:42.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.174+0000 7f086f7fe640 1 -- 172.21.15.71:0/1244198665 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f08701015e0 msgr2=0x7f0870197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:42.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.174+0000 7f086f7fe640 1 --2- 172.21.15.71:0/1244198665 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f08701015e0 0x7f0870197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.174+0000 7f086f7fe640 1 -- 172.21.15.71:0/1244198665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0870103190 msgr2=0x7f087019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:42.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.174+0000 7f086f7fe640 1 --2- 172.21.15.71:0/1244198665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0870103190 0x7f087019ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.174+0000 7f086f7fe640 1 -- 172.21.15.71:0/1244198665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087019f4a0 con 0x7f08701027d0 2024-03-04T08:58:42.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.174+0000 7f086f7fe640 1 --2- 172.21.15.71:0/1244198665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08701027d0 0x7f0870197dc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f086400d950 tx=0x7f086400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:42.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.174+0000 7f086d7fa640 1 -- 172.21.15.71:0/1244198665 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0864017030 con 0x7f08701027d0 2024-03-04T08:58:42.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.174+0000 7f08762fb640 1 -- 172.21.15.71:0/1244198665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f087019f790 con 0x7f08701027d0 2024-03-04T08:58:42.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.174+0000 7f086d7fa640 1 -- 172.21.15.71:0/1244198665 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0864004020 con 0x7f08701027d0 2024-03-04T08:58:42.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.174+0000 7f08762fb640 1 -- 172.21.15.71:0/1244198665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087019fce0 con 0x7f08701027d0 2024-03-04T08:58:42.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.174+0000 7f086d7fa640 1 -- 172.21.15.71:0/1244198665 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08640047a0 con 0x7f08701027d0 2024-03-04T08:58:42.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.175+0000 7f08762fb640 1 -- 172.21.15.71:0/1244198665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0870108980 con 0x7f08701027d0 2024-03-04T08:58:42.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.176+0000 7f086d7fa640 1 -- 172.21.15.71:0/1244198665 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f08640041d0 con 0x7f08701027d0 2024-03-04T08:58:42.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.177+0000 7f086d7fa640 1 --2- 172.21.15.71:0/1244198665 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f08580a1d80 0x7f08580a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:42.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.177+0000 7f086ffff640 1 --2- 172.21.15.71:0/1244198665 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f08580a1d80 0x7f08580a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:42.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.177+0000 7f086d7fa640 1 -- 172.21.15.71:0/1244198665 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f08640cda60 con 0x7f08701027d0 2024-03-04T08:58:42.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.177+0000 7f086ffff640 1 --2- 172.21.15.71:0/1244198665 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f08580a1d80 0x7f08580a4240 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f08600095f0 tx=0x7f0860005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:42.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.181+0000 7f086d7fa640 1 -- 172.21.15.71:0/1244198665 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f086409ab10 con 0x7f08701027d0 2024-03-04T08:58:42.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:41 smithi150 ceph-mon[36807]: pgmap v394: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:42.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:41 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2435332518' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:58:42.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:41 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/291841264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:58:42.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:41 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/423380043' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T08:58:42.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:41 smithi158 ceph-mon[37241]: pgmap v394: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:42.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:41 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2435332518' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:58:42.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:41 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/291841264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T08:58:42.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:41 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/423380043' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T08:58:42.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.355+0000 7f08762fb640 1 -- 172.21.15.71:0/1244198665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f08700621d0 con 0x7f08701027d0 2024-03-04T08:58:42.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.355+0000 7f086d7fa640 1 -- 172.21.15.71:0/1244198665 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7f086409a410 con 0x7f08701027d0 2024-03-04T08:58:42.351 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-03-04T08:58:42.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.357+0000 7f08762fb640 1 -- 172.21.15.71:0/1244198665 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f08580a1d80 msgr2=0x7f08580a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:42.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.357+0000 7f08762fb640 1 --2- 172.21.15.71:0/1244198665 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f08580a1d80 0x7f08580a4240 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f08600095f0 tx=0x7f0860005f90 comp rx=0 tx=0).stop 2024-03-04T08:58:42.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.357+0000 7f08762fb640 1 -- 172.21.15.71:0/1244198665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08701027d0 msgr2=0x7f0870197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:42.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.357+0000 7f08762fb640 1 --2- 172.21.15.71:0/1244198665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08701027d0 0x7f0870197dc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f086400d950 tx=0x7f086400de10 comp rx=0 tx=0).stop 2024-03-04T08:58:42.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.357+0000 7f08762fb640 1 -- 172.21.15.71:0/1244198665 shutdown_connections 2024-03-04T08:58:42.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.357+0000 7f08762fb640 1 --2- 172.21.15.71:0/1244198665 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f08580a1d80 0x7f08580a4240 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.357+0000 7f08762fb640 1 --2- 172.21.15.71:0/1244198665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0870103190 0x7f087019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.357+0000 7f08762fb640 1 --2- 172.21.15.71:0/1244198665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08701027d0 0x7f0870197dc0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.357+0000 7f08762fb640 1 --2- 172.21.15.71:0/1244198665 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f08701015e0 0x7f0870197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.357+0000 7f08762fb640 1 -- 172.21.15.71:0/1244198665 >> 172.21.15.71:0/1244198665 conn(0x7f08700fcb70 msgr2=0x7f0870105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:42.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.357+0000 7f08762fb640 1 -- 172.21.15.71:0/1244198665 shutdown_connections 2024-03-04T08:58:42.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.357+0000 7f08762fb640 1 -- 172.21.15.71:0/1244198665 wait complete. 2024-03-04T08:58:42.365 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:58:42.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.469+0000 7fd6608fb640 1 -- 172.21.15.71:0/888943838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd65c1007c0 msgr2=0x7fd65c100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:42.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.469+0000 7fd6608fb640 1 --2- 172.21.15.71:0/888943838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd65c1007c0 0x7fd65c100c20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd650009900 tx=0x7fd65002bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:42.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.469+0000 7fd6608fb640 1 -- 172.21.15.71:0/888943838 shutdown_connections 2024-03-04T08:58:42.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.469+0000 7fd6608fb640 1 --2- 172.21.15.71:0/888943838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd65c101160 0x7fd65c1015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.469+0000 7fd6608fb640 1 --2- 172.21.15.71:0/888943838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd65c1007c0 0x7fd65c100c20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.469+0000 7fd6608fb640 1 --2- 172.21.15.71:0/888943838 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd65c0ffe60 0x7fd65c100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.469+0000 7fd6608fb640 1 -- 172.21.15.71:0/888943838 >> 172.21.15.71:0/888943838 conn(0x7fd65c0fba60 msgr2=0x7fd65c0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:42.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.469+0000 7fd6608fb640 1 -- 172.21.15.71:0/888943838 shutdown_connections 2024-03-04T08:58:42.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.469+0000 7fd6608fb640 1 -- 172.21.15.71:0/888943838 wait complete. 2024-03-04T08:58:42.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.470+0000 7fd6608fb640 1 Processor -- start 2024-03-04T08:58:42.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.470+0000 7fd6608fb640 1 -- start start 2024-03-04T08:58:42.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.470+0000 7fd6608fb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd65c0ffe60 0x7fd65c199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:42.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.470+0000 7fd6608fb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd65c1007c0 0x7fd65c19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:42.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.470+0000 7fd6608fb640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd65c101160 0x7fd65c1a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:42.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.470+0000 7fd6608fb640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c10f430 con 0x7fd65c1007c0 2024-03-04T08:58:42.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.470+0000 7fd6608fb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c10f2b0 con 0x7fd65c0ffe60 2024-03-04T08:58:42.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.470+0000 7fd6608fb640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65c10f5b0 con 0x7fd65c101160 2024-03-04T08:58:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.470+0000 7fd659d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd65c1007c0 0x7fd65c19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.470+0000 7fd659d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd65c1007c0 0x7fd65c19a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60876/0 (socket says 172.21.15.71:60876) 2024-03-04T08:58:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.470+0000 7fd659d9b640 1 -- 172.21.15.71:0/697570383 learned_addr learned my addr 172.21.15.71:0/697570383 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.470+0000 7fd65a59c640 1 --2- 172.21.15.71:0/697570383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd65c0ffe60 0x7fd65c199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.471+0000 7fd65ad9d640 1 --2- 172.21.15.71:0/697570383 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd65c101160 0x7fd65c1a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.471+0000 7fd659d9b640 1 -- 172.21.15.71:0/697570383 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd65c101160 msgr2=0x7fd65c1a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.471+0000 7fd659d9b640 1 --2- 172.21.15.71:0/697570383 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd65c101160 0x7fd65c1a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.471+0000 7fd659d9b640 1 -- 172.21.15.71:0/697570383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd65c0ffe60 msgr2=0x7fd65c199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.471+0000 7fd659d9b640 1 --2- 172.21.15.71:0/697570383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd65c0ffe60 0x7fd65c199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.471+0000 7fd659d9b640 1 -- 172.21.15.71:0/697570383 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c1a1700 con 0x7fd65c1007c0 2024-03-04T08:58:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.471+0000 7fd659d9b640 1 --2- 172.21.15.71:0/697570383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd65c1007c0 0x7fd65c19a020 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd650009230 tx=0x7fd650005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.471+0000 7fd64b7fe640 1 -- 172.21.15.71:0/697570383 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd650041070 con 0x7fd65c1007c0 2024-03-04T08:58:42.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.471+0000 7fd64b7fe640 1 -- 172.21.15.71:0/697570383 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6500044d0 con 0x7fd65c1007c0 2024-03-04T08:58:42.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.471+0000 7fd6608fb640 1 -- 172.21.15.71:0/697570383 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd65c1a1990 con 0x7fd65c1007c0 2024-03-04T08:58:42.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.471+0000 7fd64b7fe640 1 -- 172.21.15.71:0/697570383 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd650034440 con 0x7fd65c1007c0 2024-03-04T08:58:42.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.471+0000 7fd6608fb640 1 -- 172.21.15.71:0/697570383 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65c1a1e80 con 0x7fd65c1007c0 2024-03-04T08:58:42.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.472+0000 7fd6608fb640 1 -- 172.21.15.71:0/697570383 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd65c10ad90 con 0x7fd65c1007c0 2024-03-04T08:58:42.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.473+0000 7fd64b7fe640 1 -- 172.21.15.71:0/697570383 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fd650044070 con 0x7fd65c1007c0 2024-03-04T08:58:42.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.474+0000 7fd64b7fe640 1 --2- 172.21.15.71:0/697570383 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd6440a1d80 0x7fd6440a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:42.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.474+0000 7fd65a59c640 1 --2- 172.21.15.71:0/697570383 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd6440a1d80 0x7fd6440a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:42.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.474+0000 7fd64b7fe640 1 -- 172.21.15.71:0/697570383 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7fd6500efd70 con 0x7fd65c1007c0 2024-03-04T08:58:42.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.474+0000 7fd65a59c640 1 --2- 172.21.15.71:0/697570383 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd6440a1d80 0x7fd6440a4240 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd64c005c40 tx=0x7fd64c00a590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:42.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.478+0000 7fd64b7fe640 1 -- 172.21.15.71:0/697570383 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd6500bce20 con 0x7fd65c1007c0 2024-03-04T08:58:42.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.690+0000 7fd6608fb640 1 -- 172.21.15.71:0/697570383 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd65c02cff0 con 0x7fd65c1007c0 2024-03-04T08:58:42.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.691+0000 7fd64b7fe640 1 -- 172.21.15.71:0/697570383 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11339 (secure 0 0 0) 0x7fd6500bc720 con 0x7fd65c1007c0 2024-03-04T08:58:42.687 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:58:42.687 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3975379807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3975379807},{"type":"v1","addr":"172.21.15.71:6835","nonce":3975379807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1962929347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1962929347},{"type":"v1","addr":"172.21.15.71:6837","nonce":1962929347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/1459252176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1459252176},{"type":"v1","addr":"172.21.15.71:6839","nonce":1459252176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6841/1083546600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":1083546600},{"type":"v1","addr":"172.21.15.71:6841","nonce":1083546600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/178179528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":178179528},{"type":"v1","addr":"172.21.15.158:6833","nonce":178179528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2468877675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2468877675},{"type":"v1","addr":"172.21.15.150:6833","nonce":2468877675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1652094309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1652094309},{"type":"v1","addr":"172.21.15.158:6835","nonce":1652094309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1802416462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1802416462},{"type":"v1","addr":"172.21.15.150:6835","nonce":1802416462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/617058380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":617058380},{"type":"v1","addr":"172.21.15.158:6837","nonce":617058380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/913465600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":913465600},{"type":"v1","addr":"172.21.15.150:6837","nonce":913465600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T08:56:30.343357+0000","modified":"2024-03-04T08:56:54.901214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.158:6839/2921704804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6838","nonce":2921704804},{"type":"v1","addr":"172.21.15.158:6839","nonce":2921704804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/2639278083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2639278083},{"type":"v1","addr":"172.21.15.150:6839","nonce":2639278083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491]},"id":1}]} 2024-03-04T08:58:42.687 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 20 2024-03-04T08:58:42.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.693+0000 7fd6608fb640 1 -- 172.21.15.71:0/697570383 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd6440a1d80 msgr2=0x7fd6440a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:42.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.693+0000 7fd6608fb640 1 --2- 172.21.15.71:0/697570383 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd6440a1d80 0x7fd6440a4240 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd64c005c40 tx=0x7fd64c00a590 comp rx=0 tx=0).stop 2024-03-04T08:58:42.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.693+0000 7fd6608fb640 1 -- 172.21.15.71:0/697570383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd65c1007c0 msgr2=0x7fd65c19a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:42.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.693+0000 7fd6608fb640 1 --2- 172.21.15.71:0/697570383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd65c1007c0 0x7fd65c19a020 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd650009230 tx=0x7fd650005d50 comp rx=0 tx=0).stop 2024-03-04T08:58:42.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.693+0000 7fd6608fb640 1 -- 172.21.15.71:0/697570383 shutdown_connections 2024-03-04T08:58:42.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.693+0000 7fd6608fb640 1 --2- 172.21.15.71:0/697570383 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7fd6440a1d80 0x7fd6440a4240 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.693+0000 7fd6608fb640 1 --2- 172.21.15.71:0/697570383 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd65c101160 0x7fd65c1a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.693+0000 7fd6608fb640 1 --2- 172.21.15.71:0/697570383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd65c1007c0 0x7fd65c19a020 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.693+0000 7fd6608fb640 1 --2- 172.21.15.71:0/697570383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd65c0ffe60 0x7fd65c199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.693+0000 7fd6608fb640 1 -- 172.21.15.71:0/697570383 >> 172.21.15.71:0/697570383 conn(0x7fd65c0fba60 msgr2=0x7fd65c0fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:42.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.693+0000 7fd6608fb640 1 -- 172.21.15.71:0/697570383 shutdown_connections 2024-03-04T08:58:42.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.693+0000 7fd6608fb640 1 -- 172.21.15.71:0/697570383 wait complete. 2024-03-04T08:58:42.700 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T08:58:42.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.806+0000 7f544345c640 1 -- 172.21.15.71:0/1988575835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f543c102800 msgr2=0x7f543c102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:42.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.806+0000 7f544345c640 1 --2- 172.21.15.71:0/1988575835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f543c102800 0x7f543c102c80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f542c009900 tx=0x7f542c02bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:42.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.806+0000 7f544345c640 1 -- 172.21.15.71:0/1988575835 shutdown_connections 2024-03-04T08:58:42.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.806+0000 7f544345c640 1 --2- 172.21.15.71:0/1988575835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f543c1031c0 0x7f543c109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.806+0000 7f544345c640 1 --2- 172.21.15.71:0/1988575835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f543c102800 0x7f543c102c80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.806+0000 7f544345c640 1 --2- 172.21.15.71:0/1988575835 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f543c101610 0x7f543c101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.806+0000 7f544345c640 1 -- 172.21.15.71:0/1988575835 >> 172.21.15.71:0/1988575835 conn(0x7f543c0fcb80 msgr2=0x7f543c0fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:42.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.806+0000 7f544345c640 1 -- 172.21.15.71:0/1988575835 shutdown_connections 2024-03-04T08:58:42.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.806+0000 7f544345c640 1 -- 172.21.15.71:0/1988575835 wait complete. 2024-03-04T08:58:42.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.806+0000 7f544345c640 1 Processor -- start 2024-03-04T08:58:42.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.806+0000 7f544345c640 1 -- start start 2024-03-04T08:58:42.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.806+0000 7f544345c640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f543c101610 0x7f543c197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:42.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.807+0000 7f544345c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f543c102800 0x7f543c197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:42.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.807+0000 7f544345c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f543c1031c0 0x7f543c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:42.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.807+0000 7f544345c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c10d2a0 con 0x7f543c102800 2024-03-04T08:58:42.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.807+0000 7f544345c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c10d120 con 0x7f543c1031c0 2024-03-04T08:58:42.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.807+0000 7f544345c640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c10d420 con 0x7f543c101610 2024-03-04T08:58:42.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.807+0000 7f54409f7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f543c102800 0x7f543c197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:42.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.807+0000 7f54409f7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f543c102800 0x7f543c197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60898/0 (socket says 172.21.15.71:60898) 2024-03-04T08:58:42.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.807+0000 7f54409f7640 1 -- 172.21.15.71:0/12968847 learned_addr learned my addr 172.21.15.71:0/12968847 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:42.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.807+0000 7f54419f9640 1 --2- 172.21.15.71:0/12968847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f543c1031c0 0x7f543c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:42.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.807+0000 7f54411f8640 1 --2- 172.21.15.71:0/12968847 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f543c101610 0x7f543c197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:42.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.807+0000 7f54409f7640 1 -- 172.21.15.71:0/12968847 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f543c101610 msgr2=0x7f543c197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:42.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.807+0000 7f54409f7640 1 --2- 172.21.15.71:0/12968847 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f543c101610 0x7f543c197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.807+0000 7f54409f7640 1 -- 172.21.15.71:0/12968847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f543c1031c0 msgr2=0x7f543c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:42.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.807+0000 7f54409f7640 1 --2- 172.21.15.71:0/12968847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f543c1031c0 0x7f543c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.807+0000 7f54409f7640 1 -- 172.21.15.71:0/12968847 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543c19f570 con 0x7f543c102800 2024-03-04T08:58:42.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.807+0000 7f54409f7640 1 --2- 172.21.15.71:0/12968847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f543c102800 0x7f543c197e90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f542c009230 tx=0x7f542c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:42.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.807+0000 7f54327fc640 1 -- 172.21.15.71:0/12968847 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c041070 con 0x7f543c102800 2024-03-04T08:58:42.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.807+0000 7f54327fc640 1 -- 172.21.15.71:0/12968847 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f542c0044d0 con 0x7f543c102800 2024-03-04T08:58:42.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.807+0000 7f544345c640 1 -- 172.21.15.71:0/12968847 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f543c19f800 con 0x7f543c102800 2024-03-04T08:58:42.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.807+0000 7f54327fc640 1 -- 172.21.15.71:0/12968847 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c034440 con 0x7f543c102800 2024-03-04T08:58:42.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.807+0000 7f544345c640 1 -- 172.21.15.71:0/12968847 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f543c19fcf0 con 0x7f543c102800 2024-03-04T08:58:42.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.808+0000 7f544345c640 1 -- 172.21.15.71:0/12968847 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f543c108930 con 0x7f543c102800 2024-03-04T08:58:42.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.809+0000 7f54327fc640 1 -- 172.21.15.71:0/12968847 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f542c044070 con 0x7f543c102800 2024-03-04T08:58:42.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.810+0000 7f54327fc640 1 --2- 172.21.15.71:0/12968847 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f54280a1d80 0x7f54280a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:42.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.810+0000 7f54411f8640 1 --2- 172.21.15.71:0/12968847 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f54280a1d80 0x7f54280a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:42.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.810+0000 7f54327fc640 1 -- 172.21.15.71:0/12968847 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f542c0efd80 con 0x7f543c102800 2024-03-04T08:58:42.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.810+0000 7f54411f8640 1 --2- 172.21.15.71:0/12968847 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f54280a1d80 0x7f54280a4240 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f543c102660 tx=0x7f5438009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:42.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.814+0000 7f54327fc640 1 -- 172.21.15.71:0/12968847 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f542c0bce30 con 0x7f543c102800 2024-03-04T08:58:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.991+0000 7f544345c640 1 -- 172.21.15.71:0/12968847 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f543c0621d0 con 0x7f543c102800 2024-03-04T08:58:42.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.993+0000 7f54327fc640 1 -- 172.21.15.71:0/12968847 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24727 (secure 0 0 0) 0x7f542c0bc730 con 0x7f543c102800 2024-03-04T08:58:42.989 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T08:58:42.989 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":95,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","created":"2024-03-04T08:47:28.546102+0000","modified":"2024-03-04T08:58:23.853934+0000","last_up_change":"2024-03-04T08:55:30.744961+0000","last_in_change":"2024-03-04T08:55:11.570990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T08:51:14.933487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T08:56:28.253074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T08:56:29.543622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T08:56:32.884220+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7e43b47d-8888-4631-bc96-8a96fcf8cd46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6803","nonce":1173641530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6805","nonce":1173641530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6809","nonce":1173641530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6807","nonce":1173641530}]},"public_addr":"172.21.15.71:6803/1173641530","cluster_addr":"172.21.15.71:6805/1173641530","heartbeat_back_addr":"172.21.15.71:6809/1173641530","heartbeat_front_addr":"172.21.15.71:6807/1173641530","state":["exists","up"]},{"osd":1,"uuid":"1209f214-f2e2-4209-bf69-4528a43a2123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6801","nonce":3383948110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6803","nonce":3383948110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6807","nonce":3383948110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6805","nonce":3383948110}]},"public_addr":"172.21.15.150:6801/3383948110","cluster_addr":"172.21.15.150:6803/3383948110","heartbeat_back_addr":"172.21.15.150:6807/3383948110","heartbeat_front_addr":"172.21.15.150:6805/3383948110","state":["exists","up"]},{"osd":2,"uuid":"b88c3082-93a9-4c9b-9221-a7105a44c91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6801","nonce":586944144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6803","nonce":586944144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6807","nonce":586944144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6805","nonce":586944144}]},"public_addr":"172.21.15.158:6801/586944144","cluster_addr":"172.21.15.158:6803/586944144","heartbeat_back_addr":"172.21.15.158:6807/586944144","heartbeat_front_addr":"172.21.15.158:6805/586944144","state":["exists","up"]},{"osd":3,"uuid":"e4fc7723-fd73-40b2-b9d0-7f9bda282ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6811","nonce":1311903262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6813","nonce":1311903262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6817","nonce":1311903262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6815","nonce":1311903262}]},"public_addr":"172.21.15.71:6811/1311903262","cluster_addr":"172.21.15.71:6813/1311903262","heartbeat_back_addr":"172.21.15.71:6817/1311903262","heartbeat_front_addr":"172.21.15.71:6815/1311903262","state":["exists","up"]},{"osd":4,"uuid":"e7eb72ac-f066-4ccd-b2bf-22a6a26daf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6809","nonce":2300837573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6811","nonce":2300837573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6815","nonce":2300837573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6813","nonce":2300837573}]},"public_addr":"172.21.15.150:6809/2300837573","cluster_addr":"172.21.15.150:6811/2300837573","heartbeat_back_addr":"172.21.15.150:6815/2300837573","heartbeat_front_addr":"172.21.15.150:6813/2300837573","state":["exists","up"]},{"osd":5,"uuid":"9b295840-e104-4a66-9cc4-7020627c1765","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6809","nonce":1922694045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6811","nonce":1922694045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6815","nonce":1922694045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6813","nonce":1922694045}]},"public_addr":"172.21.15.158:6809/1922694045","cluster_addr":"172.21.15.158:6811/1922694045","heartbeat_back_addr":"172.21.15.158:6815/1922694045","heartbeat_front_addr":"172.21.15.158:6813/1922694045","state":["exists","up"]},{"osd":6,"uuid":"6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6819","nonce":3848842707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6821","nonce":3848842707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6825","nonce":3848842707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6823","nonce":3848842707}]},"public_addr":"172.21.15.71:6819/3848842707","cluster_addr":"172.21.15.71:6821/3848842707","heartbeat_back_addr":"172.21.15.71:6825/3848842707","heartbeat_front_addr":"172.21.15.71:6823/3848842707","state":["exists","up"]},{"osd":7,"uuid":"6160a40a-d30f-4569-949c-0afb1bb3d133","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6817","nonce":1328330305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6819","nonce":1328330305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6823","nonce":1328330305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6821","nonce":1328330305}]},"public_addr":"172.21.15.150:6817/1328330305","cluster_addr":"172.21.15.150:6819/1328330305","heartbeat_back_addr":"172.21.15.150:6823/1328330305","heartbeat_front_addr":"172.21.15.150:6821/1328330305","state":["exists","up"]},{"osd":8,"uuid":"95ed2341-84a4-465c-9948-ac667680ce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6817","nonce":2195304577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6819","nonce":2195304577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6823","nonce":2195304577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6821","nonce":2195304577}]},"public_addr":"172.21.15.158:6817/2195304577","cluster_addr":"172.21.15.158:6819/2195304577","heartbeat_back_addr":"172.21.15.158:6823/2195304577","heartbeat_front_addr":"172.21.15.158:6821/2195304577","state":["exists","up"]},{"osd":9,"uuid":"088f6419-63a0-4768-afae-dbb8a5f5a565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6827","nonce":1913779347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6829","nonce":1913779347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6833","nonce":1913779347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6831","nonce":1913779347}]},"public_addr":"172.21.15.71:6827/1913779347","cluster_addr":"172.21.15.71:6829/1913779347","heartbeat_back_addr":"172.21.15.71:6833/1913779347","heartbeat_front_addr":"172.21.15.71:6831/1913779347","state":["exists","up"]},{"osd":10,"uuid":"18c85e24-068d-4f1e-a3b8-f7915b363234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6825","nonce":3295420996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6827","nonce":3295420996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6831","nonce":3295420996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6829","nonce":3295420996}]},"public_addr":"172.21.15.150:6825/3295420996","cluster_addr":"172.21.15.150:6827/3295420996","heartbeat_back_addr":"172.21.15.150:6831/3295420996","heartbeat_front_addr":"172.21.15.150:6829/3295420996","state":["exists","up"]},{"osd":11,"uuid":"4f6b3ddf-c482-4740-b1fd-de836e047c2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6825","nonce":3332083865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6827","nonce":3332083865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6831","nonce":3332083865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6829","nonce":3332083865}]},"public_addr":"172.21.15.158:6825/3332083865","cluster_addr":"172.21.15.158:6827/3332083865","heartbeat_back_addr":"172.21.15.158:6831/3332083865","heartbeat_front_addr":"172.21.15.158:6829/3332083865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:05.997832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:29.314235+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:08.819465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:41.194275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:06.032479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:38.814498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:13.067960+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:38.568479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:03.416713+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:38.349479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:04.096945+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:29.425250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1287712358":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/3770452860":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/718743094":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/1904717532":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:6800/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6800/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/2862184313":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/3621754630":"2024-03-05T08:48:22.593931+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T08:58:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.995+0000 7f544345c640 1 -- 172.21.15.71:0/12968847 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f54280a1d80 msgr2=0x7f54280a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.995+0000 7f544345c640 1 --2- 172.21.15.71:0/12968847 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f54280a1d80 0x7f54280a4240 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f543c102660 tx=0x7f5438009870 comp rx=0 tx=0).stop 2024-03-04T08:58:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.995+0000 7f544345c640 1 -- 172.21.15.71:0/12968847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f543c102800 msgr2=0x7f543c197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.995+0000 7f544345c640 1 --2- 172.21.15.71:0/12968847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f543c102800 0x7f543c197e90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f542c009230 tx=0x7f542c005d50 comp rx=0 tx=0).stop 2024-03-04T08:58:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.995+0000 7f544345c640 1 -- 172.21.15.71:0/12968847 shutdown_connections 2024-03-04T08:58:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.995+0000 7f544345c640 1 --2- 172.21.15.71:0/12968847 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f54280a1d80 0x7f54280a4240 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.995+0000 7f544345c640 1 --2- 172.21.15.71:0/12968847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f543c1031c0 0x7f543c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.995+0000 7f544345c640 1 --2- 172.21.15.71:0/12968847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f543c102800 0x7f543c197e90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.995+0000 7f544345c640 1 --2- 172.21.15.71:0/12968847 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f543c101610 0x7f543c197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:42.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.995+0000 7f544345c640 1 -- 172.21.15.71:0/12968847 >> 172.21.15.71:0/12968847 conn(0x7f543c0fcb80 msgr2=0x7f543c105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:42.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.995+0000 7f544345c640 1 -- 172.21.15.71:0/12968847 shutdown_connections 2024-03-04T08:58:42.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:42.995+0000 7f544345c640 1 -- 172.21.15.71:0/12968847 wait complete. 2024-03-04T08:58:43.001 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:42 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2934420036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T08:58:43.001 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:42 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1244198665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T08:58:43.001 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:42 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/697570383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:58:43.002 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-04T08:56:30.343357+0000', 'modified': '2024-03-04T08:56:54.901214+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24491}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.158:6839/2921704804', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.158:6838', 'nonce': 2921704804}, {'type': 'v1', 'addr': '172.21.15.158:6839', 'nonce': 2921704804}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24491': {'gid': 24491, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6839/2639278083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6838', 'nonce': 2639278083}, {'type': 'v1', 'addr': '172.21.15.150:6839', 'nonce': 2639278083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24491, 'qdb_cluster': [24491]} 2024-03-04T08:58:43.003 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-04T08:58:43.003 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-04T08:58:43.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.110+0000 7f4a0099a640 1 -- 172.21.15.71:0/1458446056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49fc105160 msgr2=0x7f49fc1055e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:43.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.110+0000 7f4a0099a640 1 --2- 172.21.15.71:0/1458446056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49fc105160 0x7f49fc1055e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009ca0 tx=0x7f49f402c0f0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.110+0000 7f4a0099a640 1 -- 172.21.15.71:0/1458446056 shutdown_connections 2024-03-04T08:58:43.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.110+0000 7f4a0099a640 1 --2- 172.21.15.71:0/1458446056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49fc105160 0x7f49fc1055e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.110+0000 7f4a0099a640 1 --2- 172.21.15.71:0/1458446056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49fc1027b0 0x7f49fc104c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.110+0000 7f4a0099a640 1 --2- 172.21.15.71:0/1458446056 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f49fc0ffe40 0x7f49fc102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.110+0000 7f4a0099a640 1 -- 172.21.15.71:0/1458446056 >> 172.21.15.71:0/1458446056 conn(0x7f49fc0fba60 msgr2=0x7f49fc0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:43.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.110+0000 7f4a0099a640 1 -- 172.21.15.71:0/1458446056 shutdown_connections 2024-03-04T08:58:43.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.110+0000 7f4a0099a640 1 -- 172.21.15.71:0/1458446056 wait complete. 2024-03-04T08:58:43.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.111+0000 7f4a0099a640 1 Processor -- start 2024-03-04T08:58:43.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.111+0000 7f4a0099a640 1 -- start start 2024-03-04T08:58:43.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.111+0000 7f4a0099a640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f49fc0ffe40 0x7f49fc19de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:43.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.111+0000 7f4a0099a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49fc1027b0 0x7f49fc19e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:43.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.111+0000 7f4a0099a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49fc105160 0x7f49fc1a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:43.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.111+0000 7f4a0099a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc113820 con 0x7f49fc105160 2024-03-04T08:58:43.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.111+0000 7f4a0099a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc1136a0 con 0x7f49fc1027b0 2024-03-04T08:58:43.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.111+0000 7f4a0099a640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc1139a0 con 0x7f49fc0ffe40 2024-03-04T08:58:43.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.111+0000 7f49fad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49fc105160 0x7f49fc1a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:43.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.111+0000 7f49fad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49fc105160 0x7f49fc1a53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60922/0 (socket says 172.21.15.71:60922) 2024-03-04T08:58:43.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.111+0000 7f49fad9d640 1 -- 172.21.15.71:0/3776818623 learned_addr learned my addr 172.21.15.71:0/3776818623 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:43.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.111+0000 7f49f3fff640 1 --2- 172.21.15.71:0/3776818623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49fc1027b0 0x7f49fc19e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:43.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.111+0000 7f49fa59c640 1 --2- 172.21.15.71:0/3776818623 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f49fc0ffe40 0x7f49fc19de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:43.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.111+0000 7f49fad9d640 1 -- 172.21.15.71:0/3776818623 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f49fc0ffe40 msgr2=0x7f49fc19de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:43.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.111+0000 7f49fad9d640 1 --2- 172.21.15.71:0/3776818623 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f49fc0ffe40 0x7f49fc19de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.111+0000 7f49fad9d640 1 -- 172.21.15.71:0/3776818623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49fc1027b0 msgr2=0x7f49fc19e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:43.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.111+0000 7f49fad9d640 1 --2- 172.21.15.71:0/3776818623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49fc1027b0 0x7f49fc19e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.111+0000 7f49fad9d640 1 -- 172.21.15.71:0/3776818623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49fc1a5a20 con 0x7f49fc105160 2024-03-04T08:58:43.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.111+0000 7f49f3fff640 1 --2- 172.21.15.71:0/3776818623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49fc1027b0 0x7f49fc19e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:58:43.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.112+0000 7f49fad9d640 1 --2- 172.21.15.71:0/3776818623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49fc105160 0x7f49fc1a53b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f49f4037040 tx=0x7f49f4032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:43.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.112+0000 7f49f37fe640 1 -- 172.21.15.71:0/3776818623 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f4041020 con 0x7f49fc105160 2024-03-04T08:58:43.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.112+0000 7f49f37fe640 1 -- 172.21.15.71:0/3776818623 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49f4034d90 con 0x7f49fc105160 2024-03-04T08:58:43.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.112+0000 7f4a0099a640 1 -- 172.21.15.71:0/3776818623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49fc1a5cb0 con 0x7f49fc105160 2024-03-04T08:58:43.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.112+0000 7f49f37fe640 1 -- 172.21.15.71:0/3776818623 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f4039040 con 0x7f49fc105160 2024-03-04T08:58:43.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.112+0000 7f4a0099a640 1 -- 172.21.15.71:0/3776818623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49fc1a61a0 con 0x7f49fc105160 2024-03-04T08:58:43.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.114+0000 7f49f37fe640 1 -- 172.21.15.71:0/3776818623 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f49f4044020 con 0x7f49fc105160 2024-03-04T08:58:43.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.114+0000 7f4a03584740 1 Processor -- start 2024-03-04T08:58:43.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.115+0000 7f4a03584740 1 -- start start 2024-03-04T08:58:43.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.115+0000 7f4a03584740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2cd7b0 0x556b8c2cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:43.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.115+0000 7f4a03584740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x556b8c2ce980 0x556b8c2cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:43.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.115+0000 7f4a03584740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b8c2cf340 0x556b8c2d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:43.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.115+0000 7f49f37fe640 1 --2- 172.21.15.71:0/3776818623 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f49e80a1d80 0x7f49e80a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:43.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.115+0000 7f4a03584740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b8c2326b0 con 0x556b8c2cd7b0 2024-03-04T08:58:43.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.115+0000 7f4a03584740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b8c1dba60 con 0x556b8c2cf340 2024-03-04T08:58:43.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.115+0000 7f4a03584740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b8c22e750 con 0x556b8c2ce980 2024-03-04T08:58:43.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.115+0000 7f49f3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2cd7b0 0x556b8c2cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:43.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.115+0000 7f49f3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2cd7b0 0x556b8c2cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60926/0 (socket says 172.21.15.71:60926) 2024-03-04T08:58:43.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.115+0000 7f49f3fff640 1 -- 172.21.15.71:0/3785271975 learned_addr learned my addr 172.21.15.71:0/3785271975 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:43.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.115+0000 7f49fad9d640 1 --2- 172.21.15.71:0/3776818623 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f49e80a1d80 0x7f49e80a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:43.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.115+0000 7f49f3fff640 1 --2- 172.21.15.71:0/3785271975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b8c2cf340 0x556b8c2d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:43.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.115+0000 7f49f37fe640 1 -- 172.21.15.71:0/3776818623 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f49f40f00e0 con 0x7f49fc105160 2024-03-04T08:58:43.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.115+0000 7f49fa59c640 1 --2- 172.21.15.71:0/3785271975 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x556b8c2ce980 0x556b8c2cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:43.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.115+0000 7f49f3fff640 1 -- 172.21.15.71:0/3785271975 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x556b8c2ce980 msgr2=0x556b8c2cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:58:43.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.115+0000 7f49f3fff640 1 --2- 172.21.15.71:0/3785271975 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x556b8c2ce980 0x556b8c2cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.115+0000 7f49f3fff640 1 -- 172.21.15.71:0/3785271975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b8c2cf340 msgr2=0x556b8c2d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:58:43.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.115+0000 7f49f3fff640 1 --2- 172.21.15.71:0/3785271975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b8c2cf340 0x556b8c2d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.115+0000 7f49f3fff640 1 -- 172.21.15.71:0/3785271975 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b8c1f9f30 con 0x556b8c2cd7b0 2024-03-04T08:58:43.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.116+0000 7f49f3fff640 1 --2- 172.21.15.71:0/3785271975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2cd7b0 0x556b8c2cdbd0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f49e4013c30 tx=0x7f49e4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc707d014896af9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:43.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.116+0000 7f49fad9d640 1 --2- 172.21.15.71:0/3776818623 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f49e80a1d80 0x7f49e80a4240 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f49f40c4ca0 tx=0x7f49f40323c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:43.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.116+0000 7f49f0ff9640 1 -- 172.21.15.71:0/3785271975 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e4013de0 con 0x556b8c2cd7b0 2024-03-04T08:58:43.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.116+0000 7f49f0ff9640 1 -- 172.21.15.71:0/3785271975 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49e4013170 con 0x556b8c2cd7b0 2024-03-04T08:58:43.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.116+0000 7f49f0ff9640 1 -- 172.21.15.71:0/3785271975 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e4010d80 con 0x556b8c2cd7b0 2024-03-04T08:58:43.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.116+0000 7f4a03584740 1 -- 172.21.15.71:0/3785271975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2cd7b0 msgr2=0x556b8c2cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:43.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.116+0000 7f4a03584740 1 --2- 172.21.15.71:0/3785271975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2cd7b0 0x556b8c2cdbd0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f49e4013c30 tx=0x7f49e4009b50 comp rx=0 tx=0).stop 2024-03-04T08:58:43.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.116+0000 7f4a03584740 1 -- 172.21.15.71:0/3785271975 shutdown_connections 2024-03-04T08:58:43.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.116+0000 7f4a03584740 1 --2- 172.21.15.71:0/3785271975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b8c2cf340 0x556b8c2d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.116+0000 7f4a03584740 1 --2- 172.21.15.71:0/3785271975 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x556b8c2ce980 0x556b8c2cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.116+0000 7f4a03584740 1 --2- 172.21.15.71:0/3785271975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2cd7b0 0x556b8c2cdbd0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.116+0000 7f4a03584740 1 -- 172.21.15.71:0/3785271975 >> 172.21.15.71:0/3785271975 conn(0x556b8c2c8ba0 msgr2=0x556b8c2c8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:43.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.117+0000 7f4a03584740 1 -- 172.21.15.71:0/3785271975 shutdown_connections 2024-03-04T08:58:43.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.117+0000 7f4a03584740 1 -- 172.21.15.71:0/3785271975 wait complete. 2024-03-04T08:58:43.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.117+0000 7f4a03584740 1 -- start start 2024-03-04T08:58:43.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.117+0000 7f4a03584740 1 Processor -- start 2024-03-04T08:58:43.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.117+0000 7f4a03584740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x556b8c2cd7b0 0x556b8c2d7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:43.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.117+0000 7f4a03584740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2ce980 0x556b8c2d8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:43.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.117+0000 7f4a03584740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b8c2cf340 0x556b8c2df240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:43.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.117+0000 7f4a03584740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b8c2d62f0 con 0x556b8c2ce980 2024-03-04T08:58:43.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.117+0000 7f4a03584740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b8c1eeea0 con 0x556b8c2cf340 2024-03-04T08:58:43.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.117+0000 7f4a03584740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b8c2d6470 con 0x556b8c2cd7b0 2024-03-04T08:58:43.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.117+0000 7f4a03584740 10 client.0 _ll_register_callbacks cb 0x556b8c1ee870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T08:58:43.121 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.117+0000 7f49fa59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2ce980 0x556b8c2d8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:43.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.117+0000 7f49fa59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2ce980 0x556b8c2d8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60940/0 (socket says 172.21.15.71:60940) 2024-03-04T08:58:43.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.117+0000 7f49fa59c640 1 -- 172.21.15.71:0/426148577 learned_addr learned my addr 172.21.15.71:0/426148577 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:43.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.117+0000 7f49f3fff640 1 --2- 172.21.15.71:0/426148577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b8c2cf340 0x556b8c2df240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:43.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.117+0000 7f49f3fff640 1 --2- 172.21.15.71:0/426148577 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x556b8c2cd7b0 0x556b8c2d7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:43.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.117+0000 7f49fa59c640 1 -- 172.21.15.71:0/426148577 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x556b8c2cd7b0 msgr2=0x556b8c2d7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:58:43.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.117+0000 7f49fa59c640 1 --2- 172.21.15.71:0/426148577 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x556b8c2cd7b0 0x556b8c2d7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.117+0000 7f49fa59c640 1 -- 172.21.15.71:0/426148577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b8c2cf340 msgr2=0x556b8c2df240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:58:43.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.117+0000 7f49fa59c640 1 --2- 172.21.15.71:0/426148577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b8c2cf340 0x556b8c2df240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.117+0000 7f49fa59c640 1 -- 172.21.15.71:0/426148577 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b8c1ef270 con 0x556b8c2ce980 2024-03-04T08:58:43.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.117+0000 7f49f3fff640 1 --2- 172.21.15.71:0/426148577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b8c2cf340 0x556b8c2df240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:58:43.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.118+0000 7f49fa59c640 1 --2- 172.21.15.71:0/426148577 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2ce980 0x556b8c2d8200 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f49ec010810 tx=0x7f49ec010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4069854e9de4d6c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:43.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.118+0000 7f49e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-04T08:58:43.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.118+0000 7f49e1ffb640 1 -- 172.21.15.71:0/426148577 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec00ece0 con 0x556b8c2ce980 2024-03-04T08:58:43.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.118+0000 7f49e1ffb640 1 -- 172.21.15.71:0/426148577 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f49ec01c480 con 0x556b8c2ce980 2024-03-04T08:58:43.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.118+0000 7f4a03584740 1 -- 172.21.15.71:0/426148577 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49e4013170 con 0x556b8c2ce980 2024-03-04T08:58:43.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.118+0000 7f49e1ffb640 1 -- 172.21.15.71:0/426148577 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49ec025050 con 0x556b8c2ce980 2024-03-04T08:58:43.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.118+0000 7f4a03584740 10 client.15363 fetch_fsmap learned FSMap version 20 2024-03-04T08:58:43.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.118+0000 7f4a03584740 1 -- 172.21.15.71:0/426148577 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x556b8bf472d0 con 0x556b8c2ce980 2024-03-04T08:58:43.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.118+0000 7f49e1ffb640 1 -- 172.21.15.71:0/426148577 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49e4013170 con 0x556b8c2ce980 2024-03-04T08:58:43.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.118+0000 7f49e1ffb640 1 -- 172.21.15.71:0/426148577 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 20) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f49ec011680 con 0x556b8c2ce980 2024-03-04T08:58:43.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.118+0000 7f4a03584740 10 client.15363 fetch_fsmap finished waiting for FSMap version 20 2024-03-04T08:58:43.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.118+0000 7f4a03584740 10 client.15363 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-04T08:58:43.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.118+0000 7f4a03584740 20 client.15363 populate_metadata read hostname 'smithi071' 2024-03-04T08:58:43.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.118+0000 7f4a03584740 1 --2- 172.21.15.71:0/426148577 >> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] conn(0x556b8c2e48c0 0x556b8c2e3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:43.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.118+0000 7f4a03584740 4 client.15363 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-04T08:58:43.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.118+0000 7f4a03584740 1 -- 172.21.15.71:0/426148577 --> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b8c23fbf0 con 0x556b8c2e48c0 2024-03-04T08:58:43.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.119+0000 7f49f3fff640 1 --2- 172.21.15.71:0/426148577 >> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] conn(0x556b8c2e48c0 0x556b8c2e3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:43.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.119+0000 7f49e1ffb640 10 client.15363 ms_handle_connect on v2:172.21.15.150:6838/2639278083 2024-03-04T08:58:43.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.119+0000 7f49f3fff640 1 --2- 172.21.15.71:0/426148577 >> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] conn(0x556b8c2e48c0 0x556b8c2e3970 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1635c107ed8c799 server_cookie=300483b0b0b21712 in_seq=0 out_seq=0 2024-03-04T08:58:43.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.120+0000 7f49e1ffb640 1 -- 172.21.15.71:0/426148577 <== mds.0 v2:172.21.15.150:6838/2639278083 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556b8c23fbf0 con 0x556b8c2e48c0 2024-03-04T08:58:43.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.120+0000 7f49e1ffb640 10 client.15363 handle_command_reply: tid=0 2024-03-04T08:58:43.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.120+0000 7f4a03584740 1 client.15363 shutdown 2024-03-04T08:58:43.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.120+0000 7f4a03584740 1 -- 172.21.15.71:0/426148577 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2ce980 msgr2=0x556b8c2d8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:43.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.120+0000 7f4a03584740 1 --2- 172.21.15.71:0/426148577 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2ce980 0x556b8c2d8200 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f49ec010810 tx=0x7f49ec010cd0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.120+0000 7f4a03584740 1 -- 172.21.15.71:0/426148577 shutdown_connections 2024-03-04T08:58:43.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.120+0000 7f4a03584740 1 --2- 172.21.15.71:0/426148577 >> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] conn(0x556b8c2e48c0 0x556b8c2e3970 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.120+0000 7f4a03584740 1 --2- 172.21.15.71:0/426148577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b8c2cf340 0x556b8c2df240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.120+0000 7f4a03584740 1 --2- 172.21.15.71:0/426148577 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2ce980 0x556b8c2d8200 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.120+0000 7f49e1ffb640 0 client.15363 ms_handle_reset on v2:172.21.15.150:6838/2639278083 2024-03-04T08:58:43.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.120+0000 7f4a03584740 1 --2- 172.21.15.71:0/426148577 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x556b8c2cd7b0 0x556b8c2d7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.124 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.120+0000 7f4a03584740 1 -- 172.21.15.71:0/426148577 >> 172.21.15.71:0/426148577 conn(0x556b8c2c8ba0 msgr2=0x556b8c2c7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:43.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.120+0000 7f4a03584740 1 -- 172.21.15.71:0/426148577 shutdown_connections 2024-03-04T08:58:43.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.120+0000 7f4a03584740 1 -- 172.21.15.71:0/426148577 wait complete. 2024-03-04T08:58:43.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.120+0000 7f4a03584740 20 client.15363 trim_cache size 0 max 16384 2024-03-04T08:58:43.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.136+0000 7f4a03584740 1 Processor -- start 2024-03-04T08:58:43.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.136+0000 7f4a03584740 1 -- start start 2024-03-04T08:58:43.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.136+0000 7f4a03584740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b8c2cd7b0 0x556b8c2d2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:43.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.136+0000 7f4a03584740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x556b8c2ce980 0x556b8c2d2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:43.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.136+0000 7f4a03584740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2cf340 0x556b8c2df200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:43.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.136+0000 7f4a03584740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b8c2e40e0 con 0x556b8c2cf340 2024-03-04T08:58:43.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.136+0000 7f4a03584740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b8c2e4740 con 0x556b8c2cd7b0 2024-03-04T08:58:43.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.136+0000 7f4a03584740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b8c2e4da0 con 0x556b8c2ce980 2024-03-04T08:58:43.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.136+0000 7f49f3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2cf340 0x556b8c2df200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:43.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.136+0000 7f49f3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2cf340 0x556b8c2df200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60950/0 (socket says 172.21.15.71:60950) 2024-03-04T08:58:43.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.136+0000 7f49f3fff640 1 -- 172.21.15.71:0/2252738954 learned_addr learned my addr 172.21.15.71:0/2252738954 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:43.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.136+0000 7f49f3fff640 1 --2- 172.21.15.71:0/2252738954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b8c2cd7b0 0x556b8c2d2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:43.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.137+0000 7f49fa59c640 1 --2- 172.21.15.71:0/2252738954 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x556b8c2ce980 0x556b8c2d2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:43.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.137+0000 7f49f3fff640 1 -- 172.21.15.71:0/2252738954 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x556b8c2ce980 msgr2=0x556b8c2d2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:58:43.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.137+0000 7f49f3fff640 1 --2- 172.21.15.71:0/2252738954 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x556b8c2ce980 0x556b8c2d2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.137+0000 7f49f3fff640 1 -- 172.21.15.71:0/2252738954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b8c2cd7b0 msgr2=0x556b8c2d2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:58:43.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.137+0000 7f49f3fff640 1 --2- 172.21.15.71:0/2252738954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b8c2cd7b0 0x556b8c2d2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.137+0000 7f49f3fff640 1 -- 172.21.15.71:0/2252738954 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b8c0116b0 con 0x556b8c2cf340 2024-03-04T08:58:43.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.137+0000 7f49f3fff640 1 --2- 172.21.15.71:0/2252738954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2cf340 0x556b8c2df200 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f49e4006010 tx=0x7f49e4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ff9012169b323f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:43.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.137+0000 7f49e3fff640 1 -- 172.21.15.71:0/2252738954 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e4013170 con 0x556b8c2cf340 2024-03-04T08:58:43.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.137+0000 7f49e3fff640 1 -- 172.21.15.71:0/2252738954 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49e402d2d0 con 0x556b8c2cf340 2024-03-04T08:58:43.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.137+0000 7f49e3fff640 1 -- 172.21.15.71:0/2252738954 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e40035f0 con 0x556b8c2cf340 2024-03-04T08:58:43.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.137+0000 7f4a03584740 1 -- 172.21.15.71:0/2252738954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2cf340 msgr2=0x556b8c2df200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:43.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.137+0000 7f4a03584740 1 --2- 172.21.15.71:0/2252738954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2cf340 0x556b8c2df200 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f49e4006010 tx=0x7f49e4009b50 comp rx=0 tx=0).stop 2024-03-04T08:58:43.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f4a03584740 1 -- 172.21.15.71:0/2252738954 shutdown_connections 2024-03-04T08:58:43.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f4a03584740 1 --2- 172.21.15.71:0/2252738954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2cf340 0x556b8c2df200 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f4a03584740 1 --2- 172.21.15.71:0/2252738954 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x556b8c2ce980 0x556b8c2d2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f4a03584740 1 --2- 172.21.15.71:0/2252738954 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b8c2cd7b0 0x556b8c2d2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f4a03584740 1 -- 172.21.15.71:0/2252738954 >> 172.21.15.71:0/2252738954 conn(0x556b8c2c8ba0 msgr2=0x556b8c2c8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:43.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f4a03584740 1 -- 172.21.15.71:0/2252738954 shutdown_connections 2024-03-04T08:58:43.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f4a03584740 1 -- 172.21.15.71:0/2252738954 wait complete. 2024-03-04T08:58:43.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f4a03584740 1 -- start start 2024-03-04T08:58:43.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f4a03584740 1 Processor -- start 2024-03-04T08:58:43.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f4a03584740 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x556b8c2cd7b0 0x556b8c2d9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:43.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f4a03584740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b8c2ce980 0x556b8c2d9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:43.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f4a03584740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2cf340 0x556b8c2de960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:43.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f4a03584740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b8c2e5a60 con 0x556b8c2cf340 2024-03-04T08:58:43.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f4a03584740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b8c2e5400 con 0x556b8c2ce980 2024-03-04T08:58:43.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f4a03584740 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b8c2e60c0 con 0x556b8c2cd7b0 2024-03-04T08:58:43.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f4a03584740 10 client.0 _ll_register_callbacks cb 0x7f49ec0103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T08:58:43.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f49f3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2cf340 0x556b8c2de960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:43.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f49fa59c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b8c2ce980 0x556b8c2d9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:43.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f49fa59c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b8c2ce980 0x556b8c2d9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.71:57112/0 (socket says 172.21.15.71:57112) 2024-03-04T08:58:43.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f49fa59c640 1 -- 172.21.15.71:0/3393019066 learned_addr learned my addr 172.21.15.71:0/3393019066 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:43.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f49f3fff640 1 --2- 172.21.15.71:0/3393019066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2cf340 0x556b8c2de960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60958/0 (socket says 172.21.15.71:60958) 2024-03-04T08:58:43.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f49f3fff640 1 --2- 172.21.15.71:0/3393019066 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x556b8c2cd7b0 0x556b8c2d9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:43.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f49f3fff640 1 -- 172.21.15.71:0/3393019066 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x556b8c2cd7b0 msgr2=0x556b8c2d9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:58:43.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f49f3fff640 1 --2- 172.21.15.71:0/3393019066 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x556b8c2cd7b0 0x556b8c2d9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f49f3fff640 1 -- 172.21.15.71:0/3393019066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b8c2ce980 msgr2=0x556b8c2d9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:58:43.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f49f3fff640 1 --2- 172.21.15.71:0/3393019066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b8c2ce980 0x556b8c2d9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.138+0000 7f49f3fff640 1 -- 172.21.15.71:0/3393019066 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b8c2def50 con 0x556b8c2cf340 2024-03-04T08:58:43.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.139+0000 7f49f3fff640 1 --2- 172.21.15.71:0/3393019066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2cf340 0x556b8c2de960 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x556b8c2cf0a0 tx=0x7f49e4013820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a30fbbd0318455 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:43.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.139+0000 7f49e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-03-04T08:58:43.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.139+0000 7f49e1ffb640 1 -- 172.21.15.71:0/3393019066 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e402d840 con 0x556b8c2cf340 2024-03-04T08:58:43.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.139+0000 7f49e1ffb640 1 -- 172.21.15.71:0/3393019066 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f49e4005110 con 0x556b8c2cf340 2024-03-04T08:58:43.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.139+0000 7f4a03584740 1 -- 172.21.15.71:0/3393019066 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49e402d2d0 con 0x556b8c2cf340 2024-03-04T08:58:43.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.139+0000 7f49e1ffb640 1 -- 172.21.15.71:0/3393019066 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49e402d9e0 con 0x556b8c2cf340 2024-03-04T08:58:43.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.139+0000 7f4a03584740 10 client.15369 fetch_fsmap learned FSMap version 20 2024-03-04T08:58:43.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.139+0000 7f4a03584740 1 -- 172.21.15.71:0/3393019066 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x556b8c340440 con 0x556b8c2cf340 2024-03-04T08:58:43.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.139+0000 7f49e1ffb640 1 -- 172.21.15.71:0/3393019066 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49e402d2d0 con 0x556b8c2cf340 2024-03-04T08:58:43.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.139+0000 7f49e1ffb640 1 -- 172.21.15.71:0/3393019066 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 20) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f49e4038450 con 0x556b8c2cf340 2024-03-04T08:58:43.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.139+0000 7f4a03584740 10 client.15369 fetch_fsmap finished waiting for FSMap version 20 2024-03-04T08:58:43.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.139+0000 7f4a03584740 10 client.15369 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-04T08:58:43.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.139+0000 7f4a03584740 20 client.15369 populate_metadata read hostname 'smithi071' 2024-03-04T08:58:43.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.139+0000 7f4a03584740 1 --2- 172.21.15.71:0/3393019066 >> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] conn(0x556b8c3446b0 0x556b8c340610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:43.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.139+0000 7f4a03584740 4 client.15369 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-04T08:58:43.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.139+0000 7f4a03584740 1 -- 172.21.15.71:0/3393019066 --> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556b8c239f40 con 0x556b8c3446b0 2024-03-04T08:58:43.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.139+0000 7f49fa59c640 1 --2- 172.21.15.71:0/3393019066 >> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] conn(0x556b8c3446b0 0x556b8c340610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:43.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.140+0000 7f49e1ffb640 10 client.15369 ms_handle_connect on v2:172.21.15.150:6838/2639278083 2024-03-04T08:58:43.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.140+0000 7f49fa59c640 1 --2- 172.21.15.71:0/3393019066 >> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] conn(0x556b8c3446b0 0x556b8c340610 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81b9d9809a07f141 server_cookie=d479ad7faef81170 in_seq=0 out_seq=0 2024-03-04T08:58:43.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.140+0000 7f49e1ffb640 1 -- 172.21.15.71:0/3393019066 <== mds.0 v2:172.21.15.150:6838/2639278083 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x556b8c239f40 con 0x556b8c3446b0 2024-03-04T08:58:43.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.140+0000 7f49e1ffb640 10 client.15369 handle_command_reply: tid=0 2024-03-04T08:58:43.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.140+0000 7f4a03584740 1 client.15369 shutdown 2024-03-04T08:58:43.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.140+0000 7f4a03584740 1 -- 172.21.15.71:0/3393019066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2cf340 msgr2=0x556b8c2de960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:43.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.140+0000 7f4a03584740 1 --2- 172.21.15.71:0/3393019066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2cf340 0x556b8c2de960 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x556b8c2cf0a0 tx=0x7f49e4013820 comp rx=0 tx=0).stop 2024-03-04T08:58:43.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.140+0000 7f4a03584740 1 -- 172.21.15.71:0/3393019066 shutdown_connections 2024-03-04T08:58:43.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.140+0000 7f4a03584740 1 --2- 172.21.15.71:0/3393019066 >> [v2:172.21.15.150:6838/2639278083,v1:172.21.15.150:6839/2639278083] conn(0x556b8c3446b0 0x556b8c340610 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.140+0000 7f4a03584740 1 --2- 172.21.15.71:0/3393019066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b8c2cf340 0x556b8c2de960 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.140+0000 7f49e1ffb640 0 client.15369 ms_handle_reset on v2:172.21.15.150:6838/2639278083 2024-03-04T08:58:43.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.140+0000 7f4a03584740 1 --2- 172.21.15.71:0/3393019066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556b8c2ce980 0x556b8c2d9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.140+0000 7f4a03584740 1 --2- 172.21.15.71:0/3393019066 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x556b8c2cd7b0 0x556b8c2d9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.140+0000 7f4a03584740 1 -- 172.21.15.71:0/3393019066 >> 172.21.15.71:0/3393019066 conn(0x556b8c2c8ba0 msgr2=0x556b8c2cbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:43.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.140+0000 7f4a03584740 1 -- 172.21.15.71:0/3393019066 shutdown_connections 2024-03-04T08:58:43.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.140+0000 7f4a03584740 1 -- 172.21.15.71:0/3393019066 wait complete. 2024-03-04T08:58:43.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.140+0000 7f4a03584740 20 client.15369 trim_cache size 0 max 16384 2024-03-04T08:58:43.150 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-03-04T08:58:43.150 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "b3c85c40-da03-11ee-95c6-87774f69a715", 2024-03-04T08:58:43.150 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2024-03-04T08:58:43.150 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 24491, 2024-03-04T08:58:43.151 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-03-04T08:58:43.151 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-03-04T08:58:43.151 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-03-04T08:58:43.151 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 132.148172786, 2024-03-04T08:58:43.151 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 20, 2024-03-04T08:58:43.151 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 95, 2024-03-04T08:58:43.151 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 91, 2024-03-04T08:58:43.151 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 137.29437439099999 2024-03-04T08:58:43.151 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-03-04T08:58:43.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.141+0000 7f4a0099a640 1 -- 172.21.15.71:0/3776818623 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f49e80a1d80 msgr2=0x7f49e80a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:43.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.141+0000 7f4a0099a640 1 --2- 172.21.15.71:0/3776818623 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f49e80a1d80 0x7f49e80a4240 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f49f40c4ca0 tx=0x7f49f40323c0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.141+0000 7f4a0099a640 1 -- 172.21.15.71:0/3776818623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49fc105160 msgr2=0x7f49fc1a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:43.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.141+0000 7f4a0099a640 1 --2- 172.21.15.71:0/3776818623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49fc105160 0x7f49fc1a53b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f49f4037040 tx=0x7f49f4032f20 comp rx=0 tx=0).stop 2024-03-04T08:58:43.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.141+0000 7f4a0099a640 1 -- 172.21.15.71:0/3776818623 shutdown_connections 2024-03-04T08:58:43.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.141+0000 7f4a0099a640 1 --2- 172.21.15.71:0/3776818623 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f49e80a1d80 0x7f49e80a4240 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.141+0000 7f4a0099a640 1 --2- 172.21.15.71:0/3776818623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49fc105160 0x7f49fc1a53b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.141+0000 7f4a0099a640 1 --2- 172.21.15.71:0/3776818623 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49fc1027b0 0x7f49fc19e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.141+0000 7f4a0099a640 1 --2- 172.21.15.71:0/3776818623 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f49fc0ffe40 0x7f49fc19de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:43.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.141+0000 7f4a0099a640 1 -- 172.21.15.71:0/3776818623 >> 172.21.15.71:0/3776818623 conn(0x7f49fc0fba60 msgr2=0x7f49fc1007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:43.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.141+0000 7f4a0099a640 1 -- 172.21.15.71:0/3776818623 shutdown_connections 2024-03-04T08:58:43.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:43.141+0000 7f4a0099a640 1 -- 172.21.15.71:0/3776818623 wait complete. 2024-03-04T08:58:43.154 INFO:teuthology.run_tasks:Running task kclient... 2024-03-04T08:58:43.168 INFO:tasks.kclient:Mounting kernel clients... 2024-03-04T08:58:43.168 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-04T08:58:43.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:58:43.169 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-03-04T08:58:43.185 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-04T08:58:43.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:58:43.185 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-04T08:58:43.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:58:43.252 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link delete ceph-brx 2024-03-04T08:58:43.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:42 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2934420036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T08:58:43.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:42 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1244198665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T08:58:43.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:42 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/697570383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:58:43.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:42 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2934420036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T08:58:43.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:42 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1244198665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T08:58:43.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:42 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/697570383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T08:58:43.345 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-03-04T08:58:43.346 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-04T08:58:43.346 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-04T08:58:43.346 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph fs ls 2024-03-04T08:58:43.590 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T08:58:44.034 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:43 smithi071 ceph-mon[32789]: pgmap v395: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:58:44.035 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:43 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/12968847' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:58:44.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:43 smithi150 ceph-mon[36807]: pgmap v395: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:58:44.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:43 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/12968847' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:58:44.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:43 smithi158 ceph-mon[37241]: pgmap v395: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:58:44.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:43 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/12968847' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T08:58:45.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.256+0000 7f8f3fd0c700 1 -- 172.21.15.71:0/752532454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f38102110 msgr2=0x7f8f38102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:45.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.256+0000 7f8f3fd0c700 1 --2- 172.21.15.71:0/752532454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f38102110 0x7f8f38102590 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8f280099c0 tx=0x7f8f28009cd0 comp rx=0 tx=0).stop 2024-03-04T08:58:45.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.256+0000 7f8f3fd0c700 1 -- 172.21.15.71:0/752532454 shutdown_connections 2024-03-04T08:58:45.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.256+0000 7f8f3fd0c700 1 --2- 172.21.15.71:0/752532454 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f381080f0 0x7f8f381084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:45.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.256+0000 7f8f3fd0c700 1 --2- 172.21.15.71:0/752532454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f38102ad0 0x7f8f3810ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:45.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.256+0000 7f8f3fd0c700 1 --2- 172.21.15.71:0/752532454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f38102110 0x7f8f38102590 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:45.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.256+0000 7f8f3fd0c700 1 -- 172.21.15.71:0/752532454 >> 172.21.15.71:0/752532454 conn(0x7f8f380791f0 msgr2=0x7f8f38079600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:45.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.257+0000 7f8f3fd0c700 1 -- 172.21.15.71:0/752532454 shutdown_connections 2024-03-04T08:58:45.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.257+0000 7f8f3fd0c700 1 -- 172.21.15.71:0/752532454 wait complete. 2024-03-04T08:58:45.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.257+0000 7f8f3fd0c700 1 Processor -- start 2024-03-04T08:58:45.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.257+0000 7f8f3fd0c700 1 -- start start 2024-03-04T08:58:45.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.257+0000 7f8f3fd0c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f38102110 0x7f8f3819e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:45.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.257+0000 7f8f3fd0c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f38102ad0 0x7f8f3819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:45.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.257+0000 7f8f3fd0c700 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f381080f0 0x7f8f381985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:45.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.257+0000 7f8f3fd0c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f38112390 con 0x7f8f38102110 2024-03-04T08:58:45.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.257+0000 7f8f3fd0c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f38112210 con 0x7f8f38102ad0 2024-03-04T08:58:45.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.257+0000 7f8f3fd0c700 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f38112510 con 0x7f8f381080f0 2024-03-04T08:58:45.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.258+0000 7f8f3daa8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f38102110 0x7f8f3819e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:45.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.258+0000 7f8f3daa8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f38102110 0x7f8f3819e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60968/0 (socket says 172.21.15.71:60968) 2024-03-04T08:58:45.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.258+0000 7f8f3d2a7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f38102ad0 0x7f8f3819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:45.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.258+0000 7f8f3daa8700 1 -- 172.21.15.71:0/3840590926 learned_addr learned my addr 172.21.15.71:0/3840590926 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:45.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.258+0000 7f8f3e2a9700 1 --2- 172.21.15.71:0/3840590926 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f381080f0 0x7f8f381985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:45.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.258+0000 7f8f3daa8700 1 -- 172.21.15.71:0/3840590926 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f381080f0 msgr2=0x7f8f381985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:45.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.258+0000 7f8f3daa8700 1 --2- 172.21.15.71:0/3840590926 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f381080f0 0x7f8f381985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:45.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.258+0000 7f8f3daa8700 1 -- 172.21.15.71:0/3840590926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f38102ad0 msgr2=0x7f8f3819e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:45.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.258+0000 7f8f3daa8700 1 --2- 172.21.15.71:0/3840590926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f38102ad0 0x7f8f3819e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:45.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.258+0000 7f8f3daa8700 1 -- 172.21.15.71:0/3840590926 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f38198d50 con 0x7f8f38102110 2024-03-04T08:58:45.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.258+0000 7f8f3e2a9700 1 --2- 172.21.15.71:0/3840590926 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f381080f0 0x7f8f381985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:58:45.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.258+0000 7f8f3d2a7700 1 --2- 172.21.15.71:0/3840590926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f38102ad0 0x7f8f3819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:58:45.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.258+0000 7f8f3daa8700 1 --2- 172.21.15.71:0/3840590926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f38102110 0x7f8f3819e430 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000ada0 tx=0x7f8f3000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:45.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.258+0000 7f8f2effd700 1 -- 172.21.15.71:0/3840590926 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30017020 con 0x7f8f38102110 2024-03-04T08:58:45.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.258+0000 7f8f2effd700 1 -- 172.21.15.71:0/3840590926 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f30004bb0 con 0x7f8f38102110 2024-03-04T08:58:45.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.258+0000 7f8f2effd700 1 -- 172.21.15.71:0/3840590926 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f300054a0 con 0x7f8f38102110 2024-03-04T08:58:45.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.258+0000 7f8f3fd0c700 1 -- 172.21.15.71:0/3840590926 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f38199040 con 0x7f8f38102110 2024-03-04T08:58:45.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.258+0000 7f8f3fd0c700 1 -- 172.21.15.71:0/3840590926 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f381a5300 con 0x7f8f38102110 2024-03-04T08:58:45.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.259+0000 7f8f3fd0c700 1 -- 172.21.15.71:0/3840590926 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f38068d80 con 0x7f8f38102110 2024-03-04T08:58:45.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.260+0000 7f8f2effd700 1 -- 172.21.15.71:0/3840590926 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f8f30005680 con 0x7f8f38102110 2024-03-04T08:58:45.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.261+0000 7f8f2effd700 1 --2- 172.21.15.71:0/3840590926 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f8f240a1e90 0x7f8f240a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:45.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.261+0000 7f8f2effd700 1 -- 172.21.15.71:0/3840590926 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f8f300cd7f0 con 0x7f8f38102110 2024-03-04T08:58:45.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.261+0000 7f8f3d2a7700 1 --2- 172.21.15.71:0/3840590926 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f8f240a1e90 0x7f8f240a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:45.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.261+0000 7f8f3d2a7700 1 --2- 172.21.15.71:0/3840590926 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f8f240a1e90 0x7f8f240a4350 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8f380797f0 tx=0x7f8f28005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:45.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.265+0000 7f8f2effd700 1 -- 172.21.15.71:0/3840590926 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8f30021020 con 0x7f8f38102110 2024-03-04T08:58:45.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.504+0000 7f8f3fd0c700 1 -- 172.21.15.71:0/3840590926 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8f3802d9b0 con 0x7f8f38102110 2024-03-04T08:58:45.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.505+0000 7f8f2effd700 1 -- 172.21.15.71:0/3840590926 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) v1 ==== 53+0+88 (secure 0 0 0) 0x7f8f3009a930 con 0x7f8f38102110 2024-03-04T08:58:45.501 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-04T08:58:45.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.507+0000 7f8f3fd0c700 1 -- 172.21.15.71:0/3840590926 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f8f240a1e90 msgr2=0x7f8f240a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:45.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.507+0000 7f8f3fd0c700 1 --2- 172.21.15.71:0/3840590926 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f8f240a1e90 0x7f8f240a4350 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8f380797f0 tx=0x7f8f28005fb0 comp rx=0 tx=0).stop 2024-03-04T08:58:45.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.507+0000 7f8f3fd0c700 1 -- 172.21.15.71:0/3840590926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f38102110 msgr2=0x7f8f3819e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:45.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.507+0000 7f8f3fd0c700 1 --2- 172.21.15.71:0/3840590926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f38102110 0x7f8f3819e430 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000ada0 tx=0x7f8f3000c330 comp rx=0 tx=0).stop 2024-03-04T08:58:45.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.507+0000 7f8f3fd0c700 1 -- 172.21.15.71:0/3840590926 shutdown_connections 2024-03-04T08:58:45.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.507+0000 7f8f3fd0c700 1 --2- 172.21.15.71:0/3840590926 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f8f240a1e90 0x7f8f240a4350 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:45.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.507+0000 7f8f3fd0c700 1 --2- 172.21.15.71:0/3840590926 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8f381080f0 0x7f8f381985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:45.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.507+0000 7f8f3fd0c700 1 --2- 172.21.15.71:0/3840590926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f38102110 0x7f8f3819e430 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:45.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.507+0000 7f8f3fd0c700 1 --2- 172.21.15.71:0/3840590926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f38102ad0 0x7f8f3819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:45.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.507+0000 7f8f3fd0c700 1 -- 172.21.15.71:0/3840590926 >> 172.21.15.71:0/3840590926 conn(0x7f8f380791f0 msgr2=0x7f8f38100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:45.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.507+0000 7f8f3fd0c700 1 -- 172.21.15.71:0/3840590926 shutdown_connections 2024-03-04T08:58:45.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:45.507+0000 7f8f3fd0c700 1 -- 172.21.15.71:0/3840590926 wait complete. 2024-03-04T08:58:46.149 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:45 smithi071 ceph-mon[32789]: pgmap v396: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:46.149 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:45 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3840590926' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-04T08:58:46.175 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-04T08:58:46.175 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-04T08:58:46.175 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2024-03-04T08:58:46.175 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-04T08:58:46.175 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-04T08:58:46.175 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-04T08:58:46.175 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-04T08:58:46.175 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-04T08:58:46.175 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-04T08:58:46.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:58:46.175 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2024-03-04T08:58:46.191 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-04T08:58:46.191 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-04T08:58:46.191 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2024-03-04T08:58:46.191 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-03-04T08:58:46.191 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2024-03-04T08:58:46.191 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-03-04T08:58:46.192 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T08:58:46.192 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2024-03-04T08:58:46.192 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2024-03-04T08:58:46.192 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T08:58:46.192 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2024-03-04T08:58:46.192 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2024-03-04T08:58:46.193 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T08:58:46.193 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2024-03-04T08:58:46.193 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-04T08:58:46.193 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2024-03-04T08:58:46.193 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-04T08:58:46.193 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 41988sec preferred_lft 41988sec 2024-03-04T08:58:46.193 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link 2024-03-04T08:58:46.193 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-03-04T08:58:46.194 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-04T08:58:46.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:58:46.194 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2024-03-04T08:58:46.194 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-03-04T08:58:46.194 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link add name ceph-brx type bridge 2024-03-04T08:58:46.194 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip addr flush dev ceph-brx 2024-03-04T08:58:46.194 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set ceph-brx up 2024-03-04T08:58:46.194 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-04T08:58:46.194 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-03-04T08:58:46.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:45 smithi150 ceph-mon[36807]: pgmap v396: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:46.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:45 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3840590926' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-04T08:58:46.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:45 smithi158 ceph-mon[37241]: pgmap v396: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:58:46.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:45 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3840590926' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-04T08:58:46.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:58:46.303 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-04T08:58:46.330 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-03-04T08:58:46.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:58:46.331 DEBUG:teuthology.orchestra.run.smithi071:> route 2024-03-04T08:58:46.388 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2024-03-04T08:58:46.388 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-04T08:58:46.388 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-04T08:58:46.388 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-04T08:58:46.388 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-04T08:58:46.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:58:46.389 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2024-03-04T08:58:46.389 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-03-04T08:58:46.389 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-04T08:58:46.389 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-04T08:58:46.389 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-04T08:58:46.389 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-03-04T08:58:46.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:58:46.484 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-03-04T08:58:46.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:58:46.499 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list-id 2024-03-04T08:58:46.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:58:46.554 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2024-03-04T08:58:46.554 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-03-04T08:58:46.554 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-04T08:58:46.554 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-04T08:58:46.554 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-03-04T08:58:46.636 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-04T08:58:46.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:58:46.636 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2024-03-04T08:58:46.636 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-03-04T08:58:46.636 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-04T08:58:46.636 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-04T08:58:46.636 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-04T08:58:46.637 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-04T08:58:46.637 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-04T08:58:46.637 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-03-04T08:58:46.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:58:46.763 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2024-03-04T08:58:46.763 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-03-04T08:58:46.763 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2024-03-04T08:58:46.763 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set dev brx.0 master ceph-brx 2024-03-04T08:58:46.763 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-03-04T08:58:46.806 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T08:58:46.806 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-04T08:58:46.859 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-04T08:58:46.859 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T08:58:46.859 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-04T08:58:46.914 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-04T08:58:46.914 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-04T08:58:46.914 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/60c26083-5e79-4991-b9a9-8838ea2e9acd 2024-03-04T08:58:46.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:58:46.914 DEBUG:teuthology.orchestra.run.smithi071:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/60c26083-5e79-4991-b9a9-8838ea2e9acd /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-04T08:58:46.998 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-04T08:58:46.998 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-03-04T08:58:46.998 INFO:teuthology.orchestra.run.smithi071.stdout:invalid new device string format 2024-03-04T08:58:47.116 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-04T08:58:47.116 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-03-04T08:58:47.116 INFO:teuthology.orchestra.run.smithi071.stdout:invalid new device string format 2024-03-04T08:58:47.116 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: resolved to: "172.21.15.71:6789,172.21.15.150:6789,172.21.15.158:6789" 2024-03-04T08:58:47.116 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.71:6789,172.21.15.150:6789,172.21.15.158:6789:/volumes/qa/sv_1/60c26083-5e79-4991-b9a9-8838ea2e9acd 2024-03-04T08:58:47.117 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=b3c85c40-da03-11ee-95c6-87774f69a715" will pass to kernel 2024-03-04T08:58:47.117 INFO:teuthology.orchestra.run.smithi071.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-04T08:58:47.117 INFO:teuthology.orchestra.run.smithi071.stdout: You just mounted a file system that supports labels which does not 2024-03-04T08:58:47.117 INFO:teuthology.orchestra.run.smithi071.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-04T08:58:47.117 INFO:teuthology.orchestra.run.smithi071.stdout: applications will generate AVC messages and not be allowed access to 2024-03-04T08:58:47.117 INFO:teuthology.orchestra.run.smithi071.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-04T08:58:47.119 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-04T08:58:47.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:58:47.119 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-04T08:58:47.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:58:47.148 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-04T08:58:47.204 INFO:teuthology.orchestra.run.smithi071.stdout:client25208 2024-03-04T08:58:47.205 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-04T08:58:47.205 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T08:58:47.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:58:47.262 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-04T08:58:47.318 INFO:teuthology.orchestra.run.smithi071.stdout:client25208 2024-03-04T08:58:47.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:58:47.319 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-04T08:58:47.375 INFO:teuthology.orchestra.run.smithi071.stdout:client25208 2024-03-04T08:58:47.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:58:47.375 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/b3c85c40-da03-11ee-95c6-87774f69a715.client25208/status') 2024-03-04T08:58:47.442 INFO:teuthology.orchestra.run.smithi071.stdout:instance: client.25208 (3)172.21.15.71:0/3969971160 2024-03-04T08:58:47.442 INFO:teuthology.orchestra.run.smithi071.stdout:blocklisted: false 2024-03-04T08:58:47.442 INFO:teuthology.orchestra.run.smithi071.stderr:0+1 records in 2024-03-04T08:58:47.442 INFO:teuthology.orchestra.run.smithi071.stderr:0+1 records out 2024-03-04T08:58:47.442 INFO:teuthology.orchestra.run.smithi071.stderr:71 bytes copied, 2.1825e-05 s, 3.3 MB/s 2024-03-04T08:58:47.443 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-04T08:58:47.455 INFO:teuthology.run_tasks:Running task exec... 2024-03-04T08:58:47.513 INFO:teuthology.task.exec:Executing custom commands... 2024-03-04T08:58:47.514 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi071.front.sepia.ceph.com 2024-03-04T08:58:47.514 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-04T08:58:47.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.619+0000 7f591dbae640 1 -- 172.21.15.71:0/3352011936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5918102c30 msgr2=0x7f5918103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:47.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.619+0000 7f591dbae640 1 --2- 172.21.15.71:0/3352011936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5918102c30 0x7f5918103050 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5908009900 tx=0x7f590802bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:47.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.619+0000 7f591dbae640 1 -- 172.21.15.71:0/3352011936 shutdown_connections 2024-03-04T08:58:47.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.619+0000 7f591dbae640 1 --2- 172.21.15.71:0/3352011936 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5918100060 0x7f5918109c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:47.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.619+0000 7f591dbae640 1 --2- 172.21.15.71:0/3352011936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f59180ff6a0 0x7f59180ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:47.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.619+0000 7f591dbae640 1 --2- 172.21.15.71:0/3352011936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5918102c30 0x7f5918103050 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:47.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.619+0000 7f591dbae640 1 -- 172.21.15.71:0/3352011936 >> 172.21.15.71:0/3352011936 conn(0x7f59180fb220 msgr2=0x7f59180fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:47.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.620+0000 7f591dbae640 1 -- 172.21.15.71:0/3352011936 shutdown_connections 2024-03-04T08:58:47.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.620+0000 7f591dbae640 1 -- 172.21.15.71:0/3352011936 wait complete. 2024-03-04T08:58:47.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.620+0000 7f591dbae640 1 Processor -- start 2024-03-04T08:58:47.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.620+0000 7f591dbae640 1 -- start start 2024-03-04T08:58:47.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.620+0000 7f591dbae640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59180ff6a0 0x7f59181978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:47.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.620+0000 7f591dbae640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5918100060 0x7f5918197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:47.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.620+0000 7f591dbae640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5918102c30 0x7f591819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:47.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.620+0000 7f591dbae640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591810d2c0 con 0x7f5918102c30 2024-03-04T08:58:47.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.620+0000 7f591dbae640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591810d140 con 0x7f5918100060 2024-03-04T08:58:47.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.620+0000 7f591dbae640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591810d440 con 0x7f59180ff6a0 2024-03-04T08:58:47.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.621+0000 7f5917fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5918102c30 0x7f591819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:47.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.621+0000 7f5917fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5918102c30 0x7f591819ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37096/0 (socket says 172.21.15.71:37096) 2024-03-04T08:58:47.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.621+0000 7f5917fff640 1 -- 172.21.15.71:0/3368966478 learned_addr learned my addr 172.21.15.71:0/3368966478 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:47.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.621+0000 7f59177fe640 1 --2- 172.21.15.71:0/3368966478 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59180ff6a0 0x7f59181978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:47.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.621+0000 7f5916ffd640 1 --2- 172.21.15.71:0/3368966478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5918100060 0x7f5918197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:47.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.621+0000 7f5917fff640 1 -- 172.21.15.71:0/3368966478 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59180ff6a0 msgr2=0x7f59181978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:47.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.621+0000 7f5917fff640 1 --2- 172.21.15.71:0/3368966478 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59180ff6a0 0x7f59181978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:47.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.621+0000 7f5917fff640 1 -- 172.21.15.71:0/3368966478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5918100060 msgr2=0x7f5918197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:47.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.621+0000 7f5917fff640 1 --2- 172.21.15.71:0/3368966478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5918100060 0x7f5918197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:47.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.621+0000 7f5917fff640 1 -- 172.21.15.71:0/3368966478 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591819f4f0 con 0x7f5918102c30 2024-03-04T08:58:47.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.621+0000 7f5917fff640 1 --2- 172.21.15.71:0/3368966478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5918102c30 0x7f591819ee80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f591000eaa0 tx=0x7f591000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:47.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.621+0000 7f5914ff9640 1 -- 172.21.15.71:0/3368966478 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591000cdb0 con 0x7f5918102c30 2024-03-04T08:58:47.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.621+0000 7f591dbae640 1 -- 172.21.15.71:0/3368966478 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591819f7e0 con 0x7f5918102c30 2024-03-04T08:58:47.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.621+0000 7f5914ff9640 1 -- 172.21.15.71:0/3368966478 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5910005020 con 0x7f5918102c30 2024-03-04T08:58:47.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.621+0000 7f5914ff9640 1 -- 172.21.15.71:0/3368966478 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59100116b0 con 0x7f5918102c30 2024-03-04T08:58:47.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.621+0000 7f591dbae640 1 -- 172.21.15.71:0/3368966478 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591819fd30 con 0x7f5918102c30 2024-03-04T08:58:47.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.622+0000 7f591dbae640 1 -- 172.21.15.71:0/3368966478 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5918108a90 con 0x7f5918102c30 2024-03-04T08:58:47.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.623+0000 7f5914ff9640 1 -- 172.21.15.71:0/3368966478 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f5910005560 con 0x7f5918102c30 2024-03-04T08:58:47.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.624+0000 7f5914ff9640 1 --2- 172.21.15.71:0/3368966478 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f59000a1e20 0x7f59000a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:47.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.624+0000 7f59177fe640 1 --2- 172.21.15.71:0/3368966478 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f59000a1e20 0x7f59000a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:47.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.624+0000 7f5914ff9640 1 -- 172.21.15.71:0/3368966478 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f59100ce2c0 con 0x7f5918102c30 2024-03-04T08:58:47.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.624+0000 7f59177fe640 1 --2- 172.21.15.71:0/3368966478 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f59000a1e20 0x7f59000a42e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5908000c00 tx=0x7f5908005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:47.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.628+0000 7f5914ff9640 1 -- 172.21.15.71:0/3368966478 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5910016030 con 0x7f5918102c30 2024-03-04T08:58:47.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.850+0000 7f591dbae640 1 -- 172.21.15.71:0/3368966478 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f59180621d0 con 0x7f5918102c30 2024-03-04T08:58:47.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.935+0000 7f5914ff9640 1 -- 172.21.15.71:0/3368966478 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f591009ad20 con 0x7f5918102c30 2024-03-04T08:58:47.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.938+0000 7f591dbae640 1 -- 172.21.15.71:0/3368966478 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f59000a1e20 msgr2=0x7f59000a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:47.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.938+0000 7f591dbae640 1 --2- 172.21.15.71:0/3368966478 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f59000a1e20 0x7f59000a42e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5908000c00 tx=0x7f5908005fd0 comp rx=0 tx=0).stop 2024-03-04T08:58:47.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.938+0000 7f591dbae640 1 -- 172.21.15.71:0/3368966478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5918102c30 msgr2=0x7f591819ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:47.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.938+0000 7f591dbae640 1 --2- 172.21.15.71:0/3368966478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5918102c30 0x7f591819ee80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f591000eaa0 tx=0x7f591000ef60 comp rx=0 tx=0).stop 2024-03-04T08:58:47.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.938+0000 7f591dbae640 1 -- 172.21.15.71:0/3368966478 shutdown_connections 2024-03-04T08:58:47.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.938+0000 7f591dbae640 1 --2- 172.21.15.71:0/3368966478 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f59000a1e20 0x7f59000a42e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:47.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.938+0000 7f591dbae640 1 --2- 172.21.15.71:0/3368966478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5918102c30 0x7f591819ee80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:47.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.938+0000 7f591dbae640 1 --2- 172.21.15.71:0/3368966478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5918100060 0x7f5918197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:47.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.938+0000 7f591dbae640 1 --2- 172.21.15.71:0/3368966478 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f59180ff6a0 0x7f59181978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:47.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.938+0000 7f591dbae640 1 -- 172.21.15.71:0/3368966478 >> 172.21.15.71:0/3368966478 conn(0x7f59180fb220 msgr2=0x7f5918105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:47.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.940+0000 7f591dbae640 1 -- 172.21.15.71:0/3368966478 shutdown_connections 2024-03-04T08:58:47.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:47.940+0000 7f591dbae640 1 -- 172.21.15.71:0/3368966478 wait complete. 2024-03-04T08:58:47.954 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-04T08:58:48.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.066+0000 7f602beb8640 1 -- 172.21.15.71:0/762284245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60241016e0 msgr2=0x7f6024101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:48.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.066+0000 7f602beb8640 1 --2- 172.21.15.71:0/762284245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60241016e0 0x7f6024101b60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6014009900 tx=0x7f601402bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:48.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.066+0000 7f602beb8640 1 -- 172.21.15.71:0/762284245 shutdown_connections 2024-03-04T08:58:48.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.066+0000 7f602beb8640 1 --2- 172.21.15.71:0/762284245 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f60241020a0 0x7f6024108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.066+0000 7f602beb8640 1 --2- 172.21.15.71:0/762284245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60241016e0 0x7f6024101b60 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.066+0000 7f602beb8640 1 --2- 172.21.15.71:0/762284245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60241004f0 0x7f6024100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.066+0000 7f602beb8640 1 -- 172.21.15.71:0/762284245 >> 172.21.15.71:0/762284245 conn(0x7f60240fba60 msgr2=0x7f60240fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:48.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.066+0000 7f602beb8640 1 -- 172.21.15.71:0/762284245 shutdown_connections 2024-03-04T08:58:48.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.066+0000 7f602beb8640 1 -- 172.21.15.71:0/762284245 wait complete. 2024-03-04T08:58:48.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.067+0000 7f602beb8640 1 Processor -- start 2024-03-04T08:58:48.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.067+0000 7f602beb8640 1 -- start start 2024-03-04T08:58:48.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.067+0000 7f602beb8640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f60241004f0 0x7f6024196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:48.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.067+0000 7f602beb8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60241016e0 0x7f6024196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:48.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.067+0000 7f602beb8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60241020a0 0x7f602419dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:48.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.067+0000 7f602beb8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602410c180 con 0x7f60241020a0 2024-03-04T08:58:48.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.067+0000 7f602beb8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602410c000 con 0x7f60241016e0 2024-03-04T08:58:48.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.067+0000 7f602beb8640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602410c300 con 0x7f60241004f0 2024-03-04T08:58:48.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.067+0000 7f602a455640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60241020a0 0x7f602419dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:48.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.068+0000 7f602a455640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60241020a0 0x7f602419dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37138/0 (socket says 172.21.15.71:37138) 2024-03-04T08:58:48.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.068+0000 7f602a455640 1 -- 172.21.15.71:0/1604116068 learned_addr learned my addr 172.21.15.71:0/1604116068 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:48.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.068+0000 7f6029c54640 1 --2- 172.21.15.71:0/1604116068 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f60241004f0 0x7f6024196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:48.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.068+0000 7f6029453640 1 --2- 172.21.15.71:0/1604116068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60241016e0 0x7f6024196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:48.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.068+0000 7f602a455640 1 -- 172.21.15.71:0/1604116068 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f60241004f0 msgr2=0x7f6024196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:48.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.068+0000 7f602a455640 1 --2- 172.21.15.71:0/1604116068 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f60241004f0 0x7f6024196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.068+0000 7f602a455640 1 -- 172.21.15.71:0/1604116068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60241016e0 msgr2=0x7f6024196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:48.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.068+0000 7f602a455640 1 --2- 172.21.15.71:0/1604116068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60241016e0 0x7f6024196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.068+0000 7f602a455640 1 -- 172.21.15.71:0/1604116068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f602419e370 con 0x7f60241020a0 2024-03-04T08:58:48.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.068+0000 7f602a455640 1 --2- 172.21.15.71:0/1604116068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60241020a0 0x7f602419dd00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f602000eb10 tx=0x7f602000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:48.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.068+0000 7f601affd640 1 -- 172.21.15.71:0/1604116068 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020004b20 con 0x7f60241020a0 2024-03-04T08:58:48.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.068+0000 7f601affd640 1 -- 172.21.15.71:0/1604116068 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6020004cc0 con 0x7f60241020a0 2024-03-04T08:58:48.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.068+0000 7f602beb8640 1 -- 172.21.15.71:0/1604116068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f602419e660 con 0x7f60241020a0 2024-03-04T08:58:48.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.068+0000 7f601affd640 1 -- 172.21.15.71:0/1604116068 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020010760 con 0x7f60241020a0 2024-03-04T08:58:48.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.068+0000 7f602beb8640 1 -- 172.21.15.71:0/1604116068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f602419ebd0 con 0x7f60241020a0 2024-03-04T08:58:48.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.069+0000 7f602beb8640 1 -- 172.21.15.71:0/1604116068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60241077b0 con 0x7f60241020a0 2024-03-04T08:58:48.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.070+0000 7f601affd640 1 -- 172.21.15.71:0/1604116068 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 136773+0+0 (secure 0 0 0) 0x7f6020005020 con 0x7f60241020a0 2024-03-04T08:58:48.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.071+0000 7f601affd640 1 --2- 172.21.15.71:0/1604116068 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f60100a1ef0 0x7f60100a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:48.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.071+0000 7f6029c54640 1 -- 172.21.15.71:0/1604116068 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f60100a1ef0 msgr2=0x7f60100a43b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/2901868399 2024-03-04T08:58:48.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.071+0000 7f601affd640 1 -- 172.21.15.71:0/1604116068 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f60200ce840 con 0x7f60241020a0 2024-03-04T08:58:48.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.071+0000 7f6029c54640 1 --2- 172.21.15.71:0/1604116068 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f60100a1ef0 0x7f60100a43b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:58:48.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.075+0000 7f601affd640 1 -- 172.21.15.71:0/1604116068 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f602009b890 con 0x7f60241020a0 2024-03-04T08:58:48.215 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:47 smithi071 ceph-mon[32789]: pgmap v397: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:58:48.216 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:47 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3368966478' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-04T08:58:48.216 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:47 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: ignoring --setuser ceph since I am not root 2024-03-04T08:58:48.216 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:47 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: ignoring --setgroup ceph since I am not root 2024-03-04T08:58:48.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.259+0000 7f602beb8640 1 -- 172.21.15.71:0/1604116068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f602402cff0 con 0x7f60241020a0 2024-03-04T08:58:48.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.264+0000 7f601affd640 1 -- 172.21.15.71:0/1604116068 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v26)=0 v26) v1 ==== 138+0+0 (secure 0 0 0) 0x7f602009b190 con 0x7f60241020a0 2024-03-04T08:58:48.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.266+0000 7f602beb8640 1 -- 172.21.15.71:0/1604116068 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f60100a1ef0 msgr2=0x7f60100a43b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T08:58:48.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.266+0000 7f602beb8640 1 --2- 172.21.15.71:0/1604116068 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f60100a1ef0 0x7f60100a43b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.266+0000 7f602beb8640 1 -- 172.21.15.71:0/1604116068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60241020a0 msgr2=0x7f602419dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:48.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.266+0000 7f602beb8640 1 --2- 172.21.15.71:0/1604116068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60241020a0 0x7f602419dd00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f602000eb10 tx=0x7f602000c2d0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.266+0000 7f602beb8640 1 -- 172.21.15.71:0/1604116068 shutdown_connections 2024-03-04T08:58:48.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.266+0000 7f602beb8640 1 --2- 172.21.15.71:0/1604116068 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f60100a1ef0 0x7f60100a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.266+0000 7f602beb8640 1 --2- 172.21.15.71:0/1604116068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60241020a0 0x7f602419dd00 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.266+0000 7f602beb8640 1 --2- 172.21.15.71:0/1604116068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60241016e0 0x7f6024196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.266+0000 7f602beb8640 1 --2- 172.21.15.71:0/1604116068 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f60241004f0 0x7f6024196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.266+0000 7f602beb8640 1 -- 172.21.15.71:0/1604116068 >> 172.21.15.71:0/1604116068 conn(0x7f60240fba60 msgr2=0x7f6024104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:48.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.266+0000 7f602beb8640 1 -- 172.21.15.71:0/1604116068 shutdown_connections 2024-03-04T08:58:48.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.266+0000 7f602beb8640 1 -- 172.21.15.71:0/1604116068 wait complete. 2024-03-04T08:58:48.273 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-04T08:58:48.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:47 smithi150 ceph-mon[36807]: pgmap v397: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:58:48.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:47 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3368966478' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-04T08:58:48.280 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:47 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: ignoring --setuser ceph since I am not root 2024-03-04T08:58:48.280 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:47 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: ignoring --setgroup ceph since I am not root 2024-03-04T08:58:48.280 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:47 smithi150 ceph-mgr[38200]: -- 172.21.15.150:0/1410425503 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55774eb5c5a0 con 0x55774eb30c00 2024-03-04T08:58:48.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:47 smithi158 ceph-mon[37241]: pgmap v397: 129 pgs: 129 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:58:48.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:47 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3368966478' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-04T08:58:48.302 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:47 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: ignoring --setuser ceph since I am not root 2024-03-04T08:58:48.302 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:47 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: ignoring --setgroup ceph since I am not root 2024-03-04T08:58:48.302 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:47 smithi158 ceph-mgr[39112]: -- 172.21.15.158:0/4219448083 <== mon.1 v2:172.21.15.158:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557bee87e5a0 con 0x557bee853000 2024-03-04T08:58:48.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.381+0000 7f351c8a4640 1 -- 172.21.15.71:0/3554499130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35181016e0 msgr2=0x7f3518101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:48.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.381+0000 7f351c8a4640 1 --2- 172.21.15.71:0/3554499130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35181016e0 0x7f3518101b60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3508009900 tx=0x7f350802bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:48.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.382+0000 7f351c8a4640 1 -- 172.21.15.71:0/3554499130 shutdown_connections 2024-03-04T08:58:48.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.382+0000 7f351c8a4640 1 --2- 172.21.15.71:0/3554499130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35181020a0 0x7f3518108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.382+0000 7f351c8a4640 1 --2- 172.21.15.71:0/3554499130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35181016e0 0x7f3518101b60 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.382+0000 7f351c8a4640 1 --2- 172.21.15.71:0/3554499130 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f35181004f0 0x7f3518100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.382+0000 7f351c8a4640 1 -- 172.21.15.71:0/3554499130 >> 172.21.15.71:0/3554499130 conn(0x7f35180fba60 msgr2=0x7f35180fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:48.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.382+0000 7f351c8a4640 1 -- 172.21.15.71:0/3554499130 shutdown_connections 2024-03-04T08:58:48.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.382+0000 7f351c8a4640 1 -- 172.21.15.71:0/3554499130 wait complete. 2024-03-04T08:58:48.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.382+0000 7f351c8a4640 1 Processor -- start 2024-03-04T08:58:48.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.382+0000 7f351c8a4640 1 -- start start 2024-03-04T08:58:48.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.383+0000 7f351c8a4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35181004f0 0x7f35181978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:48.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.383+0000 7f351c8a4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35181016e0 0x7f3518197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:48.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.383+0000 7f351c8a4640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f35181020a0 0x7f351819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:48.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.383+0000 7f351c8a4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351810c180 con 0x7f35181016e0 2024-03-04T08:58:48.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.383+0000 7f351c8a4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351810c000 con 0x7f35181004f0 2024-03-04T08:58:48.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.383+0000 7f351c8a4640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351810c300 con 0x7f35181020a0 2024-03-04T08:58:48.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.383+0000 7f3515d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35181016e0 0x7f3518197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:48.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.383+0000 7f3515d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35181016e0 0x7f3518197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37150/0 (socket says 172.21.15.71:37150) 2024-03-04T08:58:48.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.383+0000 7f3515d9b640 1 -- 172.21.15.71:0/2788188466 learned_addr learned my addr 172.21.15.71:0/2788188466 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:48.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.383+0000 7f351659c640 1 --2- 172.21.15.71:0/2788188466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35181004f0 0x7f35181978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:48.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.383+0000 7f3516d9d640 1 --2- 172.21.15.71:0/2788188466 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f35181020a0 0x7f351819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:48.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.383+0000 7f3515d9b640 1 -- 172.21.15.71:0/2788188466 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f35181020a0 msgr2=0x7f351819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:48.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.383+0000 7f3515d9b640 1 --2- 172.21.15.71:0/2788188466 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f35181020a0 0x7f351819ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.383+0000 7f3515d9b640 1 -- 172.21.15.71:0/2788188466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35181004f0 msgr2=0x7f35181978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:48.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.383+0000 7f3515d9b640 1 --2- 172.21.15.71:0/2788188466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35181004f0 0x7f35181978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.383+0000 7f3515d9b640 1 -- 172.21.15.71:0/2788188466 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351819f4c0 con 0x7f35181016e0 2024-03-04T08:58:48.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.383+0000 7f3515d9b640 1 --2- 172.21.15.71:0/2788188466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35181016e0 0x7f3518197de0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3508009230 tx=0x7f350802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:48.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.383+0000 7f35077fe640 1 -- 172.21.15.71:0/2788188466 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508041070 con 0x7f35181016e0 2024-03-04T08:58:48.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.383+0000 7f35077fe640 1 -- 172.21.15.71:0/2788188466 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3508004450 con 0x7f35181016e0 2024-03-04T08:58:48.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.383+0000 7f351c8a4640 1 -- 172.21.15.71:0/2788188466 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f351819f750 con 0x7f35181016e0 2024-03-04T08:58:48.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.383+0000 7f35077fe640 1 -- 172.21.15.71:0/2788188466 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508003e20 con 0x7f35181016e0 2024-03-04T08:58:48.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.383+0000 7f351c8a4640 1 -- 172.21.15.71:0/2788188466 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f351819fc40 con 0x7f35181016e0 2024-03-04T08:58:48.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.384+0000 7f351c8a4640 1 -- 172.21.15.71:0/2788188466 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f351819fea0 con 0x7f35181016e0 2024-03-04T08:58:48.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.385+0000 7f35077fe640 1 -- 172.21.15.71:0/2788188466 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 136773+0+0 (secure 0 0 0) 0x7f3508044070 con 0x7f35181016e0 2024-03-04T08:58:48.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.386+0000 7f35077fe640 1 --2- 172.21.15.71:0/2788188466 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f35000a1e70 0x7f35000a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:48.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.386+0000 7f351659c640 1 -- 172.21.15.71:0/2788188466 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f35000a1e70 msgr2=0x7f35000a4330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/2901868399 2024-03-04T08:58:48.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.386+0000 7f35077fe640 1 -- 172.21.15.71:0/2788188466 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f35080efd60 con 0x7f35181016e0 2024-03-04T08:58:48.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.386+0000 7f351659c640 1 --2- 172.21.15.71:0/2788188466 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f35000a1e70 0x7f35000a4330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:58:48.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.390+0000 7f35077fe640 1 -- 172.21.15.71:0/2788188466 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f35080bc760 con 0x7f35181016e0 2024-03-04T08:58:48.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.573+0000 7f351c8a4640 1 -- 172.21.15.71:0/2788188466 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f35181a0070 con 0x7f35181016e0 2024-03-04T08:58:48.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.577+0000 7f35077fe640 1 -- 172.21.15.71:0/2788188466 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v27)=0 v27) v1 ==== 133+0+0 (secure 0 0 0) 0x7f35080dae20 con 0x7f35181016e0 2024-03-04T08:58:48.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.579+0000 7f351c8a4640 1 -- 172.21.15.71:0/2788188466 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f35000a1e70 msgr2=0x7f35000a4330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T08:58:48.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.579+0000 7f351c8a4640 1 --2- 172.21.15.71:0/2788188466 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f35000a1e70 0x7f35000a4330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.579+0000 7f351c8a4640 1 -- 172.21.15.71:0/2788188466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35181016e0 msgr2=0x7f3518197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:48.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.579+0000 7f351c8a4640 1 --2- 172.21.15.71:0/2788188466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35181016e0 0x7f3518197de0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3508009230 tx=0x7f350802c650 comp rx=0 tx=0).stop 2024-03-04T08:58:48.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.579+0000 7f351c8a4640 1 -- 172.21.15.71:0/2788188466 shutdown_connections 2024-03-04T08:58:48.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.579+0000 7f351c8a4640 1 --2- 172.21.15.71:0/2788188466 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f35000a1e70 0x7f35000a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.579+0000 7f351c8a4640 1 --2- 172.21.15.71:0/2788188466 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f35181020a0 0x7f351819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.579+0000 7f351c8a4640 1 --2- 172.21.15.71:0/2788188466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35181016e0 0x7f3518197de0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.579+0000 7f351c8a4640 1 --2- 172.21.15.71:0/2788188466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35181004f0 0x7f35181978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.579+0000 7f351c8a4640 1 -- 172.21.15.71:0/2788188466 >> 172.21.15.71:0/2788188466 conn(0x7f35180fba60 msgr2=0x7f35180fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:48.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.579+0000 7f351c8a4640 1 -- 172.21.15.71:0/2788188466 shutdown_connections 2024-03-04T08:58:48.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.579+0000 7f351c8a4640 1 -- 172.21.15.71:0/2788188466 wait complete. 2024-03-04T08:58:48.586 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-04T08:58:48.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.696+0000 7f3d39df6640 1 -- 172.21.15.71:0/3550010671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d34101730 msgr2=0x7f3d34101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:48.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.696+0000 7f3d39df6640 1 --2- 172.21.15.71:0/3550010671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d34101730 0x7f3d34101bb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3d28009900 tx=0x7f3d2802bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:48.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.696+0000 7f3d39df6640 1 -- 172.21.15.71:0/3550010671 shutdown_connections 2024-03-04T08:58:48.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.696+0000 7f3d39df6640 1 --2- 172.21.15.71:0/3550010671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d341020f0 0x7f3d34108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.696+0000 7f3d39df6640 1 --2- 172.21.15.71:0/3550010671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d34101730 0x7f3d34101bb0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.696+0000 7f3d39df6640 1 --2- 172.21.15.71:0/3550010671 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d34100540 0x7f3d34100960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.696+0000 7f3d39df6640 1 -- 172.21.15.71:0/3550010671 >> 172.21.15.71:0/3550010671 conn(0x7f3d340fbab0 msgr2=0x7f3d340fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:48.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.697+0000 7f3d39df6640 1 -- 172.21.15.71:0/3550010671 shutdown_connections 2024-03-04T08:58:48.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.697+0000 7f3d39df6640 1 -- 172.21.15.71:0/3550010671 wait complete. 2024-03-04T08:58:48.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.697+0000 7f3d39df6640 1 Processor -- start 2024-03-04T08:58:48.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.697+0000 7f3d39df6640 1 -- start start 2024-03-04T08:58:48.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.697+0000 7f3d39df6640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d34100540 0x7f3d3406a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:48.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.697+0000 7f3d39df6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d34101730 0x7f3d3419c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:48.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.697+0000 7f3d39df6640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d341020f0 0x7f3d3419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:48.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.697+0000 7f3d39df6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3410c1d0 con 0x7f3d34101730 2024-03-04T08:58:48.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.697+0000 7f3d39df6640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3410c050 con 0x7f3d34100540 2024-03-04T08:58:48.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.697+0000 7f3d39df6640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3410c350 con 0x7f3d341020f0 2024-03-04T08:58:48.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.698+0000 7f3d32ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d34101730 0x7f3d3419c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:48.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.698+0000 7f3d32ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d34101730 0x7f3d3419c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37162/0 (socket says 172.21.15.71:37162) 2024-03-04T08:58:48.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.698+0000 7f3d32ffd640 1 -- 172.21.15.71:0/3794766701 learned_addr learned my addr 172.21.15.71:0/3794766701 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:48.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.698+0000 7f3d337fe640 1 --2- 172.21.15.71:0/3794766701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d34100540 0x7f3d3406a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:48.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.698+0000 7f3d33fff640 1 --2- 172.21.15.71:0/3794766701 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d341020f0 0x7f3d3419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:48.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.698+0000 7f3d32ffd640 1 -- 172.21.15.71:0/3794766701 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d341020f0 msgr2=0x7f3d3419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:48.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.698+0000 7f3d32ffd640 1 --2- 172.21.15.71:0/3794766701 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3d341020f0 0x7f3d3419ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.698+0000 7f3d32ffd640 1 -- 172.21.15.71:0/3794766701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d34100540 msgr2=0x7f3d3406a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:48.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.698+0000 7f3d32ffd640 1 --2- 172.21.15.71:0/3794766701 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d34100540 0x7f3d3406a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:48.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.698+0000 7f3d32ffd640 1 -- 172.21.15.71:0/3794766701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3419f4f0 con 0x7f3d34101730 2024-03-04T08:58:48.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.698+0000 7f3d32ffd640 1 --2- 172.21.15.71:0/3794766701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d34101730 0x7f3d3419c5c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3d28009230 tx=0x7f3d2802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:48.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.698+0000 7f3d30ff9640 1 -- 172.21.15.71:0/3794766701 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d28041070 con 0x7f3d34101730 2024-03-04T08:58:48.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.698+0000 7f3d39df6640 1 -- 172.21.15.71:0/3794766701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d3419f720 con 0x7f3d34101730 2024-03-04T08:58:48.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.698+0000 7f3d30ff9640 1 -- 172.21.15.71:0/3794766701 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3d28004450 con 0x7f3d34101730 2024-03-04T08:58:48.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.698+0000 7f3d39df6640 1 -- 172.21.15.71:0/3794766701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d3419fc60 con 0x7f3d34101730 2024-03-04T08:58:48.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.698+0000 7f3d30ff9640 1 -- 172.21.15.71:0/3794766701 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d28003e20 con 0x7f3d34101730 2024-03-04T08:58:48.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.699+0000 7f3d39df6640 1 -- 172.21.15.71:0/3794766701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d34107790 con 0x7f3d34101730 2024-03-04T08:58:48.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.701+0000 7f3d30ff9640 1 -- 172.21.15.71:0/3794766701 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 19) v1 ==== 136773+0+0 (secure 0 0 0) 0x7f3d28044070 con 0x7f3d34101730 2024-03-04T08:58:48.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.702+0000 7f3d30ff9640 1 --2- 172.21.15.71:0/3794766701 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3d1c0a1e70 0x7f3d1c0a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:48.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.702+0000 7f3d337fe640 1 -- 172.21.15.71:0/3794766701 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3d1c0a1e70 msgr2=0x7f3d1c0a4330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/2901868399 2024-03-04T08:58:48.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.702+0000 7f3d30ff9640 1 -- 172.21.15.71:0/3794766701 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9121+0+0 (secure 0 0 0) 0x7f3d280efd00 con 0x7f3d34101730 2024-03-04T08:58:48.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.702+0000 7f3d337fe640 1 --2- 172.21.15.71:0/3794766701 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3d1c0a1e70 0x7f3d1c0a4330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:58:48.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.706+0000 7f3d30ff9640 1 -- 172.21.15.71:0/3794766701 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3d280bc760 con 0x7f3d34101730 2024-03-04T08:58:48.779 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:48 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:48.506+0000 7f00a6f04200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T08:58:48.779 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:48 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:48.602+0000 7f00a6f04200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T08:58:48.802 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:48 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:48.511+0000 7ff835e28200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T08:58:48.802 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:48 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:48.606+0000 7ff835e28200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T08:58:48.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.903+0000 7f3d337fe640 1 -- 172.21.15.71:0/3794766701 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3d1c0a1e70 msgr2=0x7f3d1c0a4330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/2901868399 2024-03-04T08:58:48.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.903+0000 7f3d337fe640 1 --2- 172.21.15.71:0/3794766701 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3d1c0a1e70 0x7f3d1c0a4330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:58:48.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:48.909+0000 7f3d39df6640 1 -- 172.21.15.71:0/3794766701 --> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d34061a10 con 0x7f3d1c0a1e70 2024-03-04T08:58:48.921 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:48 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:48.538+0000 7fa74776f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T08:58:48.921 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:48 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:48.639+0000 7fa74776f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T08:58:49.242 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:48 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3368966478' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-04T08:58:49.242 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:48 smithi150 ceph-mon[36807]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-03-04T08:58:49.242 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:48 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1604116068' entity='client.admin' 2024-03-04T08:58:49.242 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:48 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2788188466' entity='client.admin' 2024-03-04T08:58:49.243 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:48 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3368966478' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-04T08:58:49.243 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:48 smithi158 ceph-mon[37241]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-03-04T08:58:49.243 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:48 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1604116068' entity='client.admin' 2024-03-04T08:58:49.243 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:48 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2788188466' entity='client.admin' 2024-03-04T08:58:49.281 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:48 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3368966478' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-04T08:58:49.282 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:48 smithi071 ceph-mon[32789]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-03-04T08:58:49.282 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:48 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1604116068' entity='client.admin' 2024-03-04T08:58:49.282 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:48 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2788188466' entity='client.admin' 2024-03-04T08:58:49.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:49.304+0000 7f3d337fe640 1 -- 172.21.15.71:0/3794766701 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3d1c0a1e70 msgr2=0x7f3d1c0a4330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/2901868399 2024-03-04T08:58:49.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:49.304+0000 7f3d337fe640 1 --2- 172.21.15.71:0/3794766701 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3d1c0a1e70 0x7f3d1c0a4330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:58:49.496 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:49 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:49.245+0000 7f00a6f04200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T08:58:49.496 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:49 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:49.339+0000 7f00a6f04200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T08:58:49.552 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:49 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:49.247+0000 7ff835e28200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T08:58:49.552 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:49 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:49.340+0000 7ff835e28200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T08:58:49.552 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:49 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:49.481+0000 7ff835e28200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T08:58:49.611 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:49 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:49.285+0000 7fa74776f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T08:58:49.611 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:49 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:49.378+0000 7fa74776f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T08:58:49.611 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:49 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:49.520+0000 7fa74776f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T08:58:49.779 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:49 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:49.500+0000 7f00a6f04200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T08:58:49.780 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:49 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:49.597+0000 7f00a6f04200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T08:58:49.780 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:49 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:49.680+0000 7f00a6f04200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T08:58:49.920 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:49 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:49.577+0000 7ff835e28200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T08:58:49.920 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:49 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:49.660+0000 7ff835e28200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T08:58:49.920 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:49 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:49.924+0000 7ff835e28200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-04T08:58:49.921 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:49 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:49.615+0000 7fa74776f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T08:58:49.921 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:49 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:49.698+0000 7fa74776f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T08:58:50.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:50.105+0000 7f3d337fe640 1 -- 172.21.15.71:0/3794766701 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3d1c0a1e70 msgr2=0x7f3d1c0a4330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/2901868399 2024-03-04T08:58:50.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:50.105+0000 7f3d337fe640 1 --2- 172.21.15.71:0/3794766701 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3d1c0a1e70 0x7f3d1c0a4330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:58:50.212 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:50 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:50.033+0000 7ff835e28200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T08:58:50.212 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:50 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:50.120+0000 7ff835e28200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T08:58:50.212 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:50 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:50.215+0000 7ff835e28200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T08:58:50.233 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:49 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:49.945+0000 7f00a6f04200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-04T08:58:50.233 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:50 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:50.055+0000 7f00a6f04200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T08:58:50.233 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:50 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:50.142+0000 7f00a6f04200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T08:58:50.233 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:50 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:50.237+0000 7f00a6f04200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T08:58:50.245 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:49 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:49.959+0000 7fa74776f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-04T08:58:50.245 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:50 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:50.068+0000 7fa74776f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T08:58:50.245 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:50 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:50.155+0000 7fa74776f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T08:58:50.245 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:50 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:50.249+0000 7fa74776f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T08:58:50.529 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:50 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:50.415+0000 7f00a6f04200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T08:58:50.552 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:50 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:50.391+0000 7ff835e28200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T08:58:50.671 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:50 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:50.425+0000 7fa74776f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T08:58:51.671 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:51 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:51.383+0000 7fa74776f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-04T08:58:51.672 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:51 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:51.512+0000 7fa74776f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T08:58:51.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:51.706+0000 7f3d337fe640 1 -- 172.21.15.71:0/3794766701 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3d1c0a1e70 msgr2=0x7f3d1c0a4330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/2901868399 2024-03-04T08:58:51.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:51.706+0000 7f3d337fe640 1 --2- 172.21.15.71:0/3794766701 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3d1c0a1e70 0x7f3d1c0a4330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-04T08:58:51.779 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:51 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:51.483+0000 7f00a6f04200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-04T08:58:51.779 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:51 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:51.615+0000 7f00a6f04200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T08:58:51.802 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:51 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:51.358+0000 7ff835e28200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-04T08:58:51.802 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:51 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:51.488+0000 7ff835e28200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T08:58:52.106 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:51 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:51.825+0000 7ff835e28200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T08:58:52.106 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:51 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:51.914+0000 7ff835e28200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T08:58:52.107 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:52 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:52.110+0000 7ff835e28200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T08:58:52.130 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:51 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:51.846+0000 7fa74776f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T08:58:52.130 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:51 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:51.936+0000 7fa74776f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T08:58:52.130 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:52 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:52.133+0000 7fa74776f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T08:58:52.243 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:51 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:51.956+0000 7f00a6f04200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T08:58:52.243 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:52 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:52.048+0000 7f00a6f04200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T08:58:52.243 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:52 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:52.247+0000 7f00a6f04200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T08:58:52.421 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:52 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:52.321+0000 7fa74776f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T08:58:52.529 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:52 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:52.436+0000 7f00a6f04200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T08:58:52.552 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:52 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:52.296+0000 7ff835e28200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T08:58:52.838 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:52 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:52.581+0000 7ff835e28200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T08:58:52.839 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:52 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:52.842+0000 7ff835e28200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T08:58:52.862 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:52 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:52.608+0000 7fa74776f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T08:58:52.988 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:52 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:52.724+0000 7f00a6f04200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T08:58:53.171 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:52 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:52.866+0000 7fa74776f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T08:58:53.171 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:53 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:53.103+0000 7fa74776f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T08:58:53.279 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:52 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:52.992+0000 7f00a6f04200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T08:58:53.279 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:53 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:53.231+0000 7f00a6f04200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T08:58:53.302 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:53 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:53.078+0000 7ff835e28200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T08:58:54.171 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:53 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:53.912+0000 7fa74776f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T08:58:54.302 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:53 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:53.883+0000 7ff835e28200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T08:58:54.529 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:54 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:54.085+0000 7f00a6f04200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T08:58:54.771 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:54 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:54.456+0000 7ff835e28200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T08:58:54.771 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:54 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:54.556+0000 7ff835e28200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T08:58:54.771 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 08:58:54 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z[39108]: 2024-03-04T08:58:54.775+0000 7ff835e28200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T08:58:54.801 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:54 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:54.481+0000 7fa74776f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T08:58:54.802 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:54 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:54.583+0000 7fa74776f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T08:58:54.802 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:54 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:54.805+0000 7fa74776f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T08:58:54.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:54.832+0000 7f3d30ff9640 1 -- 172.21.15.71:0/3794766701 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 20) v1 ==== 136441+0+0 (secure 0 0 0) 0x7f3d280035f0 con 0x7f3d34101730 2024-03-04T08:58:54.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:54.832+0000 7f3d30ff9640 1 -- 172.21.15.71:0/3794766701 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3d1c0a1e70 msgr2=0x7f3d1c0a4330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T08:58:54.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:54.832+0000 7f3d30ff9640 1 --2- 172.21.15.71:0/3794766701 >> [v2:172.21.15.71:6800/2901868399,v1:172.21.15.71:6801/2901868399] conn(0x7f3d1c0a1e70 0x7f3d1c0a4330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:54.993 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:54 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:54.693+0000 7f00a6f04200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T08:58:54.993 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:54 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:54.818+0000 7f00a6f04200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T08:58:54.993 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:54 smithi150 ceph-mon[36807]: Standby manager daemon z restarted 2024-03-04T08:58:54.993 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:54 smithi150 ceph-mon[36807]: Standby manager daemon z started 2024-03-04T08:58:54.993 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:54 smithi150 ceph-mon[36807]: from='mgr.? 172.21.15.158:0/1553798247' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-04T08:58:54.993 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:54 smithi150 ceph-mon[36807]: from='mgr.? 172.21.15.158:0/1553798247' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T08:58:54.993 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:54 smithi150 ceph-mon[36807]: from='mgr.? 172.21.15.158:0/1553798247' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-04T08:58:54.993 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:54 smithi150 ceph-mon[36807]: from='mgr.? 172.21.15.158:0/1553798247' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T08:58:54.994 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:54 smithi150 ceph-mon[36807]: Active manager daemon x restarted 2024-03-04T08:58:54.994 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:54 smithi150 ceph-mon[36807]: Activating manager daemon x 2024-03-04T08:58:54.994 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:54 smithi150 ceph-mon[36807]: osdmap e96: 12 total, 12 up, 12 in 2024-03-04T08:58:54.994 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:54 smithi150 ceph-mon[36807]: mgrmap e20: x(active, starting, since 0.0142931s), standbys: y, z 2024-03-04T08:58:54.994 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:54 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T08:58:54.994 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:54 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:58:54.994 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:54 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:58:54.994 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:54 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-04T08:58:55.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:54 smithi158 ceph-mon[37241]: Standby manager daemon z restarted 2024-03-04T08:58:55.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:54 smithi158 ceph-mon[37241]: Standby manager daemon z started 2024-03-04T08:58:55.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:54 smithi158 ceph-mon[37241]: from='mgr.? 172.21.15.158:0/1553798247' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-04T08:58:55.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:54 smithi158 ceph-mon[37241]: from='mgr.? 172.21.15.158:0/1553798247' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T08:58:55.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:54 smithi158 ceph-mon[37241]: from='mgr.? 172.21.15.158:0/1553798247' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-04T08:58:55.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:54 smithi158 ceph-mon[37241]: from='mgr.? 172.21.15.158:0/1553798247' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T08:58:55.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:54 smithi158 ceph-mon[37241]: Active manager daemon x restarted 2024-03-04T08:58:55.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:54 smithi158 ceph-mon[37241]: Activating manager daemon x 2024-03-04T08:58:55.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:54 smithi158 ceph-mon[37241]: osdmap e96: 12 total, 12 up, 12 in 2024-03-04T08:58:55.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:54 smithi158 ceph-mon[37241]: mgrmap e20: x(active, starting, since 0.0142931s), standbys: y, z 2024-03-04T08:58:55.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:54 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T08:58:55.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:54 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:58:55.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:54 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:58:55.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:54 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-04T08:58:55.066 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:54 smithi071 ceph-mon[32789]: Standby manager daemon z restarted 2024-03-04T08:58:55.066 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:54 smithi071 ceph-mon[32789]: Standby manager daemon z started 2024-03-04T08:58:55.066 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:54 smithi071 ceph-mon[32789]: from='mgr.? 172.21.15.158:0/1553798247' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-04T08:58:55.066 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:54 smithi071 ceph-mon[32789]: from='mgr.? 172.21.15.158:0/1553798247' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T08:58:55.066 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:54 smithi071 ceph-mon[32789]: from='mgr.? 172.21.15.158:0/1553798247' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-04T08:58:55.066 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:54 smithi071 ceph-mon[32789]: from='mgr.? 172.21.15.158:0/1553798247' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T08:58:55.066 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:54 smithi071 ceph-mon[32789]: Active manager daemon x restarted 2024-03-04T08:58:55.067 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:54 smithi071 ceph-mon[32789]: Activating manager daemon x 2024-03-04T08:58:55.067 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:54 smithi071 ceph-mon[32789]: osdmap e96: 12 total, 12 up, 12 in 2024-03-04T08:58:55.067 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:54 smithi071 ceph-mon[32789]: mgrmap e20: x(active, starting, since 0.0142931s), standbys: y, z 2024-03-04T08:58:55.067 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:54 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T08:58:55.067 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:54 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T08:58:55.067 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:54 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T08:58:55.067 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:54 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-04T08:58:55.279 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y[38196]: 2024-03-04T08:58:55.058+0000 7f00a6f04200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T08:58:55.421 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:55.069+0000 7fa6a9a09700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T08:58:55.422 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:55.069+0000 7fa6a9a09700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T08:58:55.422 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:55.069+0000 7fa6a9a09700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T08:58:55.422 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:55.069+0000 7fa6a9a09700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T08:58:55.422 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:55.069+0000 7fa6a9a09700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T08:58:55.422 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:55.070+0000 7fa6a71c4700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T08:58:55.422 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:55.070+0000 7fa6a71c4700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T08:58:55.422 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:55.070+0000 7fa6a71c4700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T08:58:55.422 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:55.070+0000 7fa6a71c4700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T08:58:55.422 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:58:55.070+0000 7fa6a71c4700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T08:58:55.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:55.858+0000 7f3d30ff9640 1 -- 172.21.15.71:0/3794766701 <== mon.0 v2:172.21.15.71:3300/0 8 ==== mgrmap(e 21) v1 ==== 136568+0+0 (secure 0 0 0) 0x7f3d280b8dd0 con 0x7f3d34101730 2024-03-04T08:58:55.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:55.859+0000 7f3d30ff9640 1 --2- 172.21.15.71:0/3794766701 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f3d1c050950 0x7f3d1c050d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:55.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:55.859+0000 7f3d30ff9640 1 -- 172.21.15.71:0/3794766701 --> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d280a5a70 con 0x7f3d1c050950 2024-03-04T08:58:55.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:55.860+0000 7f3d337fe640 1 --2- 172.21.15.71:0/3794766701 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f3d1c050950 0x7f3d1c050d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:55.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:55.861+0000 7f3d337fe640 1 --2- 172.21.15.71:0/3794766701 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f3d1c050950 0x7f3d1c050d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3d34101590 tx=0x7f3d24005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:55.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:55.923+0000 7f3d30ff9640 1 -- 172.21.15.71:0/3794766701 <== mgr.15399 v2:172.21.15.71:6800/217968316 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3d280a5a70 con 0x7f3d1c050950 2024-03-04T08:58:55.919 INFO:teuthology.orchestra.run.smithi071.stdout:Schedule set for path / 2024-03-04T08:58:55.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:55.925+0000 7f3d39df6640 1 -- 172.21.15.71:0/3794766701 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f3d1c050950 msgr2=0x7f3d1c050d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:55.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:55.925+0000 7f3d39df6640 1 --2- 172.21.15.71:0/3794766701 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f3d1c050950 0x7f3d1c050d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3d34101590 tx=0x7f3d24005f90 comp rx=0 tx=0).stop 2024-03-04T08:58:55.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:55.925+0000 7f3d39df6640 1 -- 172.21.15.71:0/3794766701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d34101730 msgr2=0x7f3d3419c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:55.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:55.925+0000 7f3d39df6640 1 --2- 172.21.15.71:0/3794766701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d34101730 0x7f3d3419c5c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3d28009230 tx=0x7f3d2802c650 comp rx=0 tx=0).stop 2024-03-04T08:58:55.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:55.925+0000 7f3d33fff640 1 -- 172.21.15.71:0/3794766701 reap_dead start 2024-03-04T08:58:55.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:55.925+0000 7f3d39df6640 1 -- 172.21.15.71:0/3794766701 shutdown_connections 2024-03-04T08:58:55.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:55.925+0000 7f3d39df6640 1 -- 172.21.15.71:0/3794766701 >> 172.21.15.71:0/3794766701 conn(0x7f3d340fbab0 msgr2=0x7f3d34104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:55.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:55.925+0000 7f3d39df6640 1 -- 172.21.15.71:0/3794766701 shutdown_connections 2024-03-04T08:58:55.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:55.925+0000 7f3d39df6640 1 -- 172.21.15.71:0/3794766701 wait complete. 2024-03-04T08:58:55.933 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-04T08:58:56.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.046+0000 7f9d01356640 1 -- 172.21.15.71:0/3879309609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cfc101610 msgr2=0x7f9cfc101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:56.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.046+0000 7f9d01356640 1 --2- 172.21.15.71:0/3879309609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cfc101610 0x7f9cfc101a30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009900 tx=0x7f9cec02bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:56.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.046+0000 7f9d01356640 1 -- 172.21.15.71:0/3879309609 shutdown_connections 2024-03-04T08:58:56.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.046+0000 7f9d01356640 1 --2- 172.21.15.71:0/3879309609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cfc1031c0 0x7f9cfc109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:56.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.046+0000 7f9d01356640 1 --2- 172.21.15.71:0/3879309609 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9cfc102800 0x7f9cfc102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:56.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.046+0000 7f9d01356640 1 --2- 172.21.15.71:0/3879309609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cfc101610 0x7f9cfc101a30 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:56.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.046+0000 7f9d01356640 1 -- 172.21.15.71:0/3879309609 >> 172.21.15.71:0/3879309609 conn(0x7f9cfc0fcb60 msgr2=0x7f9cfc0fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:56.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.046+0000 7f9d01356640 1 -- 172.21.15.71:0/3879309609 shutdown_connections 2024-03-04T08:58:56.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.046+0000 7f9d01356640 1 -- 172.21.15.71:0/3879309609 wait complete. 2024-03-04T08:58:56.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.047+0000 7f9d01356640 1 Processor -- start 2024-03-04T08:58:56.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.047+0000 7f9d01356640 1 -- start start 2024-03-04T08:58:56.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.047+0000 7f9d01356640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cfc101610 0x7f9cfc197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:56.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.047+0000 7f9d01356640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cfc102800 0x7f9cfc197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:56.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.047+0000 7f9d01356640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9cfc1031c0 0x7f9cfc19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:56.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.047+0000 7f9d01356640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc10d290 con 0x7f9cfc101610 2024-03-04T08:58:56.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.047+0000 7f9d01356640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc10d110 con 0x7f9cfc102800 2024-03-04T08:58:56.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.047+0000 7f9d01356640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc10d410 con 0x7f9cfc1031c0 2024-03-04T08:58:56.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.047+0000 7f9cfaffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cfc101610 0x7f9cfc197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:56.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.047+0000 7f9cfaffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cfc101610 0x7f9cfc197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36776/0 (socket says 172.21.15.71:36776) 2024-03-04T08:58:56.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.047+0000 7f9cfaffd640 1 -- 172.21.15.71:0/2105204166 learned_addr learned my addr 172.21.15.71:0/2105204166 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:56.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.047+0000 7f9cf3fff640 1 --2- 172.21.15.71:0/2105204166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cfc102800 0x7f9cfc197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:56.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.047+0000 7f9cfb7fe640 1 --2- 172.21.15.71:0/2105204166 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9cfc1031c0 0x7f9cfc19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:56.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.048+0000 7f9cfaffd640 1 -- 172.21.15.71:0/2105204166 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9cfc1031c0 msgr2=0x7f9cfc19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:56.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.048+0000 7f9cfaffd640 1 --2- 172.21.15.71:0/2105204166 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9cfc1031c0 0x7f9cfc19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:56.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.048+0000 7f9cfaffd640 1 -- 172.21.15.71:0/2105204166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cfc102800 msgr2=0x7f9cfc197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:56.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.048+0000 7f9cfaffd640 1 --2- 172.21.15.71:0/2105204166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cfc102800 0x7f9cfc197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:56.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.048+0000 7f9cfaffd640 1 -- 172.21.15.71:0/2105204166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cfc19f520 con 0x7f9cfc101610 2024-03-04T08:58:56.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.048+0000 7f9cfaffd640 1 --2- 172.21.15.71:0/2105204166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cfc101610 0x7f9cfc197900 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9cec000c00 tx=0x7f9cec005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:56.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.048+0000 7f9cf8ff9640 1 -- 172.21.15.71:0/2105204166 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec041070 con 0x7f9cfc101610 2024-03-04T08:58:56.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.048+0000 7f9cf8ff9640 1 -- 172.21.15.71:0/2105204166 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9cec0044d0 con 0x7f9cfc101610 2024-03-04T08:58:56.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.048+0000 7f9cf8ff9640 1 -- 172.21.15.71:0/2105204166 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec034440 con 0x7f9cfc101610 2024-03-04T08:58:56.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.048+0000 7f9d01356640 1 -- 172.21.15.71:0/2105204166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cfc19f7b0 con 0x7f9cfc101610 2024-03-04T08:58:56.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.048+0000 7f9d01356640 1 -- 172.21.15.71:0/2105204166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cfc19fcf0 con 0x7f9cfc101610 2024-03-04T08:58:56.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.049+0000 7f9d01356640 1 -- 172.21.15.71:0/2105204166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cfc108930 con 0x7f9cfc101610 2024-03-04T08:58:56.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.050+0000 7f9cf8ff9640 1 -- 172.21.15.71:0/2105204166 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 21) v1 ==== 136568+0+0 (secure 0 0 0) 0x7f9cec044070 con 0x7f9cfc101610 2024-03-04T08:58:56.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.051+0000 7f9cf8ff9640 1 --2- 172.21.15.71:0/2105204166 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f9ce80a1cb0 0x7f9ce80a4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:56.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.051+0000 7f9cf8ff9640 1 -- 172.21.15.71:0/2105204166 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9379+0+0 (secure 0 0 0) 0x7f9cec0ef8a0 con 0x7f9cfc101610 2024-03-04T08:58:56.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.051+0000 7f9cf3fff640 1 --2- 172.21.15.71:0/2105204166 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f9ce80a1cb0 0x7f9ce80a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:56.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.051+0000 7f9cf3fff640 1 --2- 172.21.15.71:0/2105204166 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f9ce80a1cb0 0x7f9ce80a4170 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc198e10 tx=0x7f9ce4005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:56.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.055+0000 7f9cf8ff9640 1 -- 172.21.15.71:0/2105204166 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9cec0bc4f0 con 0x7f9cfc101610 2024-03-04T08:58:56.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-04T08:58:56.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-04T08:58:56.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-04T08:58:56.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-04T08:58:56.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-04T08:58:56.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-04T08:58:56.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-04T08:58:56.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-04T08:58:56.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-04T08:58:56.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-04T08:58:56.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-04T08:58:56.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-04T08:58:56.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-04T08:58:56.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-04T08:58:56.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T08:58:56.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T08:58:56.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T08:58:56.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T08:58:56.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T08:58:56.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T08:58:56.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T08:58:56.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T08:58:56.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:58:56.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T08:58:56.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T08:58:56.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T08:58:56.174 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T08:58:56.174 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T08:58:56.174 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T08:58:56.174 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: Manager daemon x is now available 2024-03-04T08:58:56.174 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:58:56.174 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:58:56.174 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-04T08:58:56.174 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-04T08:58:56.174 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: Standby manager daemon y restarted 2024-03-04T08:58:56.174 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: Standby manager daemon y started 2024-03-04T08:58:56.174 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.? 172.21.15.150:0/2250417464' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-04T08:58:56.174 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.? 172.21.15.150:0/2250417464' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T08:58:56.174 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.? 172.21.15.150:0/2250417464' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-04T08:58:56.174 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.? 172.21.15.150:0/2250417464' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T08:58:56.174 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:56.175 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:56.175 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:56.175 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:56.175 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:56.175 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:55 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:56.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.263+0000 7f9d01356640 1 -- 172.21.15.71:0/2105204166 --> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f9cfc061b70 con 0x7f9ce80a1cb0 2024-03-04T08:58:56.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-04T08:58:56.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-04T08:58:56.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-04T08:58:56.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-04T08:58:56.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-04T08:58:56.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-04T08:58:56.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-04T08:58:56.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-04T08:58:56.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-04T08:58:56.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-04T08:58:56.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-04T08:58:56.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-04T08:58:56.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-04T08:58:56.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-04T08:58:56.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T08:58:56.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T08:58:56.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T08:58:56.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T08:58:56.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T08:58:56.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T08:58:56.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T08:58:56.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T08:58:56.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:58:56.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T08:58:56.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T08:58:56.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T08:58:56.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T08:58:56.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T08:58:56.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T08:58:56.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: Manager daemon x is now available 2024-03-04T08:58:56.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:58:56.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:58:56.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-04T08:58:56.282 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-04T08:58:56.283 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: Standby manager daemon y restarted 2024-03-04T08:58:56.283 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: Standby manager daemon y started 2024-03-04T08:58:56.283 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.? 172.21.15.150:0/2250417464' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-04T08:58:56.283 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.? 172.21.15.150:0/2250417464' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T08:58:56.283 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.? 172.21.15.150:0/2250417464' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-04T08:58:56.283 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.? 172.21.15.150:0/2250417464' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T08:58:56.283 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:56.283 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:56.283 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:56.283 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:56.283 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:56.283 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:55 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:56.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-04T08:58:56.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-04T08:58:56.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-04T08:58:56.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-04T08:58:56.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-04T08:58:56.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-04T08:58:56.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-04T08:58:56.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-04T08:58:56.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-04T08:58:56.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-04T08:58:56.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-04T08:58:56.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-04T08:58:56.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-04T08:58:56.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-04T08:58:56.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T08:58:56.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T08:58:56.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T08:58:56.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T08:58:56.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T08:58:56.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T08:58:56.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T08:58:56.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T08:58:56.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T08:58:56.305 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T08:58:56.306 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T08:58:56.306 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T08:58:56.306 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T08:58:56.306 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T08:58:56.306 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T08:58:56.306 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: Manager daemon x is now available 2024-03-04T08:58:56.306 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:58:56.306 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:58:56.306 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-04T08:58:56.306 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-04T08:58:56.306 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: Standby manager daemon y restarted 2024-03-04T08:58:56.306 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: Standby manager daemon y started 2024-03-04T08:58:56.306 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.? 172.21.15.150:0/2250417464' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-04T08:58:56.306 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.? 172.21.15.150:0/2250417464' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T08:58:56.307 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.? 172.21.15.150:0/2250417464' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-04T08:58:56.307 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.? 172.21.15.150:0/2250417464' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T08:58:56.307 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:56.307 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:56.307 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:56.307 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:56.307 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:56.307 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:55 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:56.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.320+0000 7f9cf8ff9640 1 -- 172.21.15.71:0/2105204166 <== mgr.15399 v2:172.21.15.71:6800/217968316 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f9cfc061b70 con 0x7f9ce80a1cb0 2024-03-04T08:58:56.316 INFO:teuthology.orchestra.run.smithi071.stdout:Retention added to path / 2024-03-04T08:58:56.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.322+0000 7f9d01356640 1 -- 172.21.15.71:0/2105204166 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f9ce80a1cb0 msgr2=0x7f9ce80a4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:56.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.322+0000 7f9d01356640 1 --2- 172.21.15.71:0/2105204166 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f9ce80a1cb0 0x7f9ce80a4170 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc198e10 tx=0x7f9ce4005f90 comp rx=0 tx=0).stop 2024-03-04T08:58:56.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.322+0000 7f9d01356640 1 -- 172.21.15.71:0/2105204166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cfc101610 msgr2=0x7f9cfc197900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:56.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.322+0000 7f9d01356640 1 --2- 172.21.15.71:0/2105204166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cfc101610 0x7f9cfc197900 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9cec000c00 tx=0x7f9cec005d50 comp rx=0 tx=0).stop 2024-03-04T08:58:56.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.322+0000 7f9d01356640 1 -- 172.21.15.71:0/2105204166 shutdown_connections 2024-03-04T08:58:56.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.322+0000 7f9d01356640 1 --2- 172.21.15.71:0/2105204166 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f9cfc1031c0 0x7f9cfc19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:56.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.322+0000 7f9d01356640 1 --2- 172.21.15.71:0/2105204166 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cfc102800 0x7f9cfc197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:56.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.322+0000 7f9d01356640 1 --2- 172.21.15.71:0/2105204166 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f9ce80a1cb0 0x7f9ce80a4170 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:56.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.322+0000 7f9d01356640 1 --2- 172.21.15.71:0/2105204166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cfc101610 0x7f9cfc197900 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:56.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.322+0000 7f9d01356640 1 -- 172.21.15.71:0/2105204166 >> 172.21.15.71:0/2105204166 conn(0x7f9cfc0fcb60 msgr2=0x7f9cfc105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:56.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.322+0000 7f9d01356640 1 -- 172.21.15.71:0/2105204166 shutdown_connections 2024-03-04T08:58:56.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.322+0000 7f9d01356640 1 -- 172.21.15.71:0/2105204166 wait complete. 2024-03-04T08:58:56.331 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-04T08:58:56.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.442+0000 7f237cf3a640 1 -- 172.21.15.71:0/908958331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2378100500 msgr2=0x7f2378100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:56.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.442+0000 7f237cf3a640 1 --2- 172.21.15.71:0/908958331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2378100500 0x7f2378100920 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2368009900 tx=0x7f236802bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:56.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.442+0000 7f237cf3a640 1 -- 172.21.15.71:0/908958331 shutdown_connections 2024-03-04T08:58:56.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.442+0000 7f237cf3a640 1 --2- 172.21.15.71:0/908958331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23781020b0 0x7f2378108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:56.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.442+0000 7f237cf3a640 1 --2- 172.21.15.71:0/908958331 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f23781016f0 0x7f2378101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:56.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.442+0000 7f237cf3a640 1 --2- 172.21.15.71:0/908958331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2378100500 0x7f2378100920 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:56.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.442+0000 7f237cf3a640 1 -- 172.21.15.71:0/908958331 >> 172.21.15.71:0/908958331 conn(0x7f23780fba70 msgr2=0x7f23780fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:56.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.442+0000 7f237cf3a640 1 -- 172.21.15.71:0/908958331 shutdown_connections 2024-03-04T08:58:56.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.442+0000 7f237cf3a640 1 -- 172.21.15.71:0/908958331 wait complete. 2024-03-04T08:58:56.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.442+0000 7f237cf3a640 1 Processor -- start 2024-03-04T08:58:56.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.442+0000 7f237cf3a640 1 -- start start 2024-03-04T08:58:56.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.443+0000 7f237cf3a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2378100500 0x7f2378197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:56.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.443+0000 7f237cf3a640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f23781016f0 0x7f2378197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:56.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.443+0000 7f237cf3a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23781020b0 0x7f237819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:56.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.443+0000 7f237cf3a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237810d2a0 con 0x7f2378100500 2024-03-04T08:58:56.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.443+0000 7f237cf3a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237810d120 con 0x7f23781020b0 2024-03-04T08:58:56.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.443+0000 7f237cf3a640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237810d420 con 0x7f23781016f0 2024-03-04T08:58:56.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.443+0000 7f237659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2378100500 0x7f2378197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:56.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.443+0000 7f237659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2378100500 0x7f2378197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36798/0 (socket says 172.21.15.71:36798) 2024-03-04T08:58:56.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.443+0000 7f237659c640 1 -- 172.21.15.71:0/2279431 learned_addr learned my addr 172.21.15.71:0/2279431 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.443+0000 7f2376d9d640 1 --2- 172.21.15.71:0/2279431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23781020b0 0x7f237819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.443+0000 7f236ffff640 1 --2- 172.21.15.71:0/2279431 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f23781016f0 0x7f2378197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.443+0000 7f237659c640 1 -- 172.21.15.71:0/2279431 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f23781016f0 msgr2=0x7f2378197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.443+0000 7f237659c640 1 --2- 172.21.15.71:0/2279431 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f23781016f0 0x7f2378197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.443+0000 7f237659c640 1 -- 172.21.15.71:0/2279431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23781020b0 msgr2=0x7f237819ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.443+0000 7f237659c640 1 --2- 172.21.15.71:0/2279431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23781020b0 0x7f237819ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.443+0000 7f237659c640 1 -- 172.21.15.71:0/2279431 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237819f4b0 con 0x7f2378100500 2024-03-04T08:58:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.443+0000 7f237659c640 1 --2- 172.21.15.71:0/2279431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2378100500 0x7f2378197890 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2368000c00 tx=0x7f2368005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.443+0000 7f236f7fe640 1 -- 172.21.15.71:0/2279431 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2368041070 con 0x7f2378100500 2024-03-04T08:58:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.443+0000 7f236f7fe640 1 -- 172.21.15.71:0/2279431 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23680044d0 con 0x7f2378100500 2024-03-04T08:58:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.443+0000 7f236f7fe640 1 -- 172.21.15.71:0/2279431 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2368034440 con 0x7f2378100500 2024-03-04T08:58:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.443+0000 7f237cf3a640 1 -- 172.21.15.71:0/2279431 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237819f740 con 0x7f2378100500 2024-03-04T08:58:56.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.443+0000 7f237cf3a640 1 -- 172.21.15.71:0/2279431 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237819fc80 con 0x7f2378100500 2024-03-04T08:58:56.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.445+0000 7f237cf3a640 1 -- 172.21.15.71:0/2279431 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23781077c0 con 0x7f2378100500 2024-03-04T08:58:56.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.446+0000 7f236f7fe640 1 -- 172.21.15.71:0/2279431 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 21) v1 ==== 136568+0+0 (secure 0 0 0) 0x7f2368044070 con 0x7f2378100500 2024-03-04T08:58:56.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.446+0000 7f236f7fe640 1 --2- 172.21.15.71:0/2279431 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f23640a1cb0 0x7f23640a4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:56.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.446+0000 7f236ffff640 1 --2- 172.21.15.71:0/2279431 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f23640a1cb0 0x7f23640a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:56.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.446+0000 7f236f7fe640 1 -- 172.21.15.71:0/2279431 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9379+0+0 (secure 0 0 0) 0x7f23680ef890 con 0x7f2378100500 2024-03-04T08:58:56.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.447+0000 7f236ffff640 1 --2- 172.21.15.71:0/2279431 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f23640a1cb0 0x7f23640a4170 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2378198da0 tx=0x7f236000b390 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:56.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.451+0000 7f236f7fe640 1 -- 172.21.15.71:0/2279431 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f23680bc4e0 con 0x7f2378100500 2024-03-04T08:58:56.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.649+0000 7f237cf3a640 1 -- 172.21.15.71:0/2279431 --> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f2378061a10 con 0x7f23640a1cb0 2024-03-04T08:58:56.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.657+0000 7f236f7fe640 1 -- 172.21.15.71:0/2279431 <== mgr.15399 v2:172.21.15.71:6800/217968316 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7f2378061a10 con 0x7f23640a1cb0 2024-03-04T08:58:56.653 INFO:teuthology.orchestra.run.smithi071.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-04T00:00:00", "created": "2024-03-04T08:58:55", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-04T08:58:56.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.659+0000 7f237cf3a640 1 -- 172.21.15.71:0/2279431 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f23640a1cb0 msgr2=0x7f23640a4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:56.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.659+0000 7f237cf3a640 1 --2- 172.21.15.71:0/2279431 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f23640a1cb0 0x7f23640a4170 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2378198da0 tx=0x7f236000b390 comp rx=0 tx=0).stop 2024-03-04T08:58:56.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.659+0000 7f237cf3a640 1 -- 172.21.15.71:0/2279431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2378100500 msgr2=0x7f2378197890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:56.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.659+0000 7f237cf3a640 1 --2- 172.21.15.71:0/2279431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2378100500 0x7f2378197890 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2368000c00 tx=0x7f2368005d50 comp rx=0 tx=0).stop 2024-03-04T08:58:56.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.659+0000 7f237cf3a640 1 -- 172.21.15.71:0/2279431 shutdown_connections 2024-03-04T08:58:56.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.659+0000 7f237cf3a640 1 --2- 172.21.15.71:0/2279431 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23781020b0 0x7f237819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:56.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.659+0000 7f237cf3a640 1 --2- 172.21.15.71:0/2279431 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f23781016f0 0x7f2378197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:56.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.659+0000 7f237cf3a640 1 --2- 172.21.15.71:0/2279431 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f23640a1cb0 0x7f23640a4170 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:56.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.659+0000 7f237cf3a640 1 --2- 172.21.15.71:0/2279431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2378100500 0x7f2378197890 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:56.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.659+0000 7f237cf3a640 1 -- 172.21.15.71:0/2279431 >> 172.21.15.71:0/2279431 conn(0x7f23780fba70 msgr2=0x7f2378104990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:56.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.659+0000 7f237cf3a640 1 -- 172.21.15.71:0/2279431 shutdown_connections 2024-03-04T08:58:56.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.659+0000 7f237cf3a640 1 -- 172.21.15.71:0/2279431 wait complete. 2024-03-04T08:58:56.667 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-04T08:58:56.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.779+0000 7fd0a613a640 1 -- 172.21.15.71:0/1760698473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0a01016f0 msgr2=0x7fd0a0101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:56.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.779+0000 7fd0a613a640 1 --2- 172.21.15.71:0/1760698473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0a01016f0 0x7fd0a0101b70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd088009900 tx=0x7fd08802bc30 comp rx=0 tx=0).stop 2024-03-04T08:58:56.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.779+0000 7fd0a613a640 1 -- 172.21.15.71:0/1760698473 shutdown_connections 2024-03-04T08:58:56.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.779+0000 7fd0a613a640 1 --2- 172.21.15.71:0/1760698473 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd0a01020b0 0x7fd0a0108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:56.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.779+0000 7fd0a613a640 1 --2- 172.21.15.71:0/1760698473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0a01016f0 0x7fd0a0101b70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:56.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.779+0000 7fd0a613a640 1 --2- 172.21.15.71:0/1760698473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0a0100500 0x7fd0a0100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:56.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.779+0000 7fd0a613a640 1 -- 172.21.15.71:0/1760698473 >> 172.21.15.71:0/1760698473 conn(0x7fd0a00fba70 msgr2=0x7fd0a00fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:56.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.779+0000 7fd0a613a640 1 -- 172.21.15.71:0/1760698473 shutdown_connections 2024-03-04T08:58:56.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.779+0000 7fd0a613a640 1 -- 172.21.15.71:0/1760698473 wait complete. 2024-03-04T08:58:56.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.780+0000 7fd0a613a640 1 Processor -- start 2024-03-04T08:58:56.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.780+0000 7fd0a613a640 1 -- start start 2024-03-04T08:58:56.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.780+0000 7fd0a613a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0a0100500 0x7fd0a0104490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:56.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.780+0000 7fd0a613a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0a01016f0 0x7fd0a0102ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:56.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.780+0000 7fd0a613a640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd0a01020b0 0x7fd0a0103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:56.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.780+0000 7fd0a613a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a010c1d0 con 0x7fd0a0100500 2024-03-04T08:58:56.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.780+0000 7fd0a613a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a010c050 con 0x7fd0a01016f0 2024-03-04T08:58:56.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.780+0000 7fd0a613a640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a010c350 con 0x7fd0a01020b0 2024-03-04T08:58:56.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.780+0000 7fd09f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0a0100500 0x7fd0a0104490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:56.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.780+0000 7fd09f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0a0100500 0x7fd0a0104490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36824/0 (socket says 172.21.15.71:36824) 2024-03-04T08:58:56.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.780+0000 7fd09f7fe640 1 -- 172.21.15.71:0/384442775 learned_addr learned my addr 172.21.15.71:0/384442775 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T08:58:56.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.780+0000 7fd096ffd640 1 --2- 172.21.15.71:0/384442775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0a01016f0 0x7fd0a0102ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.780+0000 7fd09ffff640 1 --2- 172.21.15.71:0/384442775 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd0a01020b0 0x7fd0a0103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.780+0000 7fd09f7fe640 1 -- 172.21.15.71:0/384442775 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd0a01020b0 msgr2=0x7fd0a0103010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.780+0000 7fd09f7fe640 1 --2- 172.21.15.71:0/384442775 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd0a01020b0 0x7fd0a0103010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.780+0000 7fd09f7fe640 1 -- 172.21.15.71:0/384442775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0a01016f0 msgr2=0x7fd0a0102ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.780+0000 7fd09f7fe640 1 --2- 172.21.15.71:0/384442775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0a01016f0 0x7fd0a0102ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.780+0000 7fd09f7fe640 1 -- 172.21.15.71:0/384442775 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a01038a0 con 0x7fd0a0100500 2024-03-04T08:58:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.781+0000 7fd09f7fe640 1 --2- 172.21.15.71:0/384442775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0a0100500 0x7fd0a0104490 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd09000e9c0 tx=0x7fd09000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.781+0000 7fd09d7fa640 1 -- 172.21.15.71:0/384442775 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09000cd80 con 0x7fd0a0100500 2024-03-04T08:58:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.781+0000 7fd09d7fa640 1 -- 172.21.15.71:0/384442775 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0900035f0 con 0x7fd0a0100500 2024-03-04T08:58:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.781+0000 7fd0a613a640 1 -- 172.21.15.71:0/384442775 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a01a19d0 con 0x7fd0a0100500 2024-03-04T08:58:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.781+0000 7fd09d7fa640 1 -- 172.21.15.71:0/384442775 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd090010690 con 0x7fd0a0100500 2024-03-04T08:58:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.781+0000 7fd0a613a640 1 -- 172.21.15.71:0/384442775 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a01a1f40 con 0x7fd0a0100500 2024-03-04T08:58:56.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.782+0000 7fd0a613a640 1 -- 172.21.15.71:0/384442775 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a01a21d0 con 0x7fd0a0100500 2024-03-04T08:58:56.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.783+0000 7fd09d7fa640 1 -- 172.21.15.71:0/384442775 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 21) v1 ==== 136568+0+0 (secure 0 0 0) 0x7fd0900037a0 con 0x7fd0a0100500 2024-03-04T08:58:56.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.784+0000 7fd09d7fa640 1 --2- 172.21.15.71:0/384442775 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fd08c0a1cb0 0x7fd08c0a4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:58:56.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.784+0000 7fd096ffd640 1 --2- 172.21.15.71:0/384442775 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fd08c0a1cb0 0x7fd08c0a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:58:56.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.784+0000 7fd09d7fa640 1 -- 172.21.15.71:0/384442775 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9379+0+0 (secure 0 0 0) 0x7fd0900ce240 con 0x7fd0a0100500 2024-03-04T08:58:56.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.784+0000 7fd096ffd640 1 --2- 172.21.15.71:0/384442775 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fd08c0a1cb0 0x7fd08c0a4170 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd08802c7b0 tx=0x7fd088005fd0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:58:56.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.788+0000 7fd09d7fa640 1 -- 172.21.15.71:0/384442775 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd09009b4e0 con 0x7fd0a0100500 2024-03-04T08:58:56.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.863+0000 7fd09d7fa640 1 -- 172.21.15.71:0/384442775 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 22) v1 ==== 136876+0+0 (secure 0 0 0) 0x7fd090097450 con 0x7fd0a0100500 2024-03-04T08:58:56.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:56 smithi071 ceph-mon[32789]: [04/Mar/2024:08:58:55] ENGINE Bus STARTING 2024-03-04T08:58:56.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:56 smithi071 ceph-mon[32789]: [04/Mar/2024:08:58:55] ENGINE Serving on http://172.21.15.71:8765 2024-03-04T08:58:56.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:56 smithi071 ceph-mon[32789]: [04/Mar/2024:08:58:55] ENGINE Serving on https://172.21.15.71:7150 2024-03-04T08:58:56.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:56 smithi071 ceph-mon[32789]: [04/Mar/2024:08:58:55] ENGINE Bus STARTED 2024-03-04T08:58:56.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:56 smithi071 ceph-mon[32789]: mgrmap e21: x(active, since 1.04083s), standbys: z, y 2024-03-04T08:58:56.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:56 smithi071 ceph-mon[32789]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:56.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:56 smithi071 ceph-mon[32789]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:58:56.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:56.998+0000 7fd0a613a640 1 -- 172.21.15.71:0/384442775 --> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd0a00fdb50 con 0x7fd08c0a1cb0 2024-03-04T08:58:57.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:57.004+0000 7fd09d7fa640 1 -- 172.21.15.71:0/384442775 <== mgr.15399 v2:172.21.15.71:6800/217968316 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fd0a00fdb50 con 0x7fd08c0a1cb0 2024-03-04T08:58:57.000 INFO:teuthology.orchestra.run.smithi071.stdout:/ 1m 6m3h 2024-03-04T08:58:57.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:57.006+0000 7fd0a613a640 1 -- 172.21.15.71:0/384442775 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fd08c0a1cb0 msgr2=0x7fd08c0a4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:57.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:57.006+0000 7fd0a613a640 1 --2- 172.21.15.71:0/384442775 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fd08c0a1cb0 0x7fd08c0a4170 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd08802c7b0 tx=0x7fd088005fd0 comp rx=0 tx=0).stop 2024-03-04T08:58:57.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:57.006+0000 7fd0a613a640 1 -- 172.21.15.71:0/384442775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0a0100500 msgr2=0x7fd0a0104490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:58:57.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:57.006+0000 7fd0a613a640 1 --2- 172.21.15.71:0/384442775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0a0100500 0x7fd0a0104490 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd09000e9c0 tx=0x7fd09000ee80 comp rx=0 tx=0).stop 2024-03-04T08:58:57.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:57.006+0000 7fd0a613a640 1 -- 172.21.15.71:0/384442775 shutdown_connections 2024-03-04T08:58:57.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:57.006+0000 7fd0a613a640 1 --2- 172.21.15.71:0/384442775 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fd0a01020b0 0x7fd0a0103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:57.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:57.006+0000 7fd0a613a640 1 --2- 172.21.15.71:0/384442775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd0a01016f0 0x7fd0a0102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:57.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:57.006+0000 7fd0a613a640 1 --2- 172.21.15.71:0/384442775 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fd08c0a1cb0 0x7fd08c0a4170 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:57.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:57.006+0000 7fd0a613a640 1 --2- 172.21.15.71:0/384442775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0a0100500 0x7fd0a0104490 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:58:57.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:57.006+0000 7fd0a613a640 1 -- 172.21.15.71:0/384442775 >> 172.21.15.71:0/384442775 conn(0x7fd0a00fba70 msgr2=0x7fd0a00fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:58:57.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:57.006+0000 7fd0a613a640 1 -- 172.21.15.71:0/384442775 shutdown_connections 2024-03-04T08:58:57.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T08:58:57.006+0000 7fd0a613a640 1 -- 172.21.15.71:0/384442775 wait complete. 2024-03-04T08:58:57.015 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-04T08:58:57.044 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-04T08:58:57.055 INFO:teuthology.run_tasks:Running task workunit... 2024-03-04T08:58:57.098 INFO:tasks.workunit:Pulling workunits from ref 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T08:58:57.099 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-04T08:58:57.099 INFO:tasks.workunit:timeout=3h 2024-03-04T08:58:57.099 INFO:tasks.workunit:cleanup=True 2024-03-04T08:58:57.099 DEBUG:teuthology.orchestra.run.smithi071:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-04T08:58:57.114 INFO:teuthology.orchestra.run.smithi071.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-04T08:58:57.114 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-04T08:58:57.114 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-03-04T08:58:57.114 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T08:58:57.114 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-04T08:58:57.114 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-03-04 08:56:57.114104538 +0000 2024-03-04T08:58:57.114 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-03-04 08:56:57.114104538 +0000 2024-03-04T08:58:57.114 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-03-04 08:58:47.149712618 +0000 2024-03-04T08:58:57.115 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-03-04 08:56:57.114104538 +0000 2024-03-04T08:58:57.115 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-04T08:58:57.115 DEBUG:teuthology.orchestra.run.smithi071:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-04T08:58:57.189 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T08:58:57.245 INFO:tasks.workunit.client.0.smithi071.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-04T08:58:57.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:56 smithi150 ceph-mon[36807]: [04/Mar/2024:08:58:55] ENGINE Bus STARTING 2024-03-04T08:58:57.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:56 smithi150 ceph-mon[36807]: [04/Mar/2024:08:58:55] ENGINE Serving on http://172.21.15.71:8765 2024-03-04T08:58:57.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:56 smithi150 ceph-mon[36807]: [04/Mar/2024:08:58:55] ENGINE Serving on https://172.21.15.71:7150 2024-03-04T08:58:57.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:56 smithi150 ceph-mon[36807]: [04/Mar/2024:08:58:55] ENGINE Bus STARTED 2024-03-04T08:58:57.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:56 smithi150 ceph-mon[36807]: mgrmap e21: x(active, since 1.04083s), standbys: z, y 2024-03-04T08:58:57.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:56 smithi150 ceph-mon[36807]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:57.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:56 smithi150 ceph-mon[36807]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:58:57.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:56 smithi158 ceph-mon[37241]: [04/Mar/2024:08:58:55] ENGINE Bus STARTING 2024-03-04T08:58:57.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:56 smithi158 ceph-mon[37241]: [04/Mar/2024:08:58:55] ENGINE Serving on http://172.21.15.71:8765 2024-03-04T08:58:57.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:56 smithi158 ceph-mon[37241]: [04/Mar/2024:08:58:55] ENGINE Serving on https://172.21.15.71:7150 2024-03-04T08:58:57.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:56 smithi158 ceph-mon[37241]: [04/Mar/2024:08:58:55] ENGINE Bus STARTED 2024-03-04T08:58:57.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:56 smithi158 ceph-mon[37241]: mgrmap e21: x(active, since 1.04083s), standbys: z, y 2024-03-04T08:58:57.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:56 smithi158 ceph-mon[37241]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:57.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:56 smithi158 ceph-mon[37241]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:58:58.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:57 smithi071 ceph-mon[32789]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:58.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:57 smithi071 ceph-mon[32789]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:58.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:57 smithi071 ceph-mon[32789]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:58:58.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:57 smithi071 ceph-mon[32789]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-03-04T08:58:58.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:57 smithi071 ceph-mon[32789]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:58.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:57 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:58.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:57 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:58.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:57 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:58.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:57 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:58.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:57 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:58.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:57 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:58.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:57 smithi150 ceph-mon[36807]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:58.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:57 smithi150 ceph-mon[36807]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:58.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:57 smithi150 ceph-mon[36807]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:58:58.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:57 smithi150 ceph-mon[36807]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-03-04T08:58:58.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:57 smithi150 ceph-mon[36807]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:58.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:57 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:58.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:57 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:58.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:57 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:58.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:57 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:58.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:57 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:58.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:57 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:58.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:57 smithi158 ceph-mon[37241]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:58.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:57 smithi158 ceph-mon[37241]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:58.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:57 smithi158 ceph-mon[37241]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:58:58.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:57 smithi158 ceph-mon[37241]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-03-04T08:58:58.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:57 smithi158 ceph-mon[37241]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T08:58:58.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:57 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:58.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:57 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:58.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:57 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:58.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:57 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:58.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:57 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:58.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:57 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:59.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:59.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:59.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:58:59.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:58:59.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:58:59.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:58:59.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: Adjusting osd_memory_target on smithi158 to 174.1M 2024-03-04T08:58:59.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: Unable to set osd_memory_target on smithi158 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-03-04T08:58:59.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:59.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:59.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:58:59.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:58:59.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:58:59.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:58:59.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: Adjusting osd_memory_target on smithi071 to 176.2M 2024-03-04T08:58:59.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: Unable to set osd_memory_target on smithi071 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-03-04T08:58:59.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:59.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:59.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:58:59.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:58:59.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:58:59.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:58:59.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: Adjusting osd_memory_target on smithi150 to 174.1M 2024-03-04T08:58:59.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: Unable to set osd_memory_target on smithi150 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-03-04T08:58:59.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:58:59.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:58:59.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: Updating smithi071:/etc/ceph/ceph.conf 2024-03-04T08:58:59.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: Updating smithi150:/etc/ceph/ceph.conf 2024-03-04T08:58:59.805 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: Updating smithi158:/etc/ceph/ceph.conf 2024-03-04T08:58:59.805 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:58:59.805 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: Updating smithi071:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T08:58:59.805 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: Updating smithi150:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T08:58:59.805 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:58:59 smithi158 ceph-mon[37241]: Updating smithi158:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T08:58:59.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:59.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:59.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:58:59.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:58:59.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:58:59.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:58:59.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: Adjusting osd_memory_target on smithi158 to 174.1M 2024-03-04T08:58:59.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: Unable to set osd_memory_target on smithi158 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-03-04T08:58:59.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:59.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:59.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:58:59.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:58:59.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:58:59.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:58:59.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: Adjusting osd_memory_target on smithi071 to 176.2M 2024-03-04T08:58:59.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: Unable to set osd_memory_target on smithi071 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-03-04T08:58:59.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:59.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:58:59.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:58:59.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:58:59.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:58:59.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:58:59.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: Adjusting osd_memory_target on smithi150 to 174.1M 2024-03-04T08:58:59.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: Unable to set osd_memory_target on smithi150 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-03-04T08:58:59.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:58:59.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:58:59.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: Updating smithi071:/etc/ceph/ceph.conf 2024-03-04T08:58:59.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: Updating smithi150:/etc/ceph/ceph.conf 2024-03-04T08:58:59.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: Updating smithi158:/etc/ceph/ceph.conf 2024-03-04T08:58:59.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:58:59.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: Updating smithi071:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T08:58:59.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: Updating smithi150:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T08:58:59.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:58:59 smithi071 ceph-mon[32789]: Updating smithi158:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T08:59:00.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:00.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:00.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:59:00.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:59:00.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:59:00.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:59:00.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: Adjusting osd_memory_target on smithi158 to 174.1M 2024-03-04T08:59:00.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: Unable to set osd_memory_target on smithi158 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-03-04T08:59:00.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:00.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:00.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:59:00.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:59:00.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:59:00.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:59:00.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: Adjusting osd_memory_target on smithi071 to 176.2M 2024-03-04T08:59:00.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: Unable to set osd_memory_target on smithi071 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-03-04T08:59:00.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:00.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:00.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:59:00.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:59:00.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:59:00.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-04T08:59:00.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: Adjusting osd_memory_target on smithi150 to 174.1M 2024-03-04T08:59:00.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: Unable to set osd_memory_target on smithi150 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-03-04T08:59:00.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:59:00.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:59:00.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: Updating smithi071:/etc/ceph/ceph.conf 2024-03-04T08:59:00.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: Updating smithi150:/etc/ceph/ceph.conf 2024-03-04T08:59:00.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: Updating smithi158:/etc/ceph/ceph.conf 2024-03-04T08:59:00.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:59:00.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: Updating smithi071:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T08:59:00.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: Updating smithi150:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T08:59:00.032 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:58:59 smithi150 ceph-mon[36807]: Updating smithi158:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T08:59:00.655 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:59:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:59:00.328+0000 7fa687d46700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T08:59:00.655 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:59:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:59:00.328+0000 7fa687d46700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T08:59:00.655 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:59:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:59:00.328+0000 7fa687d46700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T08:59:00.655 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:59:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:59:00.328+0000 7fa687d46700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T08:59:00.655 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 08:59:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T08:59:00.328+0000 7fa687d46700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T08:59:00.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:00 smithi071 ceph-mon[32789]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-03-04T08:59:00.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:00 smithi071 ceph-mon[32789]: Updating smithi158:/etc/ceph/ceph.client.admin.keyring 2024-03-04T08:59:00.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:00 smithi071 ceph-mon[32789]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-03-04T08:59:00.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:00 smithi071 ceph-mon[32789]: Updating smithi071:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T08:59:00.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:00 smithi071 ceph-mon[32789]: Updating smithi158:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T08:59:00.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:00 smithi071 ceph-mon[32789]: Updating smithi150:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T08:59:00.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:00 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:00.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:00 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:00.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:00 smithi071 ceph-mon[32789]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-03-04T08:59:00.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:00 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:00.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:00 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:00.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:00 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:00.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:00 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:00.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:00 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:00.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:00 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:59:00.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:00 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:59:00.923 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:00 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:59:00.924 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:00 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:01.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:00 smithi150 ceph-mon[36807]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-03-04T08:59:01.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:00 smithi150 ceph-mon[36807]: Updating smithi158:/etc/ceph/ceph.client.admin.keyring 2024-03-04T08:59:01.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:00 smithi150 ceph-mon[36807]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-03-04T08:59:01.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:00 smithi150 ceph-mon[36807]: Updating smithi071:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T08:59:01.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:00 smithi150 ceph-mon[36807]: Updating smithi158:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T08:59:01.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:00 smithi150 ceph-mon[36807]: Updating smithi150:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T08:59:01.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:00 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:01.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:00 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:01.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:00 smithi150 ceph-mon[36807]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-03-04T08:59:01.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:00 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:01.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:00 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:01.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:00 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:01.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:00 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:01.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:00 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:01.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:00 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:59:01.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:00 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:59:01.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:00 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:59:01.031 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:00 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:01.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:00 smithi158 ceph-mon[37241]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-03-04T08:59:01.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:00 smithi158 ceph-mon[37241]: Updating smithi158:/etc/ceph/ceph.client.admin.keyring 2024-03-04T08:59:01.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:00 smithi158 ceph-mon[37241]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-03-04T08:59:01.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:00 smithi158 ceph-mon[37241]: Updating smithi071:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T08:59:01.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:00 smithi158 ceph-mon[37241]: Updating smithi158:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T08:59:01.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:00 smithi158 ceph-mon[37241]: Updating smithi150:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T08:59:01.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:00 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:01.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:00 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:01.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:00 smithi158 ceph-mon[37241]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-03-04T08:59:01.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:00 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:01.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:00 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:01.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:00 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:01.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:00 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:01.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:00 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:01.053 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:00 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T08:59:01.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:00 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T08:59:01.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:00 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T08:59:01.054 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:00 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T08:59:02.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:01 smithi150 ceph-mon[36807]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:59:02.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:01 smithi158 ceph-mon[37241]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:59:02.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:01 smithi071 ceph-mon[32789]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T08:59:03.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:02 smithi150 ceph-mon[36807]: mgrmap e24: x(active, since 6s), standbys: z, y 2024-03-04T08:59:03.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:02 smithi158 ceph-mon[37241]: mgrmap e24: x(active, since 6s), standbys: z, y 2024-03-04T08:59:03.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:02 smithi071 ceph-mon[32789]: mgrmap e24: x(active, since 6s), standbys: z, y 2024-03-04T08:59:04.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:03 smithi150 ceph-mon[36807]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 171 KiB/s wr, 99 op/s 2024-03-04T08:59:04.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:03 smithi158 ceph-mon[37241]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 171 KiB/s wr, 99 op/s 2024-03-04T08:59:04.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:03 smithi071 ceph-mon[32789]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 171 KiB/s wr, 99 op/s 2024-03-04T08:59:06.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:05 smithi071 ceph-mon[32789]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 133 KiB/s wr, 77 op/s 2024-03-04T08:59:06.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:05 smithi150 ceph-mon[36807]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 133 KiB/s wr, 77 op/s 2024-03-04T08:59:06.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:05 smithi158 ceph-mon[37241]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 133 KiB/s wr, 77 op/s 2024-03-04T08:59:08.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:07 smithi071 ceph-mon[32789]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 109 KiB/s wr, 64 op/s 2024-03-04T08:59:08.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:07 smithi150 ceph-mon[36807]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 109 KiB/s wr, 64 op/s 2024-03-04T08:59:08.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:07 smithi158 ceph-mon[37241]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 109 KiB/s wr, 64 op/s 2024-03-04T08:59:10.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:09 smithi071 ceph-mon[32789]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 100 KiB/s wr, 58 op/s 2024-03-04T08:59:10.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:09 smithi150 ceph-mon[36807]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 100 KiB/s wr, 58 op/s 2024-03-04T08:59:10.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:09 smithi158 ceph-mon[37241]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 100 KiB/s wr, 58 op/s 2024-03-04T08:59:12.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:11 smithi071 ceph-mon[32789]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 100 KiB/s wr, 58 op/s 2024-03-04T08:59:12.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:11 smithi150 ceph-mon[36807]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 100 KiB/s wr, 58 op/s 2024-03-04T08:59:12.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:11 smithi158 ceph-mon[37241]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 100 KiB/s wr, 58 op/s 2024-03-04T08:59:14.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:13 smithi071 ceph-mon[32789]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 100 KiB/s wr, 59 op/s 2024-03-04T08:59:14.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:13 smithi150 ceph-mon[36807]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 100 KiB/s wr, 59 op/s 2024-03-04T08:59:14.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:13 smithi158 ceph-mon[37241]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 100 KiB/s wr, 59 op/s 2024-03-04T08:59:16.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:15 smithi071 ceph-mon[32789]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 170 B/s wr, 1 op/s 2024-03-04T08:59:16.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:15 smithi150 ceph-mon[36807]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 170 B/s wr, 1 op/s 2024-03-04T08:59:16.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:15 smithi158 ceph-mon[37241]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 170 B/s wr, 1 op/s 2024-03-04T08:59:18.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:17 smithi071 ceph-mon[32789]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-03-04T08:59:18.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:17 smithi150 ceph-mon[36807]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-03-04T08:59:18.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:17 smithi158 ceph-mon[37241]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-03-04T08:59:20.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:19 smithi071 ceph-mon[32789]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-03-04T08:59:20.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:19 smithi150 ceph-mon[36807]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-03-04T08:59:20.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:19 smithi158 ceph-mon[37241]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-03-04T08:59:22.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:21 smithi071 ceph-mon[32789]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-03-04T08:59:22.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:21 smithi150 ceph-mon[36807]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-03-04T08:59:22.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:21 smithi158 ceph-mon[37241]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-03-04T08:59:24.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:23 smithi150 ceph-mon[36807]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-04T08:59:24.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:23 smithi158 ceph-mon[37241]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-04T08:59:24.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:23 smithi071 ceph-mon[32789]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-04T08:59:26.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:25 smithi150 ceph-mon[36807]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:59:26.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:25 smithi158 ceph-mon[37241]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:59:26.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:25 smithi071 ceph-mon[32789]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:59:28.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:27 smithi150 ceph-mon[36807]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:59:28.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:27 smithi158 ceph-mon[37241]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:59:28.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:27 smithi071 ceph-mon[32789]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:59:30.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:29 smithi150 ceph-mon[36807]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:59:30.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:29 smithi158 ceph-mon[37241]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:59:30.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:29 smithi071 ceph-mon[32789]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:59:32.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:31 smithi150 ceph-mon[36807]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:59:32.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:31 smithi158 ceph-mon[37241]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:59:32.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:31 smithi071 ceph-mon[32789]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:59:34.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:33 smithi150 ceph-mon[36807]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:59:34.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:33 smithi158 ceph-mon[37241]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:59:34.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:33 smithi071 ceph-mon[32789]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:59:36.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:35 smithi150 ceph-mon[36807]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:59:36.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:35 smithi158 ceph-mon[37241]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:59:36.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:35 smithi071 ceph-mon[32789]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:59:38.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:37 smithi150 ceph-mon[36807]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-04T08:59:38.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:37 smithi158 ceph-mon[37241]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-04T08:59:38.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:37 smithi071 ceph-mon[32789]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-04T08:59:40.223 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:39 smithi071 ceph-mon[32789]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-04T08:59:40.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:39 smithi150 ceph-mon[36807]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-04T08:59:40.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:39 smithi158 ceph-mon[37241]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-04T08:59:42.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:41 smithi150 ceph-mon[36807]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-04T08:59:42.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:41 smithi158 ceph-mon[37241]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-04T08:59:42.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:41 smithi071 ceph-mon[32789]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-04T08:59:44.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:43 smithi150 ceph-mon[36807]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-04T08:59:44.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:43 smithi158 ceph-mon[37241]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-04T08:59:44.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:43 smithi071 ceph-mon[32789]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-04T08:59:46.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:45 smithi150 ceph-mon[36807]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-04T08:59:46.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:45 smithi158 ceph-mon[37241]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-04T08:59:46.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:45 smithi071 ceph-mon[32789]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-04T08:59:48.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:47 smithi150 ceph-mon[36807]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-04T08:59:48.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:47 smithi158 ceph-mon[37241]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-04T08:59:48.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:47 smithi071 ceph-mon[32789]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 938 B/s wr, 2 op/s 2024-03-04T08:59:50.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:49 smithi150 ceph-mon[36807]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:59:50.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:49 smithi158 ceph-mon[37241]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:59:50.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:49 smithi071 ceph-mon[32789]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:59:52.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:51 smithi150 ceph-mon[36807]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:59:52.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:51 smithi158 ceph-mon[37241]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:59:52.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:51 smithi071 ceph-mon[32789]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:59:54.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:53 smithi150 ceph-mon[36807]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:59:54.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:53 smithi158 ceph-mon[37241]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:59:54.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:53 smithi071 ceph-mon[32789]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:59:56.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:55 smithi150 ceph-mon[36807]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:59:56.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:55 smithi158 ceph-mon[37241]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:59:56.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:55 smithi071 ceph-mon[32789]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T08:59:58.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:57 smithi150 ceph-mon[36807]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:59:58.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:57 smithi158 ceph-mon[37241]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T08:59:58.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:57 smithi071 ceph-mon[32789]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T09:00:00.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:59 smithi150 ceph-mon[36807]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:00:00.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 08:59:59 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:00:00.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:59 smithi158 ceph-mon[37241]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:00:00.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 08:59:59 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:00:00.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:59 smithi071 ceph-mon[32789]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:00:00.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 08:59:59 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:00:01.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:00 smithi150 ceph-mon[36807]: overall HEALTH_OK 2024-03-04T09:00:01.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:00 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:00:01.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:00 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:00:01.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:00 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:00:01.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:00 smithi158 ceph-mon[37241]: overall HEALTH_OK 2024-03-04T09:00:01.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:00 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:00:01.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:00 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:00:01.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:00 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:00:01.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:00 smithi071 ceph-mon[32789]: overall HEALTH_OK 2024-03-04T09:00:01.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:00 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:00:01.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:00 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:00:01.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:00 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:00:02.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:01 smithi150 ceph-mon[36807]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:00:02.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:01 smithi158 ceph-mon[37241]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:00:02.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:01 smithi071 ceph-mon[32789]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:00:04.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:03 smithi150 ceph-mon[36807]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-03-04T09:00:04.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:03 smithi158 ceph-mon[37241]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-03-04T09:00:04.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:03 smithi071 ceph-mon[32789]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-03-04T09:00:06.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:05 smithi150 ceph-mon[36807]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-03-04T09:00:06.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:05 smithi158 ceph-mon[37241]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-03-04T09:00:06.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:05 smithi071 ceph-mon[32789]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-03-04T09:00:08.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:07 smithi150 ceph-mon[36807]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-03-04T09:00:08.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:07 smithi158 ceph-mon[37241]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-03-04T09:00:08.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:07 smithi071 ceph-mon[32789]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-03-04T09:00:10.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:09 smithi150 ceph-mon[36807]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-03-04T09:00:10.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:09 smithi158 ceph-mon[37241]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-03-04T09:00:10.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:09 smithi071 ceph-mon[32789]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-03-04T09:00:11.881 INFO:tasks.workunit.client.0.smithi071.stderr:Note: switching to '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'. 2024-03-04T09:00:11.881 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-03-04T09:00:11.881 INFO:tasks.workunit.client.0.smithi071.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-04T09:00:11.881 INFO:tasks.workunit.client.0.smithi071.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-04T09:00:11.881 INFO:tasks.workunit.client.0.smithi071.stderr:state without impacting any branches by switching back to a branch. 2024-03-04T09:00:11.881 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-03-04T09:00:11.881 INFO:tasks.workunit.client.0.smithi071.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-04T09:00:11.882 INFO:tasks.workunit.client.0.smithi071.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-04T09:00:11.882 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-03-04T09:00:11.882 INFO:tasks.workunit.client.0.smithi071.stderr: git switch -c 2024-03-04T09:00:11.882 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-03-04T09:00:11.882 INFO:tasks.workunit.client.0.smithi071.stderr:Or undo this operation with: 2024-03-04T09:00:11.882 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-03-04T09:00:11.882 INFO:tasks.workunit.client.0.smithi071.stderr: git switch - 2024-03-04T09:00:11.882 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-03-04T09:00:11.882 INFO:tasks.workunit.client.0.smithi071.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-04T09:00:11.882 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-03-04T09:00:11.882 INFO:tasks.workunit.client.0.smithi071.stderr:HEAD is now at 1bc250c7129 mds/quiesce-db: incorporate review comments 2024-03-04T09:00:11.883 DEBUG:teuthology.orchestra.run.smithi071:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-04T09:00:11.949 INFO:tasks.workunit.client.0.smithi071.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-04T09:00:11.951 INFO:tasks.workunit.client.0.smithi071.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T09:00:11.951 INFO:tasks.workunit.client.0.smithi071.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-04T09:00:12.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:12 smithi150 ceph-mon[36807]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-03-04T09:00:12.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:12 smithi158 ceph-mon[37241]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-03-04T09:00:12.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:12 smithi071 ceph-mon[32789]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-03-04T09:00:12.761 INFO:tasks.workunit.client.0.smithi071.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-04T09:00:12.824 INFO:tasks.workunit.client.0.smithi071.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-04T09:00:12.865 INFO:tasks.workunit.client.0.smithi071.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T09:00:12.867 INFO:tasks.workunit.client.0.smithi071.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T09:00:12.867 INFO:tasks.workunit.client.0.smithi071.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-04T09:00:12.907 INFO:tasks.workunit.client.0.smithi071.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T09:00:12.910 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-04T09:00:12.910 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-04T09:00:12.966 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-03-04T09:00:12.967 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-03-04T09:00:12.968 DEBUG:teuthology.orchestra.run.smithi071:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-03-04T09:00:13.027 INFO:tasks.workunit.client.0.smithi071.stderr:+ dbench 1 2024-03-04T09:00:13.033 INFO:tasks.workunit.client.0.smithi071.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-03-04T09:00:13.033 INFO:tasks.workunit.client.0.smithi071.stdout: 2024-03-04T09:00:13.033 INFO:tasks.workunit.client.0.smithi071.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-03-04T09:00:13.040 INFO:tasks.workunit.client.0.smithi071.stdout:0 of 1 processes prepared for launch 0 sec 2024-03-04T09:00:13.140 INFO:tasks.workunit.client.0.smithi071.stdout:1 of 1 processes prepared for launch 0 sec 2024-03-04T09:00:13.140 INFO:tasks.workunit.client.0.smithi071.stdout:releasing clients 2024-03-04T09:00:14.140 INFO:tasks.workunit.client.0.smithi071.stdout: 1 3339 35.38 MB/sec warmup 1 sec latency 6.917 ms 2024-03-04T09:00:14.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:14 smithi150 ceph-mon[36807]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-03-04T09:00:14.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:14 smithi158 ceph-mon[37241]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-03-04T09:00:14.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:14 smithi071 ceph-mon[32789]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-03-04T09:00:15.140 INFO:tasks.workunit.client.0.smithi071.stdout: 1 6149 24.06 MB/sec warmup 2 sec latency 5.940 ms 2024-03-04T09:00:16.140 INFO:tasks.workunit.client.0.smithi071.stdout: 1 8661 21.45 MB/sec warmup 3 sec latency 22.984 ms 2024-03-04T09:00:16.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:16 smithi150 ceph-mon[36807]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 85 B/s wr, 1 op/s 2024-03-04T09:00:16.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:16 smithi158 ceph-mon[37241]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 85 B/s wr, 1 op/s 2024-03-04T09:00:16.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:16 smithi071 ceph-mon[32789]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 85 B/s wr, 1 op/s 2024-03-04T09:00:17.140 INFO:tasks.workunit.client.0.smithi071.stdout: 1 11516 19.73 MB/sec warmup 4 sec latency 20.477 ms 2024-03-04T09:00:18.140 INFO:tasks.workunit.client.0.smithi071.stdout: 1 14662 19.49 MB/sec warmup 5 sec latency 7.031 ms 2024-03-04T09:00:18.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:18 smithi150 ceph-mon[36807]: pgmap v44: 129 pgs: 129 active+clean; 5.6 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 413 KiB/s wr, 50 op/s 2024-03-04T09:00:18.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:18 smithi158 ceph-mon[37241]: pgmap v44: 129 pgs: 129 active+clean; 5.6 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 413 KiB/s wr, 50 op/s 2024-03-04T09:00:18.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:18 smithi071 ceph-mon[32789]: pgmap v44: 129 pgs: 129 active+clean; 5.6 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 413 KiB/s wr, 50 op/s 2024-03-04T09:00:19.140 INFO:tasks.workunit.client.0.smithi071.stdout: 1 17991 19.37 MB/sec warmup 6 sec latency 34.590 ms 2024-03-04T09:00:20.140 INFO:tasks.workunit.client.0.smithi071.stdout: 1 21096 18.72 MB/sec warmup 7 sec latency 9.793 ms 2024-03-04T09:00:20.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:20 smithi150 ceph-mon[36807]: pgmap v45: 129 pgs: 129 active+clean; 5.6 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 413 KiB/s wr, 49 op/s 2024-03-04T09:00:20.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:20 smithi158 ceph-mon[37241]: pgmap v45: 129 pgs: 129 active+clean; 5.6 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 413 KiB/s wr, 49 op/s 2024-03-04T09:00:20.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:20 smithi071 ceph-mon[32789]: pgmap v45: 129 pgs: 129 active+clean; 5.6 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 413 KiB/s wr, 49 op/s 2024-03-04T09:00:21.140 INFO:tasks.workunit.client.0.smithi071.stdout: 1 23754 17.86 MB/sec warmup 8 sec latency 6.460 ms 2024-03-04T09:00:22.140 INFO:tasks.workunit.client.0.smithi071.stdout: 1 26394 17.77 MB/sec warmup 9 sec latency 28.226 ms 2024-03-04T09:00:22.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:22 smithi150 ceph-mon[36807]: pgmap v46: 129 pgs: 129 active+clean; 5.9 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 445 KiB/s wr, 50 op/s 2024-03-04T09:00:22.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:22 smithi158 ceph-mon[37241]: pgmap v46: 129 pgs: 129 active+clean; 5.9 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 445 KiB/s wr, 50 op/s 2024-03-04T09:00:22.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:22 smithi071 ceph-mon[32789]: pgmap v46: 129 pgs: 129 active+clean; 5.9 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 445 KiB/s wr, 50 op/s 2024-03-04T09:00:23.140 INFO:tasks.workunit.client.0.smithi071.stdout: 1 29217 17.45 MB/sec warmup 10 sec latency 31.999 ms 2024-03-04T09:00:24.140 INFO:tasks.workunit.client.0.smithi071.stdout: 1 32535 17.56 MB/sec warmup 11 sec latency 5.968 ms 2024-03-04T09:00:24.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:24 smithi150 ceph-mon[36807]: pgmap v47: 129 pgs: 129 active+clean; 12 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-03-04T09:00:24.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:24 smithi158 ceph-mon[37241]: pgmap v47: 129 pgs: 129 active+clean; 12 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-03-04T09:00:24.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:24 smithi071 ceph-mon[32789]: pgmap v47: 129 pgs: 129 active+clean; 12 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-03-04T09:00:25.141 INFO:tasks.workunit.client.0.smithi071.stdout: 1 35779 17.66 MB/sec warmup 12 sec latency 17.708 ms 2024-03-04T09:00:25.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:25 smithi071 ceph-mon[32789]: pgmap v48: 129 pgs: 129 active+clean; 12 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-03-04T09:00:25.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:25 smithi150 ceph-mon[36807]: pgmap v48: 129 pgs: 129 active+clean; 12 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-03-04T09:00:25.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:25 smithi158 ceph-mon[37241]: pgmap v48: 129 pgs: 129 active+clean; 12 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 1.1 MiB/s wr, 138 op/s 2024-03-04T09:00:26.141 INFO:tasks.workunit.client.0.smithi071.stdout: 1 38532 17.29 MB/sec warmup 13 sec latency 30.686 ms 2024-03-04T09:00:27.141 INFO:tasks.workunit.client.0.smithi071.stdout: 1 41455 17.02 MB/sec warmup 14 sec latency 7.640 ms 2024-03-04T09:00:27.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:27 smithi071 ceph-mon[32789]: mgrmap e25: x(active, since 91s), standbys: z, y 2024-03-04T09:00:27.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:27 smithi071 ceph-mon[32789]: pgmap v49: 129 pgs: 129 active+clean; 19 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-03-04T09:00:27.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:27 smithi150 ceph-mon[36807]: mgrmap e25: x(active, since 91s), standbys: z, y 2024-03-04T09:00:27.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:27 smithi150 ceph-mon[36807]: pgmap v49: 129 pgs: 129 active+clean; 19 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-03-04T09:00:27.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:27 smithi158 ceph-mon[37241]: mgrmap e25: x(active, since 91s), standbys: z, y 2024-03-04T09:00:27.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:27 smithi158 ceph-mon[37241]: pgmap v49: 129 pgs: 129 active+clean; 19 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-03-04T09:00:28.141 INFO:tasks.workunit.client.0.smithi071.stdout: 1 44116 17.04 MB/sec warmup 15 sec latency 19.492 ms 2024-03-04T09:00:29.141 INFO:tasks.workunit.client.0.smithi071.stdout: 1 47021 16.89 MB/sec warmup 16 sec latency 26.212 ms 2024-03-04T09:00:30.141 INFO:tasks.workunit.client.0.smithi071.stdout: 1 50159 16.98 MB/sec warmup 17 sec latency 16.334 ms 2024-03-04T09:00:30.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:29 smithi071 ceph-mon[32789]: pgmap v50: 129 pgs: 129 active+clean; 19 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-03-04T09:00:30.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:29 smithi150 ceph-mon[36807]: pgmap v50: 129 pgs: 129 active+clean; 19 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-03-04T09:00:30.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:29 smithi158 ceph-mon[37241]: pgmap v50: 129 pgs: 129 active+clean; 19 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-03-04T09:00:31.141 INFO:tasks.workunit.client.0.smithi071.stdout: 1 50159 16.04 MB/sec warmup 18 sec latency 1016.445 ms 2024-03-04T09:00:32.141 INFO:tasks.workunit.client.0.smithi071.stdout: 1 50159 15.19 MB/sec warmup 19 sec latency 2016.567 ms 2024-03-04T09:00:32.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:31 smithi071 ceph-mon[32789]: pgmap v51: 129 pgs: 129 active+clean; 19 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 1.5 MiB/s wr, 181 op/s 2024-03-04T09:00:32.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:31 smithi150 ceph-mon[36807]: pgmap v51: 129 pgs: 129 active+clean; 19 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 1.5 MiB/s wr, 181 op/s 2024-03-04T09:00:32.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:31 smithi158 ceph-mon[37241]: pgmap v51: 129 pgs: 129 active+clean; 19 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 1.5 MiB/s wr, 181 op/s 2024-03-04T09:00:33.141 INFO:tasks.workunit.client.0.smithi071.stdout: 1 50159 14.43 MB/sec warmup 20 sec latency 3016.672 ms 2024-03-04T09:00:34.141 INFO:tasks.workunit.client.0.smithi071.stdout: 1 50159 13.75 MB/sec warmup 21 sec latency 4016.757 ms 2024-03-04T09:00:35.055 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:34 smithi158 ceph-mon[37241]: pgmap v52: 129 pgs: 129 active+clean; 24 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 1.9 MiB/s wr, 236 op/s 2024-03-04T09:00:35.142 INFO:tasks.workunit.client.0.smithi071.stdout: 1 50447 13.14 MB/sec warmup 22 sec latency 4809.059 ms 2024-03-04T09:00:35.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:34 smithi071 ceph-mon[32789]: pgmap v52: 129 pgs: 129 active+clean; 24 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 1.9 MiB/s wr, 236 op/s 2024-03-04T09:00:35.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:34 smithi150 ceph-mon[36807]: pgmap v52: 129 pgs: 129 active+clean; 24 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 1.9 MiB/s wr, 236 op/s 2024-03-04T09:00:36.141 INFO:tasks.workunit.client.0.smithi071.stdout: 1 53626 13.38 MB/sec warmup 23 sec latency 8.674 ms 2024-03-04T09:00:36.180 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:35 smithi150 ceph-mon[36807]: pgmap v53: 129 pgs: 129 active+clean; 24 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-03-04T09:00:36.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:35 smithi158 ceph-mon[37241]: pgmap v53: 129 pgs: 129 active+clean; 24 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-03-04T09:00:36.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:35 smithi071 ceph-mon[32789]: pgmap v53: 129 pgs: 129 active+clean; 24 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-03-04T09:00:37.141 INFO:tasks.workunit.client.0.smithi071.stdout: 1 57012 13.61 MB/sec warmup 24 sec latency 6.810 ms 2024-03-04T09:00:38.142 INFO:tasks.workunit.client.0.smithi071.stdout: 1 59751 13.58 MB/sec warmup 25 sec latency 11.609 ms 2024-03-04T09:00:38.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:37 smithi150 ceph-mon[36807]: pgmap v54: 129 pgs: 129 active+clean; 26 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-03-04T09:00:38.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:37 smithi158 ceph-mon[37241]: pgmap v54: 129 pgs: 129 active+clean; 26 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-03-04T09:00:38.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:37 smithi071 ceph-mon[32789]: pgmap v54: 129 pgs: 129 active+clean; 26 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-03-04T09:00:39.141 INFO:tasks.workunit.client.0.smithi071.stdout: 1 61231 13.34 MB/sec warmup 26 sec latency 174.223 ms 2024-03-04T09:00:40.141 INFO:tasks.workunit.client.0.smithi071.stdout: 1 63322 13.28 MB/sec warmup 27 sec latency 507.828 ms 2024-03-04T09:00:40.223 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:39 smithi071 ceph-mon[32789]: pgmap v55: 129 pgs: 129 active+clean; 26 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 764 KiB/s wr, 96 op/s 2024-03-04T09:00:40.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:39 smithi150 ceph-mon[36807]: pgmap v55: 129 pgs: 129 active+clean; 26 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 764 KiB/s wr, 96 op/s 2024-03-04T09:00:40.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:39 smithi158 ceph-mon[37241]: pgmap v55: 129 pgs: 129 active+clean; 26 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 764 KiB/s wr, 96 op/s 2024-03-04T09:00:41.141 INFO:tasks.workunit.client.0.smithi071.stdout: 1 64862 13.08 MB/sec warmup 28 sec latency 36.091 ms 2024-03-04T09:00:42.142 INFO:tasks.workunit.client.0.smithi071.stdout: 1 67984 13.25 MB/sec warmup 29 sec latency 20.641 ms 2024-03-04T09:00:42.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:41 smithi150 ceph-mon[36807]: pgmap v56: 129 pgs: 129 active+clean; 37 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 2.9 MiB/s wr, 153 op/s 2024-03-04T09:00:42.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:41 smithi158 ceph-mon[37241]: pgmap v56: 129 pgs: 129 active+clean; 37 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 2.9 MiB/s wr, 153 op/s 2024-03-04T09:00:42.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:41 smithi071 ceph-mon[32789]: pgmap v56: 129 pgs: 129 active+clean; 37 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 2.9 MiB/s wr, 153 op/s 2024-03-04T09:00:43.142 INFO:tasks.workunit.client.0.smithi071.stdout: 1 71197 13.44 MB/sec warmup 30 sec latency 17.932 ms 2024-03-04T09:00:44.142 INFO:tasks.workunit.client.0.smithi071.stdout: 1 73874 13.38 MB/sec warmup 31 sec latency 17.387 ms 2024-03-04T09:00:44.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:43 smithi150 ceph-mon[36807]: pgmap v57: 129 pgs: 129 active+clean; 48 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 7.0 MiB/s wr, 312 op/s 2024-03-04T09:00:44.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:43 smithi158 ceph-mon[37241]: pgmap v57: 129 pgs: 129 active+clean; 48 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 7.0 MiB/s wr, 312 op/s 2024-03-04T09:00:44.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:43 smithi071 ceph-mon[32789]: pgmap v57: 129 pgs: 129 active+clean; 48 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 7.0 MiB/s wr, 312 op/s 2024-03-04T09:00:45.142 INFO:tasks.workunit.client.0.smithi071.stdout: 1 76683 13.38 MB/sec warmup 32 sec latency 17.630 ms 2024-03-04T09:00:46.142 INFO:tasks.workunit.client.0.smithi071.stdout: 1 79061 13.41 MB/sec warmup 33 sec latency 8.645 ms 2024-03-04T09:00:46.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:45 smithi150 ceph-mon[36807]: pgmap v58: 129 pgs: 129 active+clean; 48 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 6.6 MiB/s wr, 255 op/s 2024-03-04T09:00:46.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:45 smithi158 ceph-mon[37241]: pgmap v58: 129 pgs: 129 active+clean; 48 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 6.6 MiB/s wr, 255 op/s 2024-03-04T09:00:46.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:45 smithi071 ceph-mon[32789]: pgmap v58: 129 pgs: 129 active+clean; 48 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 6.6 MiB/s wr, 255 op/s 2024-03-04T09:00:47.142 INFO:tasks.workunit.client.0.smithi071.stdout: 1 82181 13.55 MB/sec warmup 34 sec latency 19.214 ms 2024-03-04T09:00:48.142 INFO:tasks.workunit.client.0.smithi071.stdout: 1 85426 13.70 MB/sec warmup 35 sec latency 12.136 ms 2024-03-04T09:00:48.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:47 smithi150 ceph-mon[36807]: pgmap v59: 129 pgs: 129 active+clean; 50 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.2 MiB/s wr, 337 op/s 2024-03-04T09:00:48.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:47 smithi158 ceph-mon[37241]: pgmap v59: 129 pgs: 129 active+clean; 50 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.2 MiB/s wr, 337 op/s 2024-03-04T09:00:48.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:47 smithi071 ceph-mon[32789]: pgmap v59: 129 pgs: 129 active+clean; 50 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.2 MiB/s wr, 337 op/s 2024-03-04T09:00:49.142 INFO:tasks.workunit.client.0.smithi071.stdout: 1 88384 13.70 MB/sec warmup 36 sec latency 19.727 ms 2024-03-04T09:00:50.142 INFO:tasks.workunit.client.0.smithi071.stdout: 1 90867 13.64 MB/sec warmup 37 sec latency 19.106 ms 2024-03-04T09:00:50.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:49 smithi071 ceph-mon[32789]: pgmap v60: 129 pgs: 129 active+clean; 50 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.9 MiB/s wr, 299 op/s 2024-03-04T09:00:50.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:49 smithi150 ceph-mon[36807]: pgmap v60: 129 pgs: 129 active+clean; 50 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.9 MiB/s wr, 299 op/s 2024-03-04T09:00:50.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:49 smithi158 ceph-mon[37241]: pgmap v60: 129 pgs: 129 active+clean; 50 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.9 MiB/s wr, 299 op/s 2024-03-04T09:00:51.142 INFO:tasks.workunit.client.0.smithi071.stdout: 1 93405 13.69 MB/sec warmup 38 sec latency 20.849 ms 2024-03-04T09:00:52.142 INFO:tasks.workunit.client.0.smithi071.stdout: 1 96481 13.78 MB/sec warmup 39 sec latency 8.125 ms 2024-03-04T09:00:52.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:51 smithi150 ceph-mon[36807]: pgmap v61: 129 pgs: 129 active+clean; 55 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.5 MiB/s wr, 375 op/s 2024-03-04T09:00:52.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:51 smithi158 ceph-mon[37241]: pgmap v61: 129 pgs: 129 active+clean; 55 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.5 MiB/s wr, 375 op/s 2024-03-04T09:00:52.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:51 smithi071 ceph-mon[32789]: pgmap v61: 129 pgs: 129 active+clean; 55 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.5 MiB/s wr, 375 op/s 2024-03-04T09:00:53.142 INFO:tasks.workunit.client.0.smithi071.stdout: 1 99629 13.90 MB/sec warmup 40 sec latency 16.457 ms 2024-03-04T09:00:54.142 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102569 13.90 MB/sec warmup 41 sec latency 6.990 ms 2024-03-04T09:00:54.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:53 smithi150 ceph-mon[36807]: pgmap v62: 129 pgs: 129 active+clean; 56 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.5 MiB/s wr, 332 op/s 2024-03-04T09:00:54.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:53 smithi158 ceph-mon[37241]: pgmap v62: 129 pgs: 129 active+clean; 56 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.5 MiB/s wr, 332 op/s 2024-03-04T09:00:54.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:53 smithi071 ceph-mon[32789]: pgmap v62: 129 pgs: 129 active+clean; 56 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.5 MiB/s wr, 332 op/s 2024-03-04T09:00:55.142 INFO:tasks.workunit.client.0.smithi071.stdout: 1 105246 13.87 MB/sec warmup 42 sec latency 14.941 ms 2024-03-04T09:00:56.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 107374 13.85 MB/sec warmup 43 sec latency 24.289 ms 2024-03-04T09:00:56.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:55 smithi150 ceph-mon[36807]: pgmap v63: 129 pgs: 129 active+clean; 56 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-03-04T09:00:56.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:55 smithi158 ceph-mon[37241]: pgmap v63: 129 pgs: 129 active+clean; 56 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-03-04T09:00:56.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:55 smithi071 ceph-mon[32789]: pgmap v63: 129 pgs: 129 active+clean; 56 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-03-04T09:00:57.142 INFO:tasks.workunit.client.0.smithi071.stdout: 1 110451 13.95 MB/sec warmup 44 sec latency 17.339 ms 2024-03-04T09:00:58.142 INFO:tasks.workunit.client.0.smithi071.stdout: 1 113659 14.03 MB/sec warmup 45 sec latency 17.213 ms 2024-03-04T09:00:58.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:57 smithi150 ceph-mon[36807]: pgmap v64: 129 pgs: 129 active+clean; 63 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 277 op/s 2024-03-04T09:00:58.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:57 smithi158 ceph-mon[37241]: pgmap v64: 129 pgs: 129 active+clean; 63 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 277 op/s 2024-03-04T09:00:58.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:57 smithi071 ceph-mon[32789]: pgmap v64: 129 pgs: 129 active+clean; 63 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 277 op/s 2024-03-04T09:00:59.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 116102 13.93 MB/sec warmup 46 sec latency 17.876 ms 2024-03-04T09:01:00.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 118490 13.98 MB/sec warmup 47 sec latency 11.885 ms 2024-03-04T09:01:00.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:00:59 smithi150 ceph-mon[36807]: pgmap v65: 129 pgs: 129 active+clean; 63 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-03-04T09:01:00.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:00:59 smithi158 ceph-mon[37241]: pgmap v65: 129 pgs: 129 active+clean; 63 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-03-04T09:01:00.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:00:59 smithi071 ceph-mon[32789]: pgmap v65: 129 pgs: 129 active+clean; 63 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-03-04T09:01:01.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 120349 13.88 MB/sec warmup 48 sec latency 53.850 ms 2024-03-04T09:01:01.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:00 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:01:01.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:00 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:01:01.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:00 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:01:02.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 122163 13.80 MB/sec warmup 49 sec latency 43.720 ms 2024-03-04T09:01:02.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:01 smithi150 ceph-mon[36807]: pgmap v66: 129 pgs: 129 active+clean; 69 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 271 op/s 2024-03-04T09:01:02.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:01 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:01:02.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:01 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:01:02.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:01 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:01:02.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:01 smithi158 ceph-mon[37241]: pgmap v66: 129 pgs: 129 active+clean; 69 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 271 op/s 2024-03-04T09:01:02.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:01 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:01:02.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:01 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:01:02.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:01 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:01:02.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:01 smithi071 ceph-mon[32789]: pgmap v66: 129 pgs: 129 active+clean; 69 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 271 op/s 2024-03-04T09:01:02.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:01 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:01:02.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:01 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:01:02.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:01 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:01:03.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 124849 13.81 MB/sec warmup 50 sec latency 14.806 ms 2024-03-04T09:01:04.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 127982 13.90 MB/sec warmup 51 sec latency 15.451 ms 2024-03-04T09:01:04.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:03 smithi150 ceph-mon[36807]: pgmap v67: 129 pgs: 129 active+clean; 70 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 230 op/s 2024-03-04T09:01:04.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:03 smithi158 ceph-mon[37241]: pgmap v67: 129 pgs: 129 active+clean; 70 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 230 op/s 2024-03-04T09:01:04.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:03 smithi071 ceph-mon[32789]: pgmap v67: 129 pgs: 129 active+clean; 70 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 230 op/s 2024-03-04T09:01:05.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 130708 13.88 MB/sec warmup 52 sec latency 25.517 ms 2024-03-04T09:01:06.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 132711 13.84 MB/sec warmup 53 sec latency 13.059 ms 2024-03-04T09:01:06.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:05 smithi150 ceph-mon[36807]: pgmap v68: 129 pgs: 129 active+clean; 70 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 216 op/s 2024-03-04T09:01:06.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:05 smithi158 ceph-mon[37241]: pgmap v68: 129 pgs: 129 active+clean; 70 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 216 op/s 2024-03-04T09:01:06.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:05 smithi071 ceph-mon[32789]: pgmap v68: 129 pgs: 129 active+clean; 70 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 216 op/s 2024-03-04T09:01:07.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 135417 13.85 MB/sec warmup 54 sec latency 19.748 ms 2024-03-04T09:01:08.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 138580 13.94 MB/sec warmup 55 sec latency 17.243 ms 2024-03-04T09:01:08.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:07 smithi150 ceph-mon[36807]: pgmap v69: 129 pgs: 129 active+clean; 77 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 310 op/s 2024-03-04T09:01:08.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:07 smithi158 ceph-mon[37241]: pgmap v69: 129 pgs: 129 active+clean; 77 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 310 op/s 2024-03-04T09:01:08.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:07 smithi071 ceph-mon[32789]: pgmap v69: 129 pgs: 129 active+clean; 77 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 310 op/s 2024-03-04T09:01:09.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 141242 13.89 MB/sec warmup 56 sec latency 16.863 ms 2024-03-04T09:01:10.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 143469 13.88 MB/sec warmup 57 sec latency 9.485 ms 2024-03-04T09:01:10.223 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:09 smithi071 ceph-mon[32789]: pgmap v70: 129 pgs: 129 active+clean; 77 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-03-04T09:01:10.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:09 smithi150 ceph-mon[36807]: pgmap v70: 129 pgs: 129 active+clean; 77 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-03-04T09:01:10.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:09 smithi158 ceph-mon[37241]: pgmap v70: 129 pgs: 129 active+clean; 77 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-03-04T09:01:11.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 146034 13.88 MB/sec warmup 58 sec latency 82.316 ms 2024-03-04T09:01:12.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 149216 13.96 MB/sec warmup 59 sec latency 10.477 ms 2024-03-04T09:01:12.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:11 smithi150 ceph-mon[36807]: pgmap v71: 129 pgs: 129 active+clean; 83 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 286 op/s 2024-03-04T09:01:12.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:11 smithi158 ceph-mon[37241]: pgmap v71: 129 pgs: 129 active+clean; 83 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 286 op/s 2024-03-04T09:01:12.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:11 smithi071 ceph-mon[32789]: pgmap v71: 129 pgs: 129 active+clean; 83 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 286 op/s 2024-03-04T09:01:13.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 151720 13.92 MB/sec warmup 60 sec latency 19.783 ms 2024-03-04T09:01:14.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 154242 13.91 MB/sec warmup 61 sec latency 15.138 ms 2024-03-04T09:01:14.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:13 smithi150 ceph-mon[36807]: pgmap v72: 129 pgs: 129 active+clean; 84 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.7 MiB/s wr, 220 op/s 2024-03-04T09:01:14.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:13 smithi158 ceph-mon[37241]: pgmap v72: 129 pgs: 129 active+clean; 84 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.7 MiB/s wr, 220 op/s 2024-03-04T09:01:14.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:13 smithi071 ceph-mon[32789]: pgmap v72: 129 pgs: 129 active+clean; 84 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.7 MiB/s wr, 220 op/s 2024-03-04T09:01:15.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 156661 13.91 MB/sec warmup 62 sec latency 95.262 ms 2024-03-04T09:01:16.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 159602 13.96 MB/sec warmup 63 sec latency 39.471 ms 2024-03-04T09:01:16.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:15 smithi150 ceph-mon[36807]: pgmap v73: 129 pgs: 129 active+clean; 84 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.5 MiB/s wr, 183 op/s 2024-03-04T09:01:16.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:15 smithi158 ceph-mon[37241]: pgmap v73: 129 pgs: 129 active+clean; 84 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.5 MiB/s wr, 183 op/s 2024-03-04T09:01:16.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:15 smithi071 ceph-mon[32789]: pgmap v73: 129 pgs: 129 active+clean; 84 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.5 MiB/s wr, 183 op/s 2024-03-04T09:01:17.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 161687 13.86 MB/sec warmup 64 sec latency 25.621 ms 2024-03-04T09:01:18.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 164055 13.87 MB/sec warmup 65 sec latency 21.588 ms 2024-03-04T09:01:18.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:17 smithi150 ceph-mon[36807]: pgmap v74: 129 pgs: 129 active+clean; 98 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 296 op/s 2024-03-04T09:01:18.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:17 smithi158 ceph-mon[37241]: pgmap v74: 129 pgs: 129 active+clean; 98 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 296 op/s 2024-03-04T09:01:18.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:17 smithi071 ceph-mon[32789]: pgmap v74: 129 pgs: 129 active+clean; 98 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 296 op/s 2024-03-04T09:01:19.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 167123 13.94 MB/sec warmup 66 sec latency 21.262 ms 2024-03-04T09:01:20.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 169660 13.91 MB/sec warmup 67 sec latency 56.873 ms 2024-03-04T09:01:20.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:19 smithi150 ceph-mon[36807]: pgmap v75: 129 pgs: 129 active+clean; 98 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 203 op/s 2024-03-04T09:01:20.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:19 smithi158 ceph-mon[37241]: pgmap v75: 129 pgs: 129 active+clean; 98 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 203 op/s 2024-03-04T09:01:20.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:19 smithi071 ceph-mon[32789]: pgmap v75: 129 pgs: 129 active+clean; 98 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 203 op/s 2024-03-04T09:01:21.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 171456 13.84 MB/sec warmup 68 sec latency 33.660 ms 2024-03-04T09:01:22.144 INFO:tasks.workunit.client.0.smithi071.stdout: 1 174390 13.88 MB/sec warmup 69 sec latency 12.696 ms 2024-03-04T09:01:22.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:21 smithi150 ceph-mon[36807]: pgmap v76: 129 pgs: 129 active+clean; 97 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 279 op/s 2024-03-04T09:01:22.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:21 smithi158 ceph-mon[37241]: pgmap v76: 129 pgs: 129 active+clean; 97 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 279 op/s 2024-03-04T09:01:22.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:21 smithi071 ceph-mon[32789]: pgmap v76: 129 pgs: 129 active+clean; 97 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 279 op/s 2024-03-04T09:01:23.143 INFO:tasks.workunit.client.0.smithi071.stdout: 1 177504 13.94 MB/sec warmup 70 sec latency 10.603 ms 2024-03-04T09:01:24.144 INFO:tasks.workunit.client.0.smithi071.stdout: 1 179961 13.88 MB/sec warmup 71 sec latency 12.943 ms 2024-03-04T09:01:24.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:23 smithi150 ceph-mon[36807]: pgmap v77: 129 pgs: 129 active+clean; 97 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 250 op/s 2024-03-04T09:01:24.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:23 smithi158 ceph-mon[37241]: pgmap v77: 129 pgs: 129 active+clean; 97 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 250 op/s 2024-03-04T09:01:24.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:23 smithi071 ceph-mon[32789]: pgmap v77: 129 pgs: 129 active+clean; 97 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 250 op/s 2024-03-04T09:01:25.144 INFO:tasks.workunit.client.0.smithi071.stdout: 1 181703 13.84 MB/sec warmup 72 sec latency 112.967 ms 2024-03-04T09:01:26.144 INFO:tasks.workunit.client.0.smithi071.stdout: 1 184620 13.89 MB/sec warmup 73 sec latency 80.138 ms 2024-03-04T09:01:26.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:25 smithi150 ceph-mon[36807]: pgmap v78: 129 pgs: 129 active+clean; 97 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 240 op/s 2024-03-04T09:01:26.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:25 smithi158 ceph-mon[37241]: pgmap v78: 129 pgs: 129 active+clean; 97 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 240 op/s 2024-03-04T09:01:26.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:25 smithi071 ceph-mon[32789]: pgmap v78: 129 pgs: 129 active+clean; 97 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 240 op/s 2024-03-04T09:01:27.144 INFO:tasks.workunit.client.0.smithi071.stdout: 1 187237 13.87 MB/sec warmup 74 sec latency 11.612 ms 2024-03-04T09:01:28.144 INFO:tasks.workunit.client.0.smithi071.stdout: 1 189430 13.86 MB/sec warmup 75 sec latency 16.080 ms 2024-03-04T09:01:28.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:27 smithi150 ceph-mon[36807]: pgmap v79: 129 pgs: 129 active+clean; 102 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.2 MiB/s wr, 362 op/s 2024-03-04T09:01:28.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:27 smithi158 ceph-mon[37241]: pgmap v79: 129 pgs: 129 active+clean; 102 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.2 MiB/s wr, 362 op/s 2024-03-04T09:01:28.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:27 smithi071 ceph-mon[32789]: pgmap v79: 129 pgs: 129 active+clean; 102 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.2 MiB/s wr, 362 op/s 2024-03-04T09:01:29.144 INFO:tasks.workunit.client.0.smithi071.stdout: 1 192162 13.86 MB/sec warmup 76 sec latency 25.761 ms 2024-03-04T09:01:30.144 INFO:tasks.workunit.client.0.smithi071.stdout: 1 195233 13.91 MB/sec warmup 77 sec latency 41.647 ms 2024-03-04T09:01:30.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:29 smithi150 ceph-mon[36807]: pgmap v80: 129 pgs: 129 active+clean; 102 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 249 op/s 2024-03-04T09:01:30.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:29 smithi158 ceph-mon[37241]: pgmap v80: 129 pgs: 129 active+clean; 102 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 249 op/s 2024-03-04T09:01:30.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:29 smithi071 ceph-mon[32789]: pgmap v80: 129 pgs: 129 active+clean; 102 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 249 op/s 2024-03-04T09:01:31.144 INFO:tasks.workunit.client.0.smithi071.stdout: 1 197236 13.83 MB/sec warmup 78 sec latency 81.404 ms 2024-03-04T09:01:32.144 INFO:tasks.workunit.client.0.smithi071.stdout: 1 199628 13.83 MB/sec warmup 79 sec latency 8.458 ms 2024-03-04T09:01:32.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:31 smithi150 ceph-mon[36807]: pgmap v81: 129 pgs: 129 active+clean; 105 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.5 MiB/s wr, 328 op/s 2024-03-04T09:01:32.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:31 smithi158 ceph-mon[37241]: pgmap v81: 129 pgs: 129 active+clean; 105 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.5 MiB/s wr, 328 op/s 2024-03-04T09:01:32.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:31 smithi071 ceph-mon[32789]: pgmap v81: 129 pgs: 129 active+clean; 105 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.5 MiB/s wr, 328 op/s 2024-03-04T09:01:33.144 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202661 13.89 MB/sec warmup 80 sec latency 23.207 ms 2024-03-04T09:01:34.144 INFO:tasks.workunit.client.0.smithi071.stdout: 1 205758 13.93 MB/sec warmup 81 sec latency 18.757 ms 2024-03-04T09:01:34.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:33 smithi150 ceph-mon[36807]: pgmap v82: 129 pgs: 129 active+clean; 107 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 294 op/s 2024-03-04T09:01:34.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:33 smithi158 ceph-mon[37241]: pgmap v82: 129 pgs: 129 active+clean; 107 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 294 op/s 2024-03-04T09:01:34.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:33 smithi071 ceph-mon[32789]: pgmap v82: 129 pgs: 129 active+clean; 107 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 294 op/s 2024-03-04T09:01:35.144 INFO:tasks.workunit.client.0.smithi071.stdout: 1 208312 13.88 MB/sec warmup 82 sec latency 22.837 ms 2024-03-04T09:01:36.144 INFO:tasks.workunit.client.0.smithi071.stdout: 1 210191 13.85 MB/sec warmup 83 sec latency 56.135 ms 2024-03-04T09:01:36.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:35 smithi150 ceph-mon[36807]: pgmap v83: 129 pgs: 129 active+clean; 107 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 242 op/s 2024-03-04T09:01:36.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:35 smithi158 ceph-mon[37241]: pgmap v83: 129 pgs: 129 active+clean; 107 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 242 op/s 2024-03-04T09:01:36.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:35 smithi071 ceph-mon[32789]: pgmap v83: 129 pgs: 129 active+clean; 107 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 242 op/s 2024-03-04T09:01:37.144 INFO:tasks.workunit.client.0.smithi071.stdout: 1 213210 13.91 MB/sec warmup 84 sec latency 19.693 ms 2024-03-04T09:01:38.144 INFO:tasks.workunit.client.0.smithi071.stdout: 1 216417 13.95 MB/sec warmup 85 sec latency 16.168 ms 2024-03-04T09:01:38.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:37 smithi150 ceph-mon[36807]: pgmap v84: 129 pgs: 129 active+clean; 119 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.0 MiB/s wr, 350 op/s 2024-03-04T09:01:38.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:37 smithi158 ceph-mon[37241]: pgmap v84: 129 pgs: 129 active+clean; 119 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.0 MiB/s wr, 350 op/s 2024-03-04T09:01:38.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:37 smithi071 ceph-mon[32789]: pgmap v84: 129 pgs: 129 active+clean; 119 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.0 MiB/s wr, 350 op/s 2024-03-04T09:01:39.144 INFO:tasks.workunit.client.0.smithi071.stdout: 1 218901 13.90 MB/sec warmup 86 sec latency 21.078 ms 2024-03-04T09:01:40.144 INFO:tasks.workunit.client.0.smithi071.stdout: 1 221145 13.89 MB/sec warmup 87 sec latency 24.731 ms 2024-03-04T09:01:40.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:39 smithi150 ceph-mon[36807]: pgmap v85: 129 pgs: 129 active+clean; 119 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 228 op/s 2024-03-04T09:01:40.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:39 smithi158 ceph-mon[37241]: pgmap v85: 129 pgs: 129 active+clean; 119 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 228 op/s 2024-03-04T09:01:40.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:39 smithi071 ceph-mon[32789]: pgmap v85: 129 pgs: 129 active+clean; 119 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 228 op/s 2024-03-04T09:01:41.144 INFO:tasks.workunit.client.0.smithi071.stdout: 1 223839 13.92 MB/sec warmup 88 sec latency 81.615 ms 2024-03-04T09:01:42.144 INFO:tasks.workunit.client.0.smithi071.stdout: 1 226409 13.91 MB/sec warmup 89 sec latency 23.308 ms 2024-03-04T09:01:42.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:42 smithi150 ceph-mon[36807]: pgmap v86: 129 pgs: 129 active+clean; 127 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.3 MiB/s wr, 309 op/s 2024-03-04T09:01:42.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:42 smithi158 ceph-mon[37241]: pgmap v86: 129 pgs: 129 active+clean; 127 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.3 MiB/s wr, 309 op/s 2024-03-04T09:01:42.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:42 smithi071 ceph-mon[32789]: pgmap v86: 129 pgs: 129 active+clean; 127 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.3 MiB/s wr, 309 op/s 2024-03-04T09:01:43.144 INFO:tasks.workunit.client.0.smithi071.stdout: 1 228715 13.89 MB/sec warmup 90 sec latency 6.896 ms 2024-03-04T09:01:44.145 INFO:tasks.workunit.client.0.smithi071.stdout: 1 228748 13.74 MB/sec warmup 91 sec latency 993.391 ms 2024-03-04T09:01:44.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:44 smithi150 ceph-mon[36807]: pgmap v87: 129 pgs: 129 active+clean; 127 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 251 op/s 2024-03-04T09:01:44.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:44 smithi158 ceph-mon[37241]: pgmap v87: 129 pgs: 129 active+clean; 127 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 251 op/s 2024-03-04T09:01:44.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:44 smithi071 ceph-mon[32789]: pgmap v87: 129 pgs: 129 active+clean; 127 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 251 op/s 2024-03-04T09:01:45.145 INFO:tasks.workunit.client.0.smithi071.stdout: 1 228748 13.59 MB/sec warmup 92 sec latency 1993.483 ms 2024-03-04T09:01:46.145 INFO:tasks.workunit.client.0.smithi071.stdout: 1 228748 13.44 MB/sec warmup 93 sec latency 2993.543 ms 2024-03-04T09:01:47.145 INFO:tasks.workunit.client.0.smithi071.stdout: 1 228748 13.30 MB/sec warmup 94 sec latency 3993.622 ms 2024-03-04T09:01:48.145 INFO:tasks.workunit.client.0.smithi071.stdout: 1 228748 13.16 MB/sec warmup 95 sec latency 4993.719 ms 2024-03-04T09:01:49.145 INFO:tasks.workunit.client.0.smithi071.stdout: 1 228748 13.02 MB/sec warmup 96 sec latency 5993.771 ms 2024-03-04T09:01:49.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:49 smithi071 ceph-mon[32789]: pgmap v88: 129 pgs: 129 active+clean; 127 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 209 op/s 2024-03-04T09:01:49.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:49 smithi150 ceph-mon[36807]: pgmap v88: 129 pgs: 129 active+clean; 127 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 209 op/s 2024-03-04T09:01:49.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:49 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-c[37224]: 2024-03-04T09:01:49.091+0000 7f99b6d7d700 -1 mon.c@1(peon).paxos(paxos updating c 754..1413) lease_expire from mon.0 v2:172.21.15.71:3300/0 is 0.084837236s seconds in the past; mons are probably laggy (or possibly clocks are too skewed) 2024-03-04T09:01:49.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:49 smithi158 ceph-mon[37241]: pgmap v88: 129 pgs: 129 active+clean; 127 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 209 op/s 2024-03-04T09:01:50.145 INFO:tasks.workunit.client.0.smithi071.stdout: 1 231066 13.03 MB/sec warmup 97 sec latency 6036.414 ms 2024-03-04T09:01:50.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:50 smithi071 ceph-mon[32789]: pgmap v89: 129 pgs: 129 active+clean; 124 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 233 op/s 2024-03-04T09:01:50.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:50 smithi071 ceph-mon[32789]: pgmap v90: 129 pgs: 129 active+clean; 124 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 125 op/s 2024-03-04T09:01:50.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:50 smithi150 ceph-mon[36807]: pgmap v89: 129 pgs: 129 active+clean; 124 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 233 op/s 2024-03-04T09:01:50.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:50 smithi150 ceph-mon[36807]: pgmap v90: 129 pgs: 129 active+clean; 124 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 125 op/s 2024-03-04T09:01:50.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:50 smithi158 ceph-mon[37241]: pgmap v89: 129 pgs: 129 active+clean; 124 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 233 op/s 2024-03-04T09:01:50.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:50 smithi158 ceph-mon[37241]: pgmap v90: 129 pgs: 129 active+clean; 124 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 125 op/s 2024-03-04T09:01:51.145 INFO:tasks.workunit.client.0.smithi071.stdout: 1 233519 13.03 MB/sec warmup 98 sec latency 69.604 ms 2024-03-04T09:01:51.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:51 smithi071 ceph-mon[32789]: pgmap v91: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 210 op/s 2024-03-04T09:01:51.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:51 smithi150 ceph-mon[36807]: pgmap v91: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 210 op/s 2024-03-04T09:01:51.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:51 smithi158 ceph-mon[37241]: pgmap v91: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 210 op/s 2024-03-04T09:01:52.145 INFO:tasks.workunit.client.0.smithi071.stdout: 1 235405 12.98 MB/sec warmup 99 sec latency 22.754 ms 2024-03-04T09:01:53.145 INFO:tasks.workunit.client.0.smithi071.stdout: 1 238385 13.02 MB/sec warmup 100 sec latency 20.405 ms 2024-03-04T09:01:54.145 INFO:tasks.workunit.client.0.smithi071.stdout: 1 241513 13.08 MB/sec warmup 101 sec latency 19.610 ms 2024-03-04T09:01:54.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:53 smithi150 ceph-mon[36807]: pgmap v92: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 129 op/s 2024-03-04T09:01:54.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:53 smithi158 ceph-mon[37241]: pgmap v92: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 129 op/s 2024-03-04T09:01:54.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:53 smithi071 ceph-mon[32789]: pgmap v92: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 129 op/s 2024-03-04T09:01:55.147 INFO:tasks.workunit.client.0.smithi071.stdout: 1 244511 13.10 MB/sec warmup 102 sec latency 7.093 ms 2024-03-04T09:01:56.145 INFO:tasks.workunit.client.0.smithi071.stdout: 1 246395 13.06 MB/sec warmup 103 sec latency 86.672 ms 2024-03-04T09:01:56.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:55 smithi150 ceph-mon[36807]: pgmap v93: 129 pgs: 129 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 191 op/s 2024-03-04T09:01:56.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:55 smithi158 ceph-mon[37241]: pgmap v93: 129 pgs: 129 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 191 op/s 2024-03-04T09:01:56.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:55 smithi071 ceph-mon[32789]: pgmap v93: 129 pgs: 129 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 191 op/s 2024-03-04T09:01:57.145 INFO:tasks.workunit.client.0.smithi071.stdout: 1 248925 13.07 MB/sec warmup 104 sec latency 24.537 ms 2024-03-04T09:01:58.145 INFO:tasks.workunit.client.0.smithi071.stdout: 1 252091 13.13 MB/sec warmup 105 sec latency 12.447 ms 2024-03-04T09:01:58.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:57 smithi150 ceph-mon[36807]: pgmap v94: 129 pgs: 129 active+clean; 133 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 233 op/s 2024-03-04T09:01:58.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:57 smithi158 ceph-mon[37241]: pgmap v94: 129 pgs: 129 active+clean; 133 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 233 op/s 2024-03-04T09:01:58.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:57 smithi071 ceph-mon[32789]: pgmap v94: 129 pgs: 129 active+clean; 133 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 233 op/s 2024-03-04T09:01:59.145 INFO:tasks.workunit.client.0.smithi071.stdout: 1 254715 13.11 MB/sec warmup 106 sec latency 11.028 ms 2024-03-04T09:02:00.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 256890 13.11 MB/sec warmup 107 sec latency 22.501 ms 2024-03-04T09:02:00.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:01:59 smithi150 ceph-mon[36807]: pgmap v95: 129 pgs: 129 active+clean; 133 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 209 op/s 2024-03-04T09:02:00.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:01:59 smithi158 ceph-mon[37241]: pgmap v95: 129 pgs: 129 active+clean; 133 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 209 op/s 2024-03-04T09:02:00.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:01:59 smithi071 ceph-mon[32789]: pgmap v95: 129 pgs: 129 active+clean; 133 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 209 op/s 2024-03-04T09:02:01.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 258678 13.07 MB/sec warmup 108 sec latency 108.282 ms 2024-03-04T09:02:02.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 259948 13.01 MB/sec warmup 109 sec latency 40.711 ms 2024-03-04T09:02:02.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:01 smithi150 ceph-mon[36807]: pgmap v96: 129 pgs: 129 active+clean; 136 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 283 op/s 2024-03-04T09:02:02.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:01 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:02:02.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:01 smithi158 ceph-mon[37241]: pgmap v96: 129 pgs: 129 active+clean; 136 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 283 op/s 2024-03-04T09:02:02.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:01 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:02:02.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:01 smithi071 ceph-mon[32789]: pgmap v96: 129 pgs: 129 active+clean; 136 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 283 op/s 2024-03-04T09:02:02.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:01 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:02:03.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 262875 13.05 MB/sec warmup 110 sec latency 82.125 ms 2024-03-04T09:02:03.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:02 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:02:03.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:02 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:02:03.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:02 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:02:03.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:02 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:02:03.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:02 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:02:03.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:02 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:02:03.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:02 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:02:03.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:02 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:02:03.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:02 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:02:04.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 265708 13.06 MB/sec warmup 111 sec latency 14.154 ms 2024-03-04T09:02:04.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:03 smithi150 ceph-mon[36807]: pgmap v97: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 262 op/s 2024-03-04T09:02:04.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:03 smithi158 ceph-mon[37241]: pgmap v97: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 262 op/s 2024-03-04T09:02:04.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:03 smithi071 ceph-mon[32789]: pgmap v97: 129 pgs: 129 active+clean; 141 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 262 op/s 2024-03-04T09:02:05.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 267836 13.04 MB/sec warmup 112 sec latency 15.214 ms 2024-03-04T09:02:06.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 270172 13.05 MB/sec warmup 113 sec latency 45.999 ms 2024-03-04T09:02:06.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:05 smithi150 ceph-mon[36807]: pgmap v98: 129 pgs: 129 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.1 MiB/s wr, 311 op/s 2024-03-04T09:02:06.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:05 smithi158 ceph-mon[37241]: pgmap v98: 129 pgs: 129 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.1 MiB/s wr, 311 op/s 2024-03-04T09:02:06.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:05 smithi071 ceph-mon[32789]: pgmap v98: 129 pgs: 129 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.1 MiB/s wr, 311 op/s 2024-03-04T09:02:07.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 273155 13.09 MB/sec warmup 114 sec latency 44.347 ms 2024-03-04T09:02:08.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 275709 13.06 MB/sec warmup 115 sec latency 8.222 ms 2024-03-04T09:02:08.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:07 smithi150 ceph-mon[36807]: pgmap v99: 129 pgs: 129 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 306 op/s 2024-03-04T09:02:08.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:07 smithi158 ceph-mon[37241]: pgmap v99: 129 pgs: 129 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 306 op/s 2024-03-04T09:02:08.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:07 smithi071 ceph-mon[32789]: pgmap v99: 129 pgs: 129 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 306 op/s 2024-03-04T09:02:09.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 277914 13.06 MB/sec warmup 116 sec latency 11.400 ms 2024-03-04T09:02:10.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 280910 13.09 MB/sec warmup 117 sec latency 21.731 ms 2024-03-04T09:02:10.223 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:09 smithi071 ceph-mon[32789]: pgmap v100: 129 pgs: 129 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 264 op/s 2024-03-04T09:02:10.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:09 smithi150 ceph-mon[36807]: pgmap v100: 129 pgs: 129 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 264 op/s 2024-03-04T09:02:10.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:09 smithi158 ceph-mon[37241]: pgmap v100: 129 pgs: 129 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 264 op/s 2024-03-04T09:02:11.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 283857 13.13 MB/sec warmup 118 sec latency 55.229 ms 2024-03-04T09:02:12.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 285610 13.08 MB/sec warmup 119 sec latency 105.844 ms 2024-03-04T09:02:12.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:11 smithi150 ceph-mon[36807]: pgmap v101: 129 pgs: 129 active+clean; 148 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.5 MiB/s wr, 370 op/s 2024-03-04T09:02:12.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:11 smithi158 ceph-mon[37241]: pgmap v101: 129 pgs: 129 active+clean; 148 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.5 MiB/s wr, 370 op/s 2024-03-04T09:02:12.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:11 smithi071 ceph-mon[32789]: pgmap v101: 129 pgs: 129 active+clean; 148 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.5 MiB/s wr, 370 op/s 2024-03-04T09:02:14.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 291282 18.30 MB/sec execute 1 sec latency 16.853 ms 2024-03-04T09:02:14.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:13 smithi150 ceph-mon[36807]: pgmap v102: 129 pgs: 129 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 306 op/s 2024-03-04T09:02:14.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:13 smithi158 ceph-mon[37241]: pgmap v102: 129 pgs: 129 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 306 op/s 2024-03-04T09:02:14.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:13 smithi071 ceph-mon[32789]: pgmap v102: 129 pgs: 129 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 306 op/s 2024-03-04T09:02:15.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 294364 17.85 MB/sec execute 2 sec latency 22.174 ms 2024-03-04T09:02:16.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 296767 14.77 MB/sec execute 3 sec latency 11.555 ms 2024-03-04T09:02:16.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:15 smithi150 ceph-mon[36807]: pgmap v103: 129 pgs: 129 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 282 op/s 2024-03-04T09:02:16.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:15 smithi158 ceph-mon[37241]: pgmap v103: 129 pgs: 129 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 282 op/s 2024-03-04T09:02:16.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:15 smithi071 ceph-mon[32789]: pgmap v103: 129 pgs: 129 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 282 op/s 2024-03-04T09:02:17.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 298662 14.20 MB/sec execute 4 sec latency 25.245 ms 2024-03-04T09:02:18.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 301768 15.07 MB/sec execute 5 sec latency 17.926 ms 2024-03-04T09:02:18.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:17 smithi150 ceph-mon[36807]: pgmap v104: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 298 op/s 2024-03-04T09:02:18.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:17 smithi158 ceph-mon[37241]: pgmap v104: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 298 op/s 2024-03-04T09:02:18.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:17 smithi071 ceph-mon[32789]: pgmap v104: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 298 op/s 2024-03-04T09:02:19.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 304399 14.49 MB/sec execute 6 sec latency 13.621 ms 2024-03-04T09:02:20.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 306553 14.27 MB/sec execute 7 sec latency 17.988 ms 2024-03-04T09:02:20.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:19 smithi150 ceph-mon[36807]: pgmap v105: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 221 op/s 2024-03-04T09:02:20.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:19 smithi158 ceph-mon[37241]: pgmap v105: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 221 op/s 2024-03-04T09:02:20.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:19 smithi071 ceph-mon[32789]: pgmap v105: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 221 op/s 2024-03-04T09:02:21.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 309169 14.24 MB/sec execute 8 sec latency 27.086 ms 2024-03-04T09:02:22.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 312121 14.59 MB/sec execute 9 sec latency 49.632 ms 2024-03-04T09:02:22.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:21 smithi150 ceph-mon[36807]: pgmap v106: 129 pgs: 129 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 266 op/s 2024-03-04T09:02:22.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:21 smithi158 ceph-mon[37241]: pgmap v106: 129 pgs: 129 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 266 op/s 2024-03-04T09:02:22.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:21 smithi071 ceph-mon[32789]: pgmap v106: 129 pgs: 129 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 266 op/s 2024-03-04T09:02:23.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 314533 14.00 MB/sec execute 10 sec latency 7.956 ms 2024-03-04T09:02:24.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 316707 13.92 MB/sec execute 11 sec latency 22.790 ms 2024-03-04T09:02:24.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:23 smithi150 ceph-mon[36807]: pgmap v107: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 230 op/s 2024-03-04T09:02:24.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:23 smithi158 ceph-mon[37241]: pgmap v107: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 230 op/s 2024-03-04T09:02:24.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:23 smithi071 ceph-mon[32789]: pgmap v107: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 230 op/s 2024-03-04T09:02:25.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 319798 14.27 MB/sec execute 12 sec latency 9.159 ms 2024-03-04T09:02:26.146 INFO:tasks.workunit.client.0.smithi071.stdout: 1 322863 14.53 MB/sec execute 13 sec latency 14.763 ms 2024-03-04T09:02:26.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:25 smithi150 ceph-mon[36807]: pgmap v108: 129 pgs: 129 active+clean; 166 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 264 op/s 2024-03-04T09:02:26.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:25 smithi158 ceph-mon[37241]: pgmap v108: 129 pgs: 129 active+clean; 166 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 264 op/s 2024-03-04T09:02:26.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:25 smithi071 ceph-mon[32789]: pgmap v108: 129 pgs: 129 active+clean; 166 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 264 op/s 2024-03-04T09:02:27.147 INFO:tasks.workunit.client.0.smithi071.stdout: 1 324016 13.75 MB/sec execute 14 sec latency 119.822 ms 2024-03-04T09:02:28.147 INFO:tasks.workunit.client.0.smithi071.stdout: 1 326885 13.97 MB/sec execute 15 sec latency 30.649 ms 2024-03-04T09:02:28.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:27 smithi150 ceph-mon[36807]: pgmap v109: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 256 op/s 2024-03-04T09:02:28.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:27 smithi158 ceph-mon[37241]: pgmap v109: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 256 op/s 2024-03-04T09:02:28.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:27 smithi071 ceph-mon[32789]: pgmap v109: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 256 op/s 2024-03-04T09:02:29.147 INFO:tasks.workunit.client.0.smithi071.stdout: 1 329966 14.20 MB/sec execute 16 sec latency 15.817 ms 2024-03-04T09:02:30.147 INFO:tasks.workunit.client.0.smithi071.stdout: 1 330325 13.45 MB/sec execute 17 sec latency 739.164 ms 2024-03-04T09:02:30.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:29 smithi150 ceph-mon[36807]: pgmap v110: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 190 op/s 2024-03-04T09:02:30.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:29 smithi158 ceph-mon[37241]: pgmap v110: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 190 op/s 2024-03-04T09:02:30.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:29 smithi071 ceph-mon[32789]: pgmap v110: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 190 op/s 2024-03-04T09:02:31.147 INFO:tasks.workunit.client.0.smithi071.stdout: 1 330325 12.70 MB/sec execute 18 sec latency 1739.213 ms 2024-03-04T09:02:32.147 INFO:tasks.workunit.client.0.smithi071.stdout: 1 330325 12.03 MB/sec execute 19 sec latency 2739.302 ms 2024-03-04T09:02:32.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:32 smithi158 ceph-mon[37241]: pgmap v111: 129 pgs: 129 active+clean; 175 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 284 op/s 2024-03-04T09:02:33.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:32 smithi150 ceph-mon[36807]: pgmap v111: 129 pgs: 129 active+clean; 175 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 284 op/s 2024-03-04T09:02:33.147 INFO:tasks.workunit.client.0.smithi071.stdout: 1 330829 11.47 MB/sec execute 20 sec latency 3368.064 ms 2024-03-04T09:02:33.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:32 smithi071 ceph-mon[32789]: pgmap v111: 129 pgs: 129 active+clean; 175 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.7 MiB/s wr, 284 op/s 2024-03-04T09:02:34.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:33 smithi150 ceph-mon[36807]: pgmap v112: 129 pgs: 129 active+clean; 175 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 256 op/s 2024-03-04T09:02:34.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:33 smithi158 ceph-mon[37241]: pgmap v112: 129 pgs: 129 active+clean; 175 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 256 op/s 2024-03-04T09:02:34.147 INFO:tasks.workunit.client.0.smithi071.stdout: 1 333830 11.80 MB/sec execute 21 sec latency 26.732 ms 2024-03-04T09:02:34.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:33 smithi071 ceph-mon[32789]: pgmap v112: 129 pgs: 129 active+clean; 175 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 256 op/s 2024-03-04T09:02:35.147 INFO:tasks.workunit.client.0.smithi071.stdout: 1 336981 12.06 MB/sec execute 22 sec latency 25.394 ms 2024-03-04T09:02:36.147 INFO:tasks.workunit.client.0.smithi071.stdout: 1 339431 11.91 MB/sec execute 23 sec latency 40.680 ms 2024-03-04T09:02:36.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:35 smithi150 ceph-mon[36807]: pgmap v113: 129 pgs: 129 active+clean; 177 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 211 op/s 2024-03-04T09:02:36.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:35 smithi158 ceph-mon[37241]: pgmap v113: 129 pgs: 129 active+clean; 177 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 211 op/s 2024-03-04T09:02:36.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:35 smithi071 ceph-mon[32789]: pgmap v113: 129 pgs: 129 active+clean; 177 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 211 op/s 2024-03-04T09:02:37.147 INFO:tasks.workunit.client.0.smithi071.stdout: 1 341256 11.93 MB/sec execute 24 sec latency 72.229 ms 2024-03-04T09:02:38.147 INFO:tasks.workunit.client.0.smithi071.stdout: 1 344292 12.19 MB/sec execute 25 sec latency 27.126 ms 2024-03-04T09:02:38.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:37 smithi150 ceph-mon[36807]: pgmap v114: 129 pgs: 129 active+clean; 179 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 222 op/s 2024-03-04T09:02:38.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:37 smithi158 ceph-mon[37241]: pgmap v114: 129 pgs: 129 active+clean; 179 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 222 op/s 2024-03-04T09:02:38.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:37 smithi071 ceph-mon[32789]: pgmap v114: 129 pgs: 129 active+clean; 179 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 222 op/s 2024-03-04T09:02:39.147 INFO:tasks.workunit.client.0.smithi071.stdout: 1 347033 12.22 MB/sec execute 26 sec latency 12.762 ms 2024-03-04T09:02:40.147 INFO:tasks.workunit.client.0.smithi071.stdout: 1 349248 12.20 MB/sec execute 27 sec latency 19.721 ms 2024-03-04T09:02:40.223 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:39 smithi071 ceph-mon[32789]: pgmap v115: 129 pgs: 129 active+clean; 179 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 198 op/s 2024-03-04T09:02:40.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:39 smithi150 ceph-mon[36807]: pgmap v115: 129 pgs: 129 active+clean; 179 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 198 op/s 2024-03-04T09:02:40.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:39 smithi158 ceph-mon[37241]: pgmap v115: 129 pgs: 129 active+clean; 179 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 198 op/s 2024-03-04T09:02:41.147 INFO:tasks.workunit.client.0.smithi071.stdout: 1 351868 12.27 MB/sec execute 28 sec latency 7.850 ms 2024-03-04T09:02:42.147 INFO:tasks.workunit.client.0.smithi071.stdout: 1 354709 12.44 MB/sec execute 29 sec latency 62.101 ms 2024-03-04T09:02:42.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:41 smithi150 ceph-mon[36807]: pgmap v116: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 257 op/s 2024-03-04T09:02:42.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:41 smithi158 ceph-mon[37241]: pgmap v116: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 257 op/s 2024-03-04T09:02:42.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:41 smithi071 ceph-mon[32789]: pgmap v116: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 257 op/s 2024-03-04T09:02:43.148 INFO:tasks.workunit.client.0.smithi071.stdout: 1 356464 12.26 MB/sec execute 30 sec latency 52.576 ms 2024-03-04T09:02:44.147 INFO:tasks.workunit.client.0.smithi071.stdout: 1 358987 12.32 MB/sec execute 31 sec latency 14.602 ms 2024-03-04T09:02:44.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:43 smithi150 ceph-mon[36807]: pgmap v117: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 218 op/s 2024-03-04T09:02:44.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:43 smithi158 ceph-mon[37241]: pgmap v117: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 218 op/s 2024-03-04T09:02:44.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:43 smithi071 ceph-mon[32789]: pgmap v117: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 218 op/s 2024-03-04T09:02:45.147 INFO:tasks.workunit.client.0.smithi071.stdout: 1 362027 12.51 MB/sec execute 32 sec latency 24.823 ms 2024-03-04T09:02:46.148 INFO:tasks.workunit.client.0.smithi071.stdout: 1 364720 12.52 MB/sec execute 33 sec latency 10.183 ms 2024-03-04T09:02:46.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:45 smithi150 ceph-mon[36807]: pgmap v118: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 213 op/s 2024-03-04T09:02:46.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:45 smithi158 ceph-mon[37241]: pgmap v118: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 213 op/s 2024-03-04T09:02:46.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:45 smithi071 ceph-mon[32789]: pgmap v118: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 213 op/s 2024-03-04T09:02:47.147 INFO:tasks.workunit.client.0.smithi071.stdout: 1 366408 12.40 MB/sec execute 34 sec latency 74.923 ms 2024-03-04T09:02:48.148 INFO:tasks.workunit.client.0.smithi071.stdout: 1 369331 12.54 MB/sec execute 35 sec latency 22.031 ms 2024-03-04T09:02:48.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:47 smithi150 ceph-mon[36807]: pgmap v119: 129 pgs: 129 active+clean; 193 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 285 op/s 2024-03-04T09:02:48.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:47 smithi158 ceph-mon[37241]: pgmap v119: 129 pgs: 129 active+clean; 193 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 285 op/s 2024-03-04T09:02:48.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:47 smithi071 ceph-mon[32789]: pgmap v119: 129 pgs: 129 active+clean; 193 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 285 op/s 2024-03-04T09:02:49.148 INFO:tasks.workunit.client.0.smithi071.stdout: 1 372462 12.68 MB/sec execute 36 sec latency 7.633 ms 2024-03-04T09:02:50.148 INFO:tasks.workunit.client.0.smithi071.stdout: 1 374990 12.60 MB/sec execute 37 sec latency 15.437 ms 2024-03-04T09:02:50.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:49 smithi150 ceph-mon[36807]: pgmap v120: 129 pgs: 129 active+clean; 192 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 237 op/s 2024-03-04T09:02:50.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:49 smithi158 ceph-mon[37241]: pgmap v120: 129 pgs: 129 active+clean; 192 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 237 op/s 2024-03-04T09:02:50.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:49 smithi071 ceph-mon[32789]: pgmap v120: 129 pgs: 129 active+clean; 192 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 237 op/s 2024-03-04T09:02:51.148 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377377 12.69 MB/sec execute 38 sec latency 10.726 ms 2024-03-04T09:02:52.148 INFO:tasks.workunit.client.0.smithi071.stdout: 1 379949 12.73 MB/sec execute 39 sec latency 82.798 ms 2024-03-04T09:02:52.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:51 smithi150 ceph-mon[36807]: pgmap v121: 129 pgs: 129 active+clean; 194 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 297 op/s 2024-03-04T09:02:52.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:51 smithi158 ceph-mon[37241]: pgmap v121: 129 pgs: 129 active+clean; 194 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 297 op/s 2024-03-04T09:02:52.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:51 smithi071 ceph-mon[32789]: pgmap v121: 129 pgs: 129 active+clean; 194 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.4 MiB/s wr, 297 op/s 2024-03-04T09:02:53.148 INFO:tasks.workunit.client.0.smithi071.stdout: 1 380747 12.46 MB/sec execute 40 sec latency 391.626 ms 2024-03-04T09:02:54.148 INFO:tasks.workunit.client.0.smithi071.stdout: 1 380747 12.16 MB/sec execute 41 sec latency 1391.747 ms 2024-03-04T09:02:54.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:53 smithi150 ceph-mon[36807]: pgmap v122: 129 pgs: 129 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.1 MiB/s wr, 287 op/s 2024-03-04T09:02:54.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:53 smithi158 ceph-mon[37241]: pgmap v122: 129 pgs: 129 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.1 MiB/s wr, 287 op/s 2024-03-04T09:02:54.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:53 smithi071 ceph-mon[32789]: pgmap v122: 129 pgs: 129 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.1 MiB/s wr, 287 op/s 2024-03-04T09:02:55.148 INFO:tasks.workunit.client.0.smithi071.stdout: 1 380747 11.87 MB/sec execute 42 sec latency 2391.836 ms 2024-03-04T09:02:56.150 INFO:tasks.workunit.client.0.smithi071.stdout: 1 380747 11.59 MB/sec execute 43 sec latency 3391.959 ms 2024-03-04T09:02:57.148 INFO:tasks.workunit.client.0.smithi071.stdout: 1 380747 11.33 MB/sec execute 44 sec latency 4391.993 ms 2024-03-04T09:02:58.148 INFO:tasks.workunit.client.0.smithi071.stdout: 1 380747 11.08 MB/sec execute 45 sec latency 5392.081 ms 2024-03-04T09:02:59.148 INFO:tasks.workunit.client.0.smithi071.stdout: 1 380747 10.84 MB/sec execute 46 sec latency 6392.177 ms 2024-03-04T09:03:00.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:02:59 smithi150 ceph-mon[36807]: pgmap v123: 129 pgs: 129 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 233 op/s 2024-03-04T09:03:00.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:59 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-c[37224]: 2024-03-04T09:02:59.712+0000 7f99b6d7d700 -1 mon.c@1(peon).paxos(paxos updating c 754..1483) lease_expire from mon.0 v2:172.21.15.71:3300/0 is 0.740680337s seconds in the past; mons are probably laggy (or possibly clocks are too skewed) 2024-03-04T09:03:00.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:02:59 smithi158 ceph-mon[37241]: pgmap v123: 129 pgs: 129 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 233 op/s 2024-03-04T09:03:00.148 INFO:tasks.workunit.client.0.smithi071.stdout: 1 381953 10.71 MB/sec execute 47 sec latency 6991.540 ms 2024-03-04T09:03:00.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:02:59 smithi071 ceph-mon[32789]: pgmap v123: 129 pgs: 129 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 233 op/s 2024-03-04T09:03:01.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:00 smithi150 ceph-mon[36807]: pgmap v124: 129 pgs: 129 active+clean; 201 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 261 op/s 2024-03-04T09:03:01.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:00 smithi150 ceph-mon[36807]: pgmap v125: 129 pgs: 129 active+clean; 201 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 165 op/s 2024-03-04T09:03:01.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:00 smithi158 ceph-mon[37241]: pgmap v124: 129 pgs: 129 active+clean; 201 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 261 op/s 2024-03-04T09:03:01.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:00 smithi158 ceph-mon[37241]: pgmap v125: 129 pgs: 129 active+clean; 201 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 165 op/s 2024-03-04T09:03:01.148 INFO:tasks.workunit.client.0.smithi071.stdout: 1 383844 10.75 MB/sec execute 48 sec latency 49.579 ms 2024-03-04T09:03:01.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:00 smithi071 ceph-mon[32789]: pgmap v124: 129 pgs: 129 active+clean; 201 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.1 MiB/s wr, 261 op/s 2024-03-04T09:03:01.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:00 smithi071 ceph-mon[32789]: pgmap v125: 129 pgs: 129 active+clean; 201 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 165 op/s 2024-03-04T09:03:02.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:01 smithi150 ceph-mon[36807]: pgmap v126: 129 pgs: 129 active+clean; 200 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 205 op/s 2024-03-04T09:03:02.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:01 smithi158 ceph-mon[37241]: pgmap v126: 129 pgs: 129 active+clean; 200 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 205 op/s 2024-03-04T09:03:02.148 INFO:tasks.workunit.client.0.smithi071.stdout: 1 384668 10.62 MB/sec execute 49 sec latency 63.466 ms 2024-03-04T09:03:02.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:01 smithi071 ceph-mon[32789]: pgmap v126: 129 pgs: 129 active+clean; 200 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 205 op/s 2024-03-04T09:03:03.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:02 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:03:03.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:02 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:03:03.148 INFO:tasks.workunit.client.0.smithi071.stdout: 1 387101 10.69 MB/sec execute 50 sec latency 63.803 ms 2024-03-04T09:03:03.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:02 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:03:04.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:03 smithi150 ceph-mon[36807]: pgmap v127: 129 pgs: 129 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 145 op/s 2024-03-04T09:03:04.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:03 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:03:04.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:03 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:03:04.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:03 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:03:04.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:03 smithi158 ceph-mon[37241]: pgmap v127: 129 pgs: 129 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 145 op/s 2024-03-04T09:03:04.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:03 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:03:04.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:03 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:03:04.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:03 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:03:04.149 INFO:tasks.workunit.client.0.smithi071.stdout: 1 390025 10.77 MB/sec execute 51 sec latency 23.440 ms 2024-03-04T09:03:04.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:03 smithi071 ceph-mon[32789]: pgmap v127: 129 pgs: 129 active+clean; 199 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 145 op/s 2024-03-04T09:03:04.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:03 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:03:04.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:03 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:03:04.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:03 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:03:05.149 INFO:tasks.workunit.client.0.smithi071.stdout: 1 392360 10.76 MB/sec execute 52 sec latency 7.622 ms 2024-03-04T09:03:06.149 INFO:tasks.workunit.client.0.smithi071.stdout: 1 394556 10.82 MB/sec execute 53 sec latency 19.823 ms 2024-03-04T09:03:06.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:05 smithi150 ceph-mon[36807]: pgmap v128: 129 pgs: 129 active+clean; 205 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 167 op/s 2024-03-04T09:03:06.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:05 smithi158 ceph-mon[37241]: pgmap v128: 129 pgs: 129 active+clean; 205 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 167 op/s 2024-03-04T09:03:06.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:05 smithi071 ceph-mon[32789]: pgmap v128: 129 pgs: 129 active+clean; 205 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 167 op/s 2024-03-04T09:03:07.149 INFO:tasks.workunit.client.0.smithi071.stdout: 1 397613 10.96 MB/sec execute 54 sec latency 7.469 ms 2024-03-04T09:03:08.149 INFO:tasks.workunit.client.0.smithi071.stdout: 1 399678 10.89 MB/sec execute 55 sec latency 42.377 ms 2024-03-04T09:03:08.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:07 smithi150 ceph-mon[36807]: pgmap v129: 129 pgs: 129 active+clean; 206 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 210 op/s 2024-03-04T09:03:08.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:07 smithi158 ceph-mon[37241]: pgmap v129: 129 pgs: 129 active+clean; 206 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 210 op/s 2024-03-04T09:03:08.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:07 smithi071 ceph-mon[32789]: pgmap v129: 129 pgs: 129 active+clean; 206 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 210 op/s 2024-03-04T09:03:09.149 INFO:tasks.workunit.client.0.smithi071.stdout: 1 401771 10.93 MB/sec execute 56 sec latency 29.507 ms 2024-03-04T09:03:10.149 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404771 11.06 MB/sec execute 57 sec latency 22.007 ms 2024-03-04T09:03:10.223 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:09 smithi071 ceph-mon[32789]: pgmap v130: 129 pgs: 129 active+clean; 206 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 169 op/s 2024-03-04T09:03:10.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:09 smithi150 ceph-mon[36807]: pgmap v130: 129 pgs: 129 active+clean; 206 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 169 op/s 2024-03-04T09:03:10.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:09 smithi158 ceph-mon[37241]: pgmap v130: 129 pgs: 129 active+clean; 206 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 169 op/s 2024-03-04T09:03:11.149 INFO:tasks.workunit.client.0.smithi071.stdout: 1 407763 11.12 MB/sec execute 58 sec latency 10.540 ms 2024-03-04T09:03:12.149 INFO:tasks.workunit.client.0.smithi071.stdout: 1 409775 11.10 MB/sec execute 59 sec latency 26.402 ms 2024-03-04T09:03:12.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:11 smithi150 ceph-mon[36807]: pgmap v131: 129 pgs: 129 active+clean; 214 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 274 op/s 2024-03-04T09:03:12.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:11 smithi158 ceph-mon[37241]: pgmap v131: 129 pgs: 129 active+clean; 214 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 274 op/s 2024-03-04T09:03:12.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:11 smithi071 ceph-mon[32789]: pgmap v131: 129 pgs: 129 active+clean; 214 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 274 op/s 2024-03-04T09:03:13.149 INFO:tasks.workunit.client.0.smithi071.stdout: 1 411949 11.14 MB/sec execute 60 sec latency 89.158 ms 2024-03-04T09:03:14.149 INFO:tasks.workunit.client.0.smithi071.stdout: 1 412901 11.04 MB/sec execute 61 sec latency 618.499 ms 2024-03-04T09:03:14.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:13 smithi150 ceph-mon[36807]: pgmap v132: 129 pgs: 129 active+clean; 213 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 233 op/s 2024-03-04T09:03:14.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:13 smithi158 ceph-mon[37241]: pgmap v132: 129 pgs: 129 active+clean; 213 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 233 op/s 2024-03-04T09:03:14.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:13 smithi071 ceph-mon[32789]: pgmap v132: 129 pgs: 129 active+clean; 213 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 233 op/s 2024-03-04T09:03:15.149 INFO:tasks.workunit.client.0.smithi071.stdout: 1 412901 10.86 MB/sec execute 62 sec latency 1618.549 ms 2024-03-04T09:03:16.149 INFO:tasks.workunit.client.0.smithi071.stdout: 1 412901 10.69 MB/sec execute 63 sec latency 2618.643 ms 2024-03-04T09:03:17.149 INFO:tasks.workunit.client.0.smithi071.stdout: 1 412901 10.52 MB/sec execute 64 sec latency 3618.798 ms 2024-03-04T09:03:18.149 INFO:tasks.workunit.client.0.smithi071.stdout: 1 412901 10.36 MB/sec execute 65 sec latency 4618.862 ms 2024-03-04T09:03:18.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:18 smithi150 ceph-mon[36807]: pgmap v133: 129 pgs: 129 active+clean; 213 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 225 op/s 2024-03-04T09:03:18.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:18 smithi158 ceph-mon[37241]: pgmap v133: 129 pgs: 129 active+clean; 213 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 225 op/s 2024-03-04T09:03:18.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:18 smithi071 ceph-mon[32789]: pgmap v133: 129 pgs: 129 active+clean; 213 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 225 op/s 2024-03-04T09:03:19.149 INFO:tasks.workunit.client.0.smithi071.stdout: 1 414680 10.33 MB/sec execute 66 sec latency 4777.598 ms 2024-03-04T09:03:19.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:19 smithi150 ceph-mon[36807]: pgmap v134: 129 pgs: 129 active+clean; 219 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 183 op/s 2024-03-04T09:03:19.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:19 smithi150 ceph-mon[36807]: pgmap v135: 129 pgs: 129 active+clean; 227 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 196 op/s 2024-03-04T09:03:19.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:19 smithi158 ceph-mon[37241]: pgmap v134: 129 pgs: 129 active+clean; 219 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 183 op/s 2024-03-04T09:03:19.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:19 smithi158 ceph-mon[37241]: pgmap v135: 129 pgs: 129 active+clean; 227 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 196 op/s 2024-03-04T09:03:19.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:19 smithi071 ceph-mon[32789]: pgmap v134: 129 pgs: 129 active+clean; 219 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 183 op/s 2024-03-04T09:03:19.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:19 smithi071 ceph-mon[32789]: pgmap v135: 129 pgs: 129 active+clean; 227 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 196 op/s 2024-03-04T09:03:20.149 INFO:tasks.workunit.client.0.smithi071.stdout: 1 416901 10.34 MB/sec execute 67 sec latency 9.087 ms 2024-03-04T09:03:21.150 INFO:tasks.workunit.client.0.smithi071.stdout: 1 419272 10.40 MB/sec execute 68 sec latency 23.732 ms 2024-03-04T09:03:22.150 INFO:tasks.workunit.client.0.smithi071.stdout: 1 422216 10.50 MB/sec execute 69 sec latency 85.482 ms 2024-03-04T09:03:22.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:21 smithi150 ceph-mon[36807]: pgmap v136: 129 pgs: 129 active+clean; 227 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 239 op/s 2024-03-04T09:03:22.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:21 smithi158 ceph-mon[37241]: pgmap v136: 129 pgs: 129 active+clean; 227 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 239 op/s 2024-03-04T09:03:22.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:21 smithi071 ceph-mon[32789]: pgmap v136: 129 pgs: 129 active+clean; 227 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 239 op/s 2024-03-04T09:03:23.150 INFO:tasks.workunit.client.0.smithi071.stdout: 1 423758 10.44 MB/sec execute 70 sec latency 82.730 ms 2024-03-04T09:03:24.150 INFO:tasks.workunit.client.0.smithi071.stdout: 1 426357 10.50 MB/sec execute 71 sec latency 19.055 ms 2024-03-04T09:03:24.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:23 smithi150 ceph-mon[36807]: pgmap v137: 129 pgs: 129 active+clean; 225 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 157 op/s 2024-03-04T09:03:24.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:23 smithi158 ceph-mon[37241]: pgmap v137: 129 pgs: 129 active+clean; 225 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 157 op/s 2024-03-04T09:03:24.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:23 smithi071 ceph-mon[32789]: pgmap v137: 129 pgs: 129 active+clean; 225 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 157 op/s 2024-03-04T09:03:25.150 INFO:tasks.workunit.client.0.smithi071.stdout: 1 429515 10.61 MB/sec execute 72 sec latency 25.488 ms 2024-03-04T09:03:26.150 INFO:tasks.workunit.client.0.smithi071.stdout: 1 431988 10.62 MB/sec execute 73 sec latency 17.039 ms 2024-03-04T09:03:26.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:25 smithi150 ceph-mon[36807]: pgmap v138: 129 pgs: 129 active+clean; 226 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 178 op/s 2024-03-04T09:03:26.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:25 smithi158 ceph-mon[37241]: pgmap v138: 129 pgs: 129 active+clean; 226 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 178 op/s 2024-03-04T09:03:26.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:25 smithi071 ceph-mon[32789]: pgmap v138: 129 pgs: 129 active+clean; 226 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 178 op/s 2024-03-04T09:03:27.150 INFO:tasks.workunit.client.0.smithi071.stdout: 1 434190 10.65 MB/sec execute 74 sec latency 15.474 ms 2024-03-04T09:03:28.150 INFO:tasks.workunit.client.0.smithi071.stdout: 1 436979 10.70 MB/sec execute 75 sec latency 7.390 ms 2024-03-04T09:03:28.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:27 smithi150 ceph-mon[36807]: pgmap v139: 129 pgs: 129 active+clean; 235 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.8 MiB/s wr, 271 op/s 2024-03-04T09:03:28.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:27 smithi158 ceph-mon[37241]: pgmap v139: 129 pgs: 129 active+clean; 235 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.8 MiB/s wr, 271 op/s 2024-03-04T09:03:28.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:27 smithi071 ceph-mon[32789]: pgmap v139: 129 pgs: 129 active+clean; 235 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.8 MiB/s wr, 271 op/s 2024-03-04T09:03:29.150 INFO:tasks.workunit.client.0.smithi071.stdout: 1 439940 10.79 MB/sec execute 76 sec latency 28.604 ms 2024-03-04T09:03:30.150 INFO:tasks.workunit.client.0.smithi071.stdout: 1 442334 10.77 MB/sec execute 77 sec latency 27.571 ms 2024-03-04T09:03:30.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:29 smithi150 ceph-mon[36807]: pgmap v140: 129 pgs: 129 active+clean; 235 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 283 op/s 2024-03-04T09:03:30.301 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:29 smithi158 ceph-mon[37241]: pgmap v140: 129 pgs: 129 active+clean; 235 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 283 op/s 2024-03-04T09:03:30.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:29 smithi071 ceph-mon[32789]: pgmap v140: 129 pgs: 129 active+clean; 235 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 283 op/s 2024-03-04T09:03:31.150 INFO:tasks.workunit.client.0.smithi071.stdout: 1 444512 10.80 MB/sec execute 78 sec latency 10.774 ms 2024-03-04T09:03:32.150 INFO:tasks.workunit.client.0.smithi071.stdout: 1 447482 10.88 MB/sec execute 79 sec latency 9.971 ms 2024-03-04T09:03:32.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:31 smithi150 ceph-mon[36807]: pgmap v141: 129 pgs: 129 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 238 op/s 2024-03-04T09:03:32.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:31 smithi158 ceph-mon[37241]: pgmap v141: 129 pgs: 129 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 238 op/s 2024-03-04T09:03:32.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:31 smithi071 ceph-mon[32789]: pgmap v141: 129 pgs: 129 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 238 op/s 2024-03-04T09:03:33.150 INFO:tasks.workunit.client.0.smithi071.stdout: 1 450489 10.96 MB/sec execute 80 sec latency 25.087 ms 2024-03-04T09:03:33.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:33 smithi150 ceph-mon[36807]: Health check failed: Slow OSD heartbeats on back (longest 1025.744ms) (OSD_SLOW_PING_TIME_BACK) 2024-03-04T09:03:33.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:33 smithi150 ceph-mon[36807]: Health check failed: Slow OSD heartbeats on front (longest 1030.137ms) (OSD_SLOW_PING_TIME_FRONT) 2024-03-04T09:03:33.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:33 smithi158 ceph-mon[37241]: Health check failed: Slow OSD heartbeats on back (longest 1025.744ms) (OSD_SLOW_PING_TIME_BACK) 2024-03-04T09:03:33.304 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:33 smithi158 ceph-mon[37241]: Health check failed: Slow OSD heartbeats on front (longest 1030.137ms) (OSD_SLOW_PING_TIME_FRONT) 2024-03-04T09:03:33.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:33 smithi071 ceph-mon[32789]: Health check failed: Slow OSD heartbeats on back (longest 1025.744ms) (OSD_SLOW_PING_TIME_BACK) 2024-03-04T09:03:33.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:33 smithi071 ceph-mon[32789]: Health check failed: Slow OSD heartbeats on front (longest 1030.137ms) (OSD_SLOW_PING_TIME_FRONT) 2024-03-04T09:03:34.150 INFO:tasks.workunit.client.0.smithi071.stdout: 1 452744 10.92 MB/sec execute 81 sec latency 25.951 ms 2024-03-04T09:03:34.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:34 smithi150 ceph-mon[36807]: pgmap v142: 129 pgs: 129 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 201 op/s 2024-03-04T09:03:34.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:34 smithi158 ceph-mon[37241]: pgmap v142: 129 pgs: 129 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 201 op/s 2024-03-04T09:03:34.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:34 smithi071 ceph-mon[32789]: pgmap v142: 129 pgs: 129 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 201 op/s 2024-03-04T09:03:35.150 INFO:tasks.workunit.client.0.smithi071.stdout: 1 454857 10.96 MB/sec execute 82 sec latency 15.628 ms 2024-03-04T09:03:36.150 INFO:tasks.workunit.client.0.smithi071.stdout: 1 457946 11.05 MB/sec execute 83 sec latency 12.114 ms 2024-03-04T09:03:36.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:36 smithi150 ceph-mon[36807]: pgmap v143: 129 pgs: 129 active+clean; 243 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 231 op/s 2024-03-04T09:03:36.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:36 smithi158 ceph-mon[37241]: pgmap v143: 129 pgs: 129 active+clean; 243 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 231 op/s 2024-03-04T09:03:36.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:36 smithi071 ceph-mon[32789]: pgmap v143: 129 pgs: 129 active+clean; 243 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 231 op/s 2024-03-04T09:03:37.150 INFO:tasks.workunit.client.0.smithi071.stdout: 1 459525 11.19 MB/sec execute 84 sec latency 61.859 ms 2024-03-04T09:03:38.150 INFO:tasks.workunit.client.0.smithi071.stdout: 1 462367 11.26 MB/sec execute 85 sec latency 52.111 ms 2024-03-04T09:03:38.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:38 smithi150 ceph-mon[36807]: pgmap v144: 129 pgs: 129 active+clean; 250 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 283 op/s 2024-03-04T09:03:38.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:38 smithi158 ceph-mon[37241]: pgmap v144: 129 pgs: 129 active+clean; 250 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 283 op/s 2024-03-04T09:03:38.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:38 smithi071 ceph-mon[32789]: pgmap v144: 129 pgs: 129 active+clean; 250 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 283 op/s 2024-03-04T09:03:39.150 INFO:tasks.workunit.client.0.smithi071.stdout: 1 464914 11.28 MB/sec execute 86 sec latency 63.674 ms 2024-03-04T09:03:40.150 INFO:tasks.workunit.client.0.smithi071.stdout: 1 467848 11.30 MB/sec execute 87 sec latency 8.459 ms 2024-03-04T09:03:40.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:40 smithi158 ceph-mon[37241]: pgmap v145: 129 pgs: 129 active+clean; 253 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 203 op/s 2024-03-04T09:03:40.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:40 smithi071 ceph-mon[32789]: pgmap v145: 129 pgs: 129 active+clean; 253 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 203 op/s 2024-03-04T09:03:40.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:40 smithi150 ceph-mon[36807]: pgmap v145: 129 pgs: 129 active+clean; 253 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 203 op/s 2024-03-04T09:03:41.150 INFO:tasks.workunit.client.0.smithi071.stdout: 1 469936 11.32 MB/sec execute 88 sec latency 24.035 ms 2024-03-04T09:03:42.151 INFO:tasks.workunit.client.0.smithi071.stdout: 1 472996 11.40 MB/sec execute 89 sec latency 7.946 ms 2024-03-04T09:03:42.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:42 smithi158 ceph-mon[37241]: pgmap v146: 129 pgs: 129 active+clean; 259 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 227 op/s 2024-03-04T09:03:42.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:42 smithi071 ceph-mon[32789]: pgmap v146: 129 pgs: 129 active+clean; 259 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 227 op/s 2024-03-04T09:03:42.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:42 smithi150 ceph-mon[36807]: pgmap v146: 129 pgs: 129 active+clean; 259 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 227 op/s 2024-03-04T09:03:43.151 INFO:tasks.workunit.client.0.smithi071.stdout: 1 476146 11.47 MB/sec execute 90 sec latency 7.515 ms 2024-03-04T09:03:44.151 INFO:tasks.workunit.client.0.smithi071.stdout: 1 477594 11.42 MB/sec execute 91 sec latency 50.620 ms 2024-03-04T09:03:44.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:44 smithi158 ceph-mon[37241]: pgmap v147: 129 pgs: 129 active+clean; 259 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 226 op/s 2024-03-04T09:03:44.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:44 smithi071 ceph-mon[32789]: pgmap v147: 129 pgs: 129 active+clean; 259 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 226 op/s 2024-03-04T09:03:44.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:44 smithi150 ceph-mon[36807]: pgmap v147: 129 pgs: 129 active+clean; 259 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 226 op/s 2024-03-04T09:03:45.150 INFO:tasks.workunit.client.0.smithi071.stdout: 1 480390 11.45 MB/sec execute 92 sec latency 10.559 ms 2024-03-04T09:03:45.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:45 smithi071 ceph-mon[32789]: pgmap v148: 129 pgs: 129 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 284 op/s 2024-03-04T09:03:45.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:45 smithi150 ceph-mon[36807]: pgmap v148: 129 pgs: 129 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 284 op/s 2024-03-04T09:03:45.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:45 smithi158 ceph-mon[37241]: pgmap v148: 129 pgs: 129 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 284 op/s 2024-03-04T09:03:46.151 INFO:tasks.workunit.client.0.smithi071.stdout: 1 483527 11.52 MB/sec execute 93 sec latency 11.698 ms 2024-03-04T09:03:47.150 INFO:tasks.workunit.client.0.smithi071.stdout: 1 486111 11.53 MB/sec execute 94 sec latency 38.600 ms 2024-03-04T09:03:48.151 INFO:tasks.workunit.client.0.smithi071.stdout: 1 488320 11.54 MB/sec execute 95 sec latency 8.856 ms 2024-03-04T09:03:48.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:47 smithi150 ceph-mon[36807]: pgmap v149: 129 pgs: 129 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 260 op/s 2024-03-04T09:03:48.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:47 smithi158 ceph-mon[37241]: pgmap v149: 129 pgs: 129 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 260 op/s 2024-03-04T09:03:48.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:47 smithi071 ceph-mon[32789]: pgmap v149: 129 pgs: 129 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 260 op/s 2024-03-04T09:03:49.151 INFO:tasks.workunit.client.0.smithi071.stdout: 1 490907 11.57 MB/sec execute 96 sec latency 25.642 ms 2024-03-04T09:03:50.151 INFO:tasks.workunit.client.0.smithi071.stdout: 1 494092 11.64 MB/sec execute 97 sec latency 8.377 ms 2024-03-04T09:03:50.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:49 smithi150 ceph-mon[36807]: pgmap v150: 129 pgs: 129 active+clean; 268 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 260 op/s 2024-03-04T09:03:50.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:49 smithi158 ceph-mon[37241]: pgmap v150: 129 pgs: 129 active+clean; 268 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 260 op/s 2024-03-04T09:03:50.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:49 smithi071 ceph-mon[32789]: pgmap v150: 129 pgs: 129 active+clean; 268 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 260 op/s 2024-03-04T09:03:51.151 INFO:tasks.workunit.client.0.smithi071.stdout: 1 496453 11.61 MB/sec execute 98 sec latency 16.231 ms 2024-03-04T09:03:52.151 INFO:tasks.workunit.client.0.smithi071.stdout: 1 498620 11.62 MB/sec execute 99 sec latency 37.794 ms 2024-03-04T09:03:52.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:51 smithi150 ceph-mon[36807]: pgmap v151: 129 pgs: 129 active+clean; 269 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 257 op/s 2024-03-04T09:03:52.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:51 smithi158 ceph-mon[37241]: pgmap v151: 129 pgs: 129 active+clean; 269 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 257 op/s 2024-03-04T09:03:52.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:51 smithi071 ceph-mon[32789]: pgmap v151: 129 pgs: 129 active+clean; 269 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 257 op/s 2024-03-04T09:03:53.152 INFO:tasks.workunit.client.0.smithi071.stdout: 1 501521 11.68 MB/sec execute 100 sec latency 7.089 ms 2024-03-04T09:03:54.151 INFO:tasks.workunit.client.0.smithi071.stdout: 1 503856 11.69 MB/sec execute 101 sec latency 71.649 ms 2024-03-04T09:03:54.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:53 smithi150 ceph-mon[36807]: pgmap v152: 129 pgs: 129 active+clean; 274 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 234 op/s 2024-03-04T09:03:54.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:53 smithi158 ceph-mon[37241]: pgmap v152: 129 pgs: 129 active+clean; 274 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 234 op/s 2024-03-04T09:03:54.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:53 smithi071 ceph-mon[32789]: pgmap v152: 129 pgs: 129 active+clean; 274 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 234 op/s 2024-03-04T09:03:55.151 INFO:tasks.workunit.client.0.smithi071.stdout: 1 506691 11.69 MB/sec execute 102 sec latency 8.541 ms 2024-03-04T09:03:56.151 INFO:tasks.workunit.client.0.smithi071.stdout: 1 509013 11.72 MB/sec execute 103 sec latency 7.307 ms 2024-03-04T09:03:56.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:55 smithi150 ceph-mon[36807]: pgmap v153: 129 pgs: 129 active+clean; 274 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 229 op/s 2024-03-04T09:03:56.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:55 smithi158 ceph-mon[37241]: pgmap v153: 129 pgs: 129 active+clean; 274 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 229 op/s 2024-03-04T09:03:56.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:55 smithi071 ceph-mon[32789]: pgmap v153: 129 pgs: 129 active+clean; 274 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 229 op/s 2024-03-04T09:03:57.151 INFO:tasks.workunit.client.0.smithi071.stdout: 1 511834 11.78 MB/sec execute 104 sec latency 99.440 ms 2024-03-04T09:03:58.151 INFO:tasks.workunit.client.0.smithi071.stdout: 1 514481 11.78 MB/sec execute 105 sec latency 8.638 ms 2024-03-04T09:03:58.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:57 smithi150 ceph-mon[36807]: pgmap v154: 129 pgs: 129 active+clean; 280 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 255 op/s 2024-03-04T09:03:58.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:57 smithi158 ceph-mon[37241]: pgmap v154: 129 pgs: 129 active+clean; 280 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 255 op/s 2024-03-04T09:03:58.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:57 smithi071 ceph-mon[32789]: pgmap v154: 129 pgs: 129 active+clean; 280 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 255 op/s 2024-03-04T09:03:59.151 INFO:tasks.workunit.client.0.smithi071.stdout: 1 516069 11.75 MB/sec execute 106 sec latency 89.545 ms 2024-03-04T09:04:00.151 INFO:tasks.workunit.client.0.smithi071.stdout: 1 519236 11.81 MB/sec execute 107 sec latency 6.483 ms 2024-03-04T09:04:00.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:03:59 smithi150 ceph-mon[36807]: pgmap v155: 129 pgs: 129 active+clean; 283 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 267 op/s 2024-03-04T09:04:00.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:03:59 smithi158 ceph-mon[37241]: pgmap v155: 129 pgs: 129 active+clean; 283 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 267 op/s 2024-03-04T09:04:00.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:03:59 smithi071 ceph-mon[32789]: pgmap v155: 129 pgs: 129 active+clean; 283 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 267 op/s 2024-03-04T09:04:01.151 INFO:tasks.workunit.client.0.smithi071.stdout: 1 521913 11.82 MB/sec execute 108 sec latency 54.964 ms 2024-03-04T09:04:02.151 INFO:tasks.workunit.client.0.smithi071.stdout: 1 522946 11.77 MB/sec execute 109 sec latency 45.930 ms 2024-03-04T09:04:02.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:01 smithi150 ceph-mon[36807]: pgmap v156: 129 pgs: 129 active+clean; 283 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 229 op/s 2024-03-04T09:04:02.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:01 smithi158 ceph-mon[37241]: pgmap v156: 129 pgs: 129 active+clean; 283 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 229 op/s 2024-03-04T09:04:02.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:01 smithi071 ceph-mon[32789]: pgmap v156: 129 pgs: 129 active+clean; 283 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 229 op/s 2024-03-04T09:04:03.151 INFO:tasks.workunit.client.0.smithi071.stdout: 1 525879 11.82 MB/sec execute 110 sec latency 43.385 ms 2024-03-04T09:04:04.151 INFO:tasks.workunit.client.0.smithi071.stdout: 1 527026 11.74 MB/sec execute 111 sec latency 121.054 ms 2024-03-04T09:04:04.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:03 smithi150 ceph-mon[36807]: pgmap v157: 129 pgs: 129 active+clean; 284 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 231 op/s 2024-03-04T09:04:04.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:03 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:04:04.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:03 smithi158 ceph-mon[37241]: pgmap v157: 129 pgs: 129 active+clean; 284 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 231 op/s 2024-03-04T09:04:04.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:03 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:04:04.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:03 smithi071 ceph-mon[32789]: pgmap v157: 129 pgs: 129 active+clean; 284 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 231 op/s 2024-03-04T09:04:04.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:03 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:04:05.152 INFO:tasks.workunit.client.0.smithi071.stdout: 1 529754 11.79 MB/sec execute 112 sec latency 32.409 ms 2024-03-04T09:04:05.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:04 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:04:05.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:04 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:04:05.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:04 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:04:05.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:04 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:04:05.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:04 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:04:05.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:04 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:04:05.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:04 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:04:05.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:04 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:04:05.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:04 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:04:06.151 INFO:tasks.workunit.client.0.smithi071.stdout: 1 533017 11.85 MB/sec execute 113 sec latency 6.386 ms 2024-03-04T09:04:06.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:05 smithi150 ceph-mon[36807]: pgmap v158: 129 pgs: 129 active+clean; 291 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 270 op/s 2024-03-04T09:04:06.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:05 smithi158 ceph-mon[37241]: pgmap v158: 129 pgs: 129 active+clean; 291 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 270 op/s 2024-03-04T09:04:06.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:05 smithi071 ceph-mon[32789]: pgmap v158: 129 pgs: 129 active+clean; 291 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 270 op/s 2024-03-04T09:04:07.151 INFO:tasks.workunit.client.0.smithi071.stdout: 1 535765 11.85 MB/sec execute 114 sec latency 8.422 ms 2024-03-04T09:04:08.151 INFO:tasks.workunit.client.0.smithi071.stdout: 1 537594 11.84 MB/sec execute 115 sec latency 20.902 ms 2024-03-04T09:04:08.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:07 smithi150 ceph-mon[36807]: pgmap v159: 129 pgs: 129 active+clean; 293 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.6 MiB/s wr, 320 op/s 2024-03-04T09:04:08.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:07 smithi158 ceph-mon[37241]: pgmap v159: 129 pgs: 129 active+clean; 293 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.6 MiB/s wr, 320 op/s 2024-03-04T09:04:08.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:07 smithi071 ceph-mon[32789]: pgmap v159: 129 pgs: 129 active+clean; 293 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.6 MiB/s wr, 320 op/s 2024-03-04T09:04:09.152 INFO:tasks.workunit.client.0.smithi071.stdout: 1 540107 11.87 MB/sec execute 116 sec latency 123.016 ms 2024-03-04T09:04:10.152 INFO:tasks.workunit.client.0.smithi071.stdout: 1 541869 11.82 MB/sec execute 117 sec latency 35.104 ms 2024-03-04T09:04:10.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:09 smithi150 ceph-mon[36807]: pgmap v160: 129 pgs: 129 active+clean; 297 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 270 op/s 2024-03-04T09:04:10.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:09 smithi158 ceph-mon[37241]: pgmap v160: 129 pgs: 129 active+clean; 297 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 270 op/s 2024-03-04T09:04:10.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:09 smithi071 ceph-mon[32789]: pgmap v160: 129 pgs: 129 active+clean; 297 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 270 op/s 2024-03-04T09:04:11.152 INFO:tasks.workunit.client.0.smithi071.stdout: 1 544231 11.84 MB/sec execute 118 sec latency 11.218 ms 2024-03-04T09:04:12.152 INFO:tasks.workunit.client.0.smithi071.stdout: 1 547307 11.90 MB/sec execute 119 sec latency 22.264 ms 2024-03-04T09:04:12.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:11 smithi150 ceph-mon[36807]: pgmap v161: 129 pgs: 129 active+clean; 306 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 274 op/s 2024-03-04T09:04:12.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:11 smithi158 ceph-mon[37241]: pgmap v161: 129 pgs: 129 active+clean; 306 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 274 op/s 2024-03-04T09:04:12.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:11 smithi071 ceph-mon[32789]: pgmap v161: 129 pgs: 129 active+clean; 306 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 274 op/s 2024-03-04T09:04:13.152 INFO:tasks.workunit.client.0.smithi071.stdout: 1 549843 11.89 MB/sec execute 120 sec latency 24.015 ms 2024-03-04T09:04:14.152 INFO:tasks.workunit.client.0.smithi071.stdout: 1 551527 11.87 MB/sec execute 121 sec latency 103.795 ms 2024-03-04T09:04:14.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:14 smithi150 ceph-mon[36807]: pgmap v162: 129 pgs: 129 active+clean; 309 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 243 op/s 2024-03-04T09:04:14.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:14 smithi158 ceph-mon[37241]: pgmap v162: 129 pgs: 129 active+clean; 309 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 243 op/s 2024-03-04T09:04:14.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:13 smithi071 ceph-mon[32789]: pgmap v162: 129 pgs: 129 active+clean; 309 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 243 op/s 2024-03-04T09:04:15.152 INFO:tasks.workunit.client.0.smithi071.stdout: 1 552015 11.81 MB/sec execute 122 sec latency 861.262 ms 2024-03-04T09:04:16.152 INFO:tasks.workunit.client.0.smithi071.stdout: 1 552015 11.71 MB/sec execute 123 sec latency 1861.384 ms 2024-03-04T09:04:16.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:16 smithi150 ceph-mon[36807]: pgmap v163: 129 pgs: 129 active+clean; 326 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.1 MiB/s wr, 311 op/s 2024-03-04T09:04:16.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:16 smithi158 ceph-mon[37241]: pgmap v163: 129 pgs: 129 active+clean; 326 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.1 MiB/s wr, 311 op/s 2024-03-04T09:04:16.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:16 smithi071 ceph-mon[32789]: pgmap v163: 129 pgs: 129 active+clean; 326 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.1 MiB/s wr, 311 op/s 2024-03-04T09:04:17.152 INFO:tasks.workunit.client.0.smithi071.stdout: 1 552015 11.62 MB/sec execute 124 sec latency 2861.518 ms 2024-03-04T09:04:18.152 INFO:tasks.workunit.client.0.smithi071.stdout: 1 552015 11.52 MB/sec execute 125 sec latency 3861.572 ms 2024-03-04T09:04:18.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:18 smithi150 ceph-mon[36807]: pgmap v164: 129 pgs: 129 active+clean; 326 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.7 MiB/s wr, 260 op/s 2024-03-04T09:04:18.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:18 smithi158 ceph-mon[37241]: pgmap v164: 129 pgs: 129 active+clean; 326 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.7 MiB/s wr, 260 op/s 2024-03-04T09:04:18.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:18 smithi071 ceph-mon[32789]: pgmap v164: 129 pgs: 129 active+clean; 326 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.7 MiB/s wr, 260 op/s 2024-03-04T09:04:19.152 INFO:tasks.workunit.client.0.smithi071.stdout: 1 552015 11.43 MB/sec execute 126 sec latency 4861.630 ms 2024-03-04T09:04:20.152 INFO:tasks.workunit.client.0.smithi071.stdout: 1 552015 11.34 MB/sec execute 127 sec latency 5861.745 ms 2024-03-04T09:04:20.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:20 smithi150 ceph-mon[36807]: pgmap v165: 129 pgs: 129 active+clean; 328 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.6 MiB/s wr, 232 op/s 2024-03-04T09:04:20.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:20 smithi158 ceph-mon[37241]: pgmap v165: 129 pgs: 129 active+clean; 328 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.6 MiB/s wr, 232 op/s 2024-03-04T09:04:20.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:20 smithi071 ceph-mon[32789]: pgmap v165: 129 pgs: 129 active+clean; 328 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.6 MiB/s wr, 232 op/s 2024-03-04T09:04:21.152 INFO:tasks.workunit.client.0.smithi071.stdout: 1 552305 11.26 MB/sec execute 128 sec latency 6669.948 ms 2024-03-04T09:04:22.152 INFO:tasks.workunit.client.0.smithi071.stdout: 1 554699 11.28 MB/sec execute 129 sec latency 29.264 ms 2024-03-04T09:04:22.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:22 smithi150 ceph-mon[36807]: pgmap v166: 129 pgs: 129 active+clean; 333 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 200 op/s 2024-03-04T09:04:22.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:22 smithi158 ceph-mon[37241]: pgmap v166: 129 pgs: 129 active+clean; 333 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 200 op/s 2024-03-04T09:04:22.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:22 smithi071 ceph-mon[32789]: pgmap v166: 129 pgs: 129 active+clean; 333 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 200 op/s 2024-03-04T09:04:23.152 INFO:tasks.workunit.client.0.smithi071.stdout: 1 557159 11.29 MB/sec execute 130 sec latency 27.633 ms 2024-03-04T09:04:24.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 559235 11.29 MB/sec execute 131 sec latency 15.667 ms 2024-03-04T09:04:24.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:24 smithi150 ceph-mon[36807]: pgmap v167: 129 pgs: 129 active+clean; 335 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 159 op/s 2024-03-04T09:04:24.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:24 smithi158 ceph-mon[37241]: pgmap v167: 129 pgs: 129 active+clean; 335 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 159 op/s 2024-03-04T09:04:24.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:24 smithi071 ceph-mon[32789]: pgmap v167: 129 pgs: 129 active+clean; 335 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 159 op/s 2024-03-04T09:04:25.152 INFO:tasks.workunit.client.0.smithi071.stdout: 1 561493 11.31 MB/sec execute 132 sec latency 91.905 ms 2024-03-04T09:04:26.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 563644 11.28 MB/sec execute 133 sec latency 14.542 ms 2024-03-04T09:04:26.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:26 smithi150 ceph-mon[36807]: pgmap v168: 129 pgs: 129 active+clean; 338 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 193 op/s 2024-03-04T09:04:26.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:26 smithi158 ceph-mon[37241]: pgmap v168: 129 pgs: 129 active+clean; 338 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 193 op/s 2024-03-04T09:04:26.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:26 smithi071 ceph-mon[32789]: pgmap v168: 129 pgs: 129 active+clean; 338 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 193 op/s 2024-03-04T09:04:27.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 565604 11.29 MB/sec execute 134 sec latency 20.742 ms 2024-03-04T09:04:28.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 568487 11.33 MB/sec execute 135 sec latency 21.866 ms 2024-03-04T09:04:28.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:28 smithi150 ceph-mon[36807]: pgmap v169: 129 pgs: 129 active+clean; 331 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 155 op/s 2024-03-04T09:04:28.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:28 smithi158 ceph-mon[37241]: pgmap v169: 129 pgs: 129 active+clean; 331 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 155 op/s 2024-03-04T09:04:28.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:28 smithi071 ceph-mon[32789]: pgmap v169: 129 pgs: 129 active+clean; 331 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 155 op/s 2024-03-04T09:04:29.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 570259 11.30 MB/sec execute 136 sec latency 34.792 ms 2024-03-04T09:04:30.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 572294 11.32 MB/sec execute 137 sec latency 70.236 ms 2024-03-04T09:04:30.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:30 smithi150 ceph-mon[36807]: pgmap v170: 129 pgs: 129 active+clean; 338 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 200 op/s 2024-03-04T09:04:30.301 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:30 smithi158 ceph-mon[37241]: pgmap v170: 129 pgs: 129 active+clean; 338 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 200 op/s 2024-03-04T09:04:30.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:30 smithi071 ceph-mon[32789]: pgmap v170: 129 pgs: 129 active+clean; 338 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 200 op/s 2024-03-04T09:04:31.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 574483 11.29 MB/sec execute 138 sec latency 17.051 ms 2024-03-04T09:04:32.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 576859 11.33 MB/sec execute 139 sec latency 28.121 ms 2024-03-04T09:04:32.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:32 smithi150 ceph-mon[36807]: pgmap v171: 129 pgs: 129 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 218 op/s 2024-03-04T09:04:32.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:32 smithi158 ceph-mon[37241]: pgmap v171: 129 pgs: 129 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 218 op/s 2024-03-04T09:04:32.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:32 smithi071 ceph-mon[32789]: pgmap v171: 129 pgs: 129 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 218 op/s 2024-03-04T09:04:33.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 579380 11.35 MB/sec execute 140 sec latency 24.367 ms 2024-03-04T09:04:33.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:33 smithi071 ceph-mon[32789]: pgmap v172: 129 pgs: 129 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-03-04T09:04:33.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:33 smithi071 ceph-mon[32789]: Health check cleared: OSD_SLOW_PING_TIME_BACK (was: Slow OSD heartbeats on back (longest 1025.744ms)) 2024-03-04T09:04:33.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:33 smithi071 ceph-mon[32789]: Health check cleared: OSD_SLOW_PING_TIME_FRONT (was: Slow OSD heartbeats on front (longest 1030.137ms)) 2024-03-04T09:04:33.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:33 smithi071 ceph-mon[32789]: Cluster is now healthy 2024-03-04T09:04:33.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:33 smithi150 ceph-mon[36807]: pgmap v172: 129 pgs: 129 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-03-04T09:04:33.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:33 smithi150 ceph-mon[36807]: Health check cleared: OSD_SLOW_PING_TIME_BACK (was: Slow OSD heartbeats on back (longest 1025.744ms)) 2024-03-04T09:04:33.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:33 smithi150 ceph-mon[36807]: Health check cleared: OSD_SLOW_PING_TIME_FRONT (was: Slow OSD heartbeats on front (longest 1030.137ms)) 2024-03-04T09:04:33.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:33 smithi150 ceph-mon[36807]: Cluster is now healthy 2024-03-04T09:04:33.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:33 smithi158 ceph-mon[37241]: pgmap v172: 129 pgs: 129 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-03-04T09:04:33.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:33 smithi158 ceph-mon[37241]: Health check cleared: OSD_SLOW_PING_TIME_BACK (was: Slow OSD heartbeats on back (longest 1025.744ms)) 2024-03-04T09:04:33.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:33 smithi158 ceph-mon[37241]: Health check cleared: OSD_SLOW_PING_TIME_FRONT (was: Slow OSD heartbeats on front (longest 1030.137ms)) 2024-03-04T09:04:33.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:33 smithi158 ceph-mon[37241]: Cluster is now healthy 2024-03-04T09:04:34.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 581905 11.35 MB/sec execute 141 sec latency 15.569 ms 2024-03-04T09:04:35.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 583400 11.33 MB/sec execute 142 sec latency 64.265 ms 2024-03-04T09:04:36.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 586183 11.37 MB/sec execute 143 sec latency 15.485 ms 2024-03-04T09:04:36.231 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:35 smithi150 ceph-mon[36807]: pgmap v173: 129 pgs: 129 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 241 op/s 2024-03-04T09:04:36.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:35 smithi158 ceph-mon[37241]: pgmap v173: 129 pgs: 129 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 241 op/s 2024-03-04T09:04:36.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:35 smithi071 ceph-mon[32789]: pgmap v173: 129 pgs: 129 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 241 op/s 2024-03-04T09:04:37.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 587847 11.33 MB/sec execute 144 sec latency 30.350 ms 2024-03-04T09:04:38.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 590287 11.35 MB/sec execute 145 sec latency 15.912 ms 2024-03-04T09:04:38.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:37 smithi150 ceph-mon[36807]: pgmap v174: 129 pgs: 129 active+clean; 349 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 285 op/s 2024-03-04T09:04:38.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:37 smithi158 ceph-mon[37241]: pgmap v174: 129 pgs: 129 active+clean; 349 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 285 op/s 2024-03-04T09:04:38.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:37 smithi071 ceph-mon[32789]: pgmap v174: 129 pgs: 129 active+clean; 349 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 285 op/s 2024-03-04T09:04:39.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 592969 11.37 MB/sec execute 146 sec latency 21.851 ms 2024-03-04T09:04:40.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 594143 11.33 MB/sec execute 147 sec latency 59.937 ms 2024-03-04T09:04:40.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:39 smithi150 ceph-mon[36807]: pgmap v175: 129 pgs: 129 active+clean; 349 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 260 op/s 2024-03-04T09:04:40.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:39 smithi158 ceph-mon[37241]: pgmap v175: 129 pgs: 129 active+clean; 349 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 260 op/s 2024-03-04T09:04:40.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:39 smithi071 ceph-mon[32789]: pgmap v175: 129 pgs: 129 active+clean; 349 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 260 op/s 2024-03-04T09:04:41.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 596985 11.38 MB/sec execute 148 sec latency 12.716 ms 2024-03-04T09:04:42.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 599119 11.35 MB/sec execute 149 sec latency 25.139 ms 2024-03-04T09:04:42.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:41 smithi150 ceph-mon[36807]: pgmap v176: 129 pgs: 129 active+clean; 354 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 242 op/s 2024-03-04T09:04:42.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:41 smithi158 ceph-mon[37241]: pgmap v176: 129 pgs: 129 active+clean; 354 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 242 op/s 2024-03-04T09:04:42.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:41 smithi071 ceph-mon[32789]: pgmap v176: 129 pgs: 129 active+clean; 354 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 242 op/s 2024-03-04T09:04:43.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 601229 11.36 MB/sec execute 150 sec latency 33.271 ms 2024-03-04T09:04:44.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 603861 11.40 MB/sec execute 151 sec latency 39.144 ms 2024-03-04T09:04:44.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:43 smithi150 ceph-mon[36807]: pgmap v177: 129 pgs: 129 active+clean; 354 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 206 op/s 2024-03-04T09:04:44.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:43 smithi158 ceph-mon[37241]: pgmap v177: 129 pgs: 129 active+clean; 354 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 206 op/s 2024-03-04T09:04:44.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:43 smithi071 ceph-mon[32789]: pgmap v177: 129 pgs: 129 active+clean; 354 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 206 op/s 2024-03-04T09:04:45.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 604967 11.34 MB/sec execute 152 sec latency 105.314 ms 2024-03-04T09:04:46.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 607613 11.38 MB/sec execute 153 sec latency 15.472 ms 2024-03-04T09:04:46.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:45 smithi150 ceph-mon[36807]: pgmap v178: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 236 op/s 2024-03-04T09:04:46.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:45 smithi158 ceph-mon[37241]: pgmap v178: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 236 op/s 2024-03-04T09:04:46.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:45 smithi071 ceph-mon[32789]: pgmap v178: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 236 op/s 2024-03-04T09:04:47.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 609323 11.34 MB/sec execute 154 sec latency 28.851 ms 2024-03-04T09:04:48.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 611695 11.36 MB/sec execute 155 sec latency 11.141 ms 2024-03-04T09:04:48.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:47 smithi150 ceph-mon[36807]: pgmap v179: 129 pgs: 129 active+clean; 365 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.8 MiB/s wr, 293 op/s 2024-03-04T09:04:48.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:47 smithi158 ceph-mon[37241]: pgmap v179: 129 pgs: 129 active+clean; 365 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.8 MiB/s wr, 293 op/s 2024-03-04T09:04:48.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:47 smithi071 ceph-mon[32789]: pgmap v179: 129 pgs: 129 active+clean; 365 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.8 MiB/s wr, 293 op/s 2024-03-04T09:04:49.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 614575 11.40 MB/sec execute 156 sec latency 21.051 ms 2024-03-04T09:04:50.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 616776 11.38 MB/sec execute 157 sec latency 14.350 ms 2024-03-04T09:04:50.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:50 smithi150 ceph-mon[36807]: pgmap v180: 129 pgs: 129 active+clean; 367 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 238 op/s 2024-03-04T09:04:50.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:50 smithi158 ceph-mon[37241]: pgmap v180: 129 pgs: 129 active+clean; 367 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 238 op/s 2024-03-04T09:04:50.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:50 smithi071 ceph-mon[32789]: pgmap v180: 129 pgs: 129 active+clean; 367 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 238 op/s 2024-03-04T09:04:51.154 INFO:tasks.workunit.client.0.smithi071.stdout: 1 618716 11.39 MB/sec execute 158 sec latency 18.349 ms 2024-03-04T09:04:52.153 INFO:tasks.workunit.client.0.smithi071.stdout: 1 621510 11.42 MB/sec execute 159 sec latency 16.212 ms 2024-03-04T09:04:52.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:52 smithi150 ceph-mon[36807]: pgmap v181: 129 pgs: 129 active+clean; 369 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 240 op/s 2024-03-04T09:04:52.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:52 smithi158 ceph-mon[37241]: pgmap v181: 129 pgs: 129 active+clean; 369 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 240 op/s 2024-03-04T09:04:52.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:52 smithi071 ceph-mon[32789]: pgmap v181: 129 pgs: 129 active+clean; 369 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 240 op/s 2024-03-04T09:04:53.154 INFO:tasks.workunit.client.0.smithi071.stdout: 1 623219 11.39 MB/sec execute 160 sec latency 14.342 ms 2024-03-04T09:04:54.154 INFO:tasks.workunit.client.0.smithi071.stdout: 1 625505 11.41 MB/sec execute 161 sec latency 112.004 ms 2024-03-04T09:04:54.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:54 smithi150 ceph-mon[36807]: pgmap v182: 129 pgs: 129 active+clean; 371 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 197 op/s 2024-03-04T09:04:54.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:54 smithi158 ceph-mon[37241]: pgmap v182: 129 pgs: 129 active+clean; 371 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 197 op/s 2024-03-04T09:04:54.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:54 smithi071 ceph-mon[32789]: pgmap v182: 129 pgs: 129 active+clean; 371 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 197 op/s 2024-03-04T09:04:55.154 INFO:tasks.workunit.client.0.smithi071.stdout: 1 627497 11.38 MB/sec execute 162 sec latency 83.100 ms 2024-03-04T09:04:56.154 INFO:tasks.workunit.client.0.smithi071.stdout: 1 629560 11.39 MB/sec execute 163 sec latency 32.818 ms 2024-03-04T09:04:56.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:56 smithi150 ceph-mon[36807]: pgmap v183: 129 pgs: 129 active+clean; 373 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 212 op/s 2024-03-04T09:04:56.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:56 smithi158 ceph-mon[37241]: pgmap v183: 129 pgs: 129 active+clean; 373 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 212 op/s 2024-03-04T09:04:56.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:56 smithi071 ceph-mon[32789]: pgmap v183: 129 pgs: 129 active+clean; 373 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 212 op/s 2024-03-04T09:04:57.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 632342 11.43 MB/sec execute 164 sec latency 25.722 ms 2024-03-04T09:04:58.154 INFO:tasks.workunit.client.0.smithi071.stdout: 1 634462 11.41 MB/sec execute 165 sec latency 15.697 ms 2024-03-04T09:04:58.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:04:58 smithi150 ceph-mon[36807]: pgmap v184: 129 pgs: 129 active+clean; 385 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 246 op/s 2024-03-04T09:04:58.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:04:58 smithi158 ceph-mon[37241]: pgmap v184: 129 pgs: 129 active+clean; 385 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 246 op/s 2024-03-04T09:04:58.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:04:58 smithi071 ceph-mon[32789]: pgmap v184: 129 pgs: 129 active+clean; 385 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 246 op/s 2024-03-04T09:04:59.154 INFO:tasks.workunit.client.0.smithi071.stdout: 1 636454 11.41 MB/sec execute 166 sec latency 16.155 ms 2024-03-04T09:05:00.154 INFO:tasks.workunit.client.0.smithi071.stdout: 1 638613 11.41 MB/sec execute 167 sec latency 52.331 ms 2024-03-04T09:05:00.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:00 smithi150 ceph-mon[36807]: pgmap v185: 129 pgs: 129 active+clean; 386 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 181 op/s 2024-03-04T09:05:00.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:00 smithi158 ceph-mon[37241]: pgmap v185: 129 pgs: 129 active+clean; 386 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 181 op/s 2024-03-04T09:05:00.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:00 smithi071 ceph-mon[32789]: pgmap v185: 129 pgs: 129 active+clean; 386 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 181 op/s 2024-03-04T09:05:01.154 INFO:tasks.workunit.client.0.smithi071.stdout: 1 640037 11.39 MB/sec execute 168 sec latency 89.022 ms 2024-03-04T09:05:02.154 INFO:tasks.workunit.client.0.smithi071.stdout: 1 640681 11.35 MB/sec execute 169 sec latency 60.464 ms 2024-03-04T09:05:02.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:02 smithi150 ceph-mon[36807]: pgmap v186: 129 pgs: 129 active+clean; 381 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 194 op/s 2024-03-04T09:05:02.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:02 smithi158 ceph-mon[37241]: pgmap v186: 129 pgs: 129 active+clean; 381 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 194 op/s 2024-03-04T09:05:02.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:02 smithi071 ceph-mon[32789]: pgmap v186: 129 pgs: 129 active+clean; 381 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 194 op/s 2024-03-04T09:05:03.154 INFO:tasks.workunit.client.0.smithi071.stdout: 1 643103 11.37 MB/sec execute 170 sec latency 16.405 ms 2024-03-04T09:05:04.154 INFO:tasks.workunit.client.0.smithi071.stdout: 1 645000 11.34 MB/sec execute 171 sec latency 40.379 ms 2024-03-04T09:05:04.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:04 smithi150 ceph-mon[36807]: pgmap v187: 129 pgs: 129 active+clean; 378 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 210 op/s 2024-03-04T09:05:04.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:04 smithi158 ceph-mon[37241]: pgmap v187: 129 pgs: 129 active+clean; 378 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 210 op/s 2024-03-04T09:05:04.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:04 smithi071 ceph-mon[32789]: pgmap v187: 129 pgs: 129 active+clean; 378 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 210 op/s 2024-03-04T09:05:05.154 INFO:tasks.workunit.client.0.smithi071.stdout: 1 646951 11.35 MB/sec execute 172 sec latency 18.416 ms 2024-03-04T09:05:05.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:05 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:05:05.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:05 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:05:05.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:05 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:05:06.154 INFO:tasks.workunit.client.0.smithi071.stdout: 1 648829 11.33 MB/sec execute 173 sec latency 95.021 ms 2024-03-04T09:05:06.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:06 smithi158 ceph-mon[37241]: pgmap v188: 129 pgs: 129 active+clean; 369 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 222 op/s 2024-03-04T09:05:06.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:06 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:05:06.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:06 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:05:06.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:06 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:05:06.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:06 smithi071 ceph-mon[32789]: pgmap v188: 129 pgs: 129 active+clean; 369 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 222 op/s 2024-03-04T09:05:06.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:06 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:05:06.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:06 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:05:06.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:06 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:05:06.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:06 smithi150 ceph-mon[36807]: pgmap v188: 129 pgs: 129 active+clean; 369 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 222 op/s 2024-03-04T09:05:06.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:06 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:05:06.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:06 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:05:06.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:06 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:05:07.154 INFO:tasks.workunit.client.0.smithi071.stdout: 1 650998 11.34 MB/sec execute 174 sec latency 14.216 ms 2024-03-04T09:05:08.154 INFO:tasks.workunit.client.0.smithi071.stdout: 1 653701 11.37 MB/sec execute 175 sec latency 35.032 ms 2024-03-04T09:05:08.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:08 smithi158 ceph-mon[37241]: pgmap v189: 129 pgs: 129 active+clean; 373 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 276 op/s 2024-03-04T09:05:08.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:08 smithi071 ceph-mon[32789]: pgmap v189: 129 pgs: 129 active+clean; 373 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 276 op/s 2024-03-04T09:05:08.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:08 smithi150 ceph-mon[36807]: pgmap v189: 129 pgs: 129 active+clean; 373 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 276 op/s 2024-03-04T09:05:09.154 INFO:tasks.workunit.client.0.smithi071.stdout: 1 655829 11.35 MB/sec execute 176 sec latency 18.211 ms 2024-03-04T09:05:10.154 INFO:tasks.workunit.client.0.smithi071.stdout: 1 657916 11.36 MB/sec execute 177 sec latency 11.539 ms 2024-03-04T09:05:10.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:10 smithi158 ceph-mon[37241]: pgmap v190: 129 pgs: 129 active+clean; 377 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 238 op/s 2024-03-04T09:05:10.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:10 smithi071 ceph-mon[32789]: pgmap v190: 129 pgs: 129 active+clean; 377 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 238 op/s 2024-03-04T09:05:10.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:10 smithi150 ceph-mon[36807]: pgmap v190: 129 pgs: 129 active+clean; 377 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 238 op/s 2024-03-04T09:05:11.154 INFO:tasks.workunit.client.0.smithi071.stdout: 1 660184 11.36 MB/sec execute 178 sec latency 50.001 ms 2024-03-04T09:05:12.154 INFO:tasks.workunit.client.0.smithi071.stdout: 1 662089 11.36 MB/sec execute 179 sec latency 16.775 ms 2024-03-04T09:05:12.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:12 smithi158 ceph-mon[37241]: pgmap v191: 129 pgs: 129 active+clean; 382 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 271 op/s 2024-03-04T09:05:12.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:12 smithi071 ceph-mon[32789]: pgmap v191: 129 pgs: 129 active+clean; 382 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 271 op/s 2024-03-04T09:05:12.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:12 smithi150 ceph-mon[36807]: pgmap v191: 129 pgs: 129 active+clean; 382 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 271 op/s 2024-03-04T09:05:13.154 INFO:tasks.workunit.client.0.smithi071.stdout: 1 664523 11.37 MB/sec execute 180 sec latency 26.790 ms 2024-03-04T09:05:14.154 INFO:tasks.workunit.client.0.smithi071.stdout: 1 666971 11.38 MB/sec execute 181 sec latency 18.839 ms 2024-03-04T09:05:14.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:14 smithi158 ceph-mon[37241]: pgmap v192: 129 pgs: 129 active+clean; 382 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 237 op/s 2024-03-04T09:05:14.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:14 smithi071 ceph-mon[32789]: pgmap v192: 129 pgs: 129 active+clean; 382 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 237 op/s 2024-03-04T09:05:14.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:14 smithi150 ceph-mon[36807]: pgmap v192: 129 pgs: 129 active+clean; 382 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 237 op/s 2024-03-04T09:05:15.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 669091 11.38 MB/sec execute 182 sec latency 11.043 ms 2024-03-04T09:05:16.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 669714 11.32 MB/sec execute 183 sec latency 624.091 ms 2024-03-04T09:05:16.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:16 smithi158 ceph-mon[37241]: pgmap v193: 129 pgs: 129 active+clean; 386 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 243 op/s 2024-03-04T09:05:16.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:16 smithi071 ceph-mon[32789]: pgmap v193: 129 pgs: 129 active+clean; 386 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 243 op/s 2024-03-04T09:05:16.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:16 smithi150 ceph-mon[36807]: pgmap v193: 129 pgs: 129 active+clean; 386 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 243 op/s 2024-03-04T09:05:17.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 669714 11.26 MB/sec execute 184 sec latency 1624.188 ms 2024-03-04T09:05:18.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 669714 11.20 MB/sec execute 185 sec latency 2624.273 ms 2024-03-04T09:05:19.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 669764 11.14 MB/sec execute 186 sec latency 3553.698 ms 2024-03-04T09:05:19.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:19 smithi158 ceph-mon[37241]: pgmap v194: 129 pgs: 129 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.6 MiB/s wr, 287 op/s 2024-03-04T09:05:19.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:19 smithi071 ceph-mon[32789]: pgmap v194: 129 pgs: 129 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.6 MiB/s wr, 287 op/s 2024-03-04T09:05:19.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:19 smithi150 ceph-mon[36807]: pgmap v194: 129 pgs: 129 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.6 MiB/s wr, 287 op/s 2024-03-04T09:05:20.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 671963 11.16 MB/sec execute 187 sec latency 82.018 ms 2024-03-04T09:05:20.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:20 smithi158 ceph-mon[37241]: pgmap v195: 129 pgs: 129 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 209 op/s 2024-03-04T09:05:20.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:20 smithi071 ceph-mon[32789]: pgmap v195: 129 pgs: 129 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 209 op/s 2024-03-04T09:05:20.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:20 smithi150 ceph-mon[36807]: pgmap v195: 129 pgs: 129 active+clean; 394 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 209 op/s 2024-03-04T09:05:21.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 673996 11.15 MB/sec execute 188 sec latency 137.512 ms 2024-03-04T09:05:21.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:21 smithi071 ceph-mon[32789]: pgmap v196: 129 pgs: 129 active+clean; 396 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 206 op/s 2024-03-04T09:05:21.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:21 smithi150 ceph-mon[36807]: pgmap v196: 129 pgs: 129 active+clean; 396 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 206 op/s 2024-03-04T09:05:21.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:21 smithi158 ceph-mon[37241]: pgmap v196: 129 pgs: 129 active+clean; 396 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 206 op/s 2024-03-04T09:05:22.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 676151 11.16 MB/sec execute 189 sec latency 15.493 ms 2024-03-04T09:05:23.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 678679 11.18 MB/sec execute 190 sec latency 21.429 ms 2024-03-04T09:05:24.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 680917 11.16 MB/sec execute 191 sec latency 14.995 ms 2024-03-04T09:05:24.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:23 smithi150 ceph-mon[36807]: pgmap v197: 129 pgs: 129 active+clean; 397 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 165 op/s 2024-03-04T09:05:24.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:23 smithi158 ceph-mon[37241]: pgmap v197: 129 pgs: 129 active+clean; 397 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 165 op/s 2024-03-04T09:05:24.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:23 smithi071 ceph-mon[32789]: pgmap v197: 129 pgs: 129 active+clean; 397 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 165 op/s 2024-03-04T09:05:25.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 682983 11.17 MB/sec execute 192 sec latency 14.889 ms 2024-03-04T09:05:26.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 685412 11.19 MB/sec execute 193 sec latency 92.850 ms 2024-03-04T09:05:26.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:25 smithi150 ceph-mon[36807]: pgmap v198: 129 pgs: 129 active+clean; 401 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 220 op/s 2024-03-04T09:05:26.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:25 smithi158 ceph-mon[37241]: pgmap v198: 129 pgs: 129 active+clean; 401 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 220 op/s 2024-03-04T09:05:26.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:25 smithi071 ceph-mon[32789]: pgmap v198: 129 pgs: 129 active+clean; 401 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 220 op/s 2024-03-04T09:05:27.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 687095 11.17 MB/sec execute 194 sec latency 30.291 ms 2024-03-04T09:05:28.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 689625 11.19 MB/sec execute 195 sec latency 12.631 ms 2024-03-04T09:05:28.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:28 smithi150 ceph-mon[36807]: pgmap v199: 129 pgs: 129 active+clean; 400 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 209 op/s 2024-03-04T09:05:28.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:28 smithi158 ceph-mon[37241]: pgmap v199: 129 pgs: 129 active+clean; 400 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 209 op/s 2024-03-04T09:05:28.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:28 smithi071 ceph-mon[32789]: pgmap v199: 129 pgs: 129 active+clean; 400 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 209 op/s 2024-03-04T09:05:29.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 692122 11.20 MB/sec execute 196 sec latency 86.315 ms 2024-03-04T09:05:30.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 693984 11.20 MB/sec execute 197 sec latency 18.480 ms 2024-03-04T09:05:30.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:30 smithi150 ceph-mon[36807]: pgmap v200: 129 pgs: 129 active+clean; 400 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 138 op/s 2024-03-04T09:05:30.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:30 smithi158 ceph-mon[37241]: pgmap v200: 129 pgs: 129 active+clean; 400 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 138 op/s 2024-03-04T09:05:30.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:30 smithi071 ceph-mon[32789]: pgmap v200: 129 pgs: 129 active+clean; 400 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 138 op/s 2024-03-04T09:05:31.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 696281 11.21 MB/sec execute 198 sec latency 71.294 ms 2024-03-04T09:05:32.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 698429 11.19 MB/sec execute 199 sec latency 16.590 ms 2024-03-04T09:05:32.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:32 smithi150 ceph-mon[36807]: pgmap v201: 129 pgs: 129 active+clean; 410 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 227 op/s 2024-03-04T09:05:32.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:32 smithi158 ceph-mon[37241]: pgmap v201: 129 pgs: 129 active+clean; 410 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 227 op/s 2024-03-04T09:05:32.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:32 smithi071 ceph-mon[32789]: pgmap v201: 129 pgs: 129 active+clean; 410 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 227 op/s 2024-03-04T09:05:33.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 700519 11.20 MB/sec execute 200 sec latency 19.105 ms 2024-03-04T09:05:34.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 703338 11.23 MB/sec execute 201 sec latency 16.011 ms 2024-03-04T09:05:34.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:34 smithi150 ceph-mon[36807]: pgmap v202: 129 pgs: 129 active+clean; 410 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 202 op/s 2024-03-04T09:05:34.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:34 smithi158 ceph-mon[37241]: pgmap v202: 129 pgs: 129 active+clean; 410 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 202 op/s 2024-03-04T09:05:34.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:34 smithi071 ceph-mon[32789]: pgmap v202: 129 pgs: 129 active+clean; 410 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 202 op/s 2024-03-04T09:05:35.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 705330 11.21 MB/sec execute 202 sec latency 31.582 ms 2024-03-04T09:05:36.156 INFO:tasks.workunit.client.0.smithi071.stdout: 1 707355 11.22 MB/sec execute 203 sec latency 65.827 ms 2024-03-04T09:05:36.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:36 smithi150 ceph-mon[36807]: pgmap v203: 129 pgs: 129 active+clean; 415 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-03-04T09:05:36.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:36 smithi158 ceph-mon[37241]: pgmap v203: 129 pgs: 129 active+clean; 415 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-03-04T09:05:36.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:36 smithi071 ceph-mon[32789]: pgmap v203: 129 pgs: 129 active+clean; 415 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-03-04T09:05:37.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 709841 11.23 MB/sec execute 204 sec latency 19.450 ms 2024-03-04T09:05:38.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 711503 11.21 MB/sec execute 205 sec latency 29.979 ms 2024-03-04T09:05:38.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:38 smithi150 ceph-mon[36807]: pgmap v204: 129 pgs: 129 active+clean; 417 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.8 MiB/s wr, 231 op/s 2024-03-04T09:05:38.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:38 smithi158 ceph-mon[37241]: pgmap v204: 129 pgs: 129 active+clean; 417 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.8 MiB/s wr, 231 op/s 2024-03-04T09:05:38.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:38 smithi071 ceph-mon[32789]: pgmap v204: 129 pgs: 129 active+clean; 417 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.8 MiB/s wr, 231 op/s 2024-03-04T09:05:39.155 INFO:tasks.workunit.client.0.smithi071.stdout: 1 714009 11.24 MB/sec execute 206 sec latency 33.132 ms 2024-03-04T09:05:40.156 INFO:tasks.workunit.client.0.smithi071.stdout: 1 715657 11.21 MB/sec execute 207 sec latency 29.795 ms 2024-03-04T09:05:40.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:40 smithi150 ceph-mon[36807]: pgmap v205: 129 pgs: 129 active+clean; 417 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 201 op/s 2024-03-04T09:05:40.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:40 smithi158 ceph-mon[37241]: pgmap v205: 129 pgs: 129 active+clean; 417 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 201 op/s 2024-03-04T09:05:40.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:40 smithi071 ceph-mon[32789]: pgmap v205: 129 pgs: 129 active+clean; 417 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 201 op/s 2024-03-04T09:05:41.156 INFO:tasks.workunit.client.0.smithi071.stdout: 1 717713 11.22 MB/sec execute 208 sec latency 72.604 ms 2024-03-04T09:05:42.156 INFO:tasks.workunit.client.0.smithi071.stdout: 1 719894 11.21 MB/sec execute 209 sec latency 26.858 ms 2024-03-04T09:05:42.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:42 smithi150 ceph-mon[36807]: pgmap v206: 129 pgs: 129 active+clean; 419 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 251 op/s 2024-03-04T09:05:42.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:42 smithi158 ceph-mon[37241]: pgmap v206: 129 pgs: 129 active+clean; 419 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 251 op/s 2024-03-04T09:05:42.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:42 smithi071 ceph-mon[32789]: pgmap v206: 129 pgs: 129 active+clean; 419 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 251 op/s 2024-03-04T09:05:43.156 INFO:tasks.workunit.client.0.smithi071.stdout: 1 721780 11.21 MB/sec execute 210 sec latency 19.379 ms 2024-03-04T09:05:44.156 INFO:tasks.workunit.client.0.smithi071.stdout: 1 724533 11.24 MB/sec execute 211 sec latency 47.625 ms 2024-03-04T09:05:44.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:44 smithi150 ceph-mon[36807]: pgmap v207: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 200 op/s 2024-03-04T09:05:44.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:44 smithi158 ceph-mon[37241]: pgmap v207: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 200 op/s 2024-03-04T09:05:44.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:44 smithi071 ceph-mon[32789]: pgmap v207: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 200 op/s 2024-03-04T09:05:45.156 INFO:tasks.workunit.client.0.smithi071.stdout: 1 726099 11.22 MB/sec execute 212 sec latency 36.262 ms 2024-03-04T09:05:46.156 INFO:tasks.workunit.client.0.smithi071.stdout: 1 728540 11.23 MB/sec execute 213 sec latency 72.993 ms 2024-03-04T09:05:46.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:46 smithi150 ceph-mon[36807]: pgmap v208: 129 pgs: 129 active+clean; 410 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 260 op/s 2024-03-04T09:05:46.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:46 smithi158 ceph-mon[37241]: pgmap v208: 129 pgs: 129 active+clean; 410 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 260 op/s 2024-03-04T09:05:46.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:46 smithi071 ceph-mon[32789]: pgmap v208: 129 pgs: 129 active+clean; 410 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 260 op/s 2024-03-04T09:05:47.156 INFO:tasks.workunit.client.0.smithi071.stdout: 1 730715 11.23 MB/sec execute 214 sec latency 143.312 ms 2024-03-04T09:05:48.156 INFO:tasks.workunit.client.0.smithi071.stdout: 1 732564 11.22 MB/sec execute 215 sec latency 31.123 ms 2024-03-04T09:05:48.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:48 smithi150 ceph-mon[36807]: pgmap v209: 129 pgs: 129 active+clean; 394 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 244 op/s 2024-03-04T09:05:48.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:48 smithi158 ceph-mon[37241]: pgmap v209: 129 pgs: 129 active+clean; 394 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 244 op/s 2024-03-04T09:05:48.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:48 smithi071 ceph-mon[32789]: pgmap v209: 129 pgs: 129 active+clean; 394 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.6 MiB/s wr, 244 op/s 2024-03-04T09:05:49.156 INFO:tasks.workunit.client.0.smithi071.stdout: 1 735300 11.25 MB/sec execute 216 sec latency 19.155 ms 2024-03-04T09:05:50.156 INFO:tasks.workunit.client.0.smithi071.stdout: 1 737376 11.23 MB/sec execute 217 sec latency 16.291 ms 2024-03-04T09:05:50.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:50 smithi150 ceph-mon[36807]: pgmap v210: 129 pgs: 129 active+clean; 394 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 185 op/s 2024-03-04T09:05:50.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:50 smithi158 ceph-mon[37241]: pgmap v210: 129 pgs: 129 active+clean; 394 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 185 op/s 2024-03-04T09:05:50.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:50 smithi071 ceph-mon[32789]: pgmap v210: 129 pgs: 129 active+clean; 394 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 185 op/s 2024-03-04T09:05:51.156 INFO:tasks.workunit.client.0.smithi071.stdout: 1 739473 11.24 MB/sec execute 218 sec latency 21.968 ms 2024-03-04T09:05:52.156 INFO:tasks.workunit.client.0.smithi071.stdout: 1 742268 11.26 MB/sec execute 219 sec latency 54.966 ms 2024-03-04T09:05:52.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:52 smithi150 ceph-mon[36807]: pgmap v211: 129 pgs: 129 active+clean; 405 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.5 MiB/s wr, 292 op/s 2024-03-04T09:05:52.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:52 smithi158 ceph-mon[37241]: pgmap v211: 129 pgs: 129 active+clean; 405 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.5 MiB/s wr, 292 op/s 2024-03-04T09:05:52.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:52 smithi071 ceph-mon[32789]: pgmap v211: 129 pgs: 129 active+clean; 405 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.5 MiB/s wr, 292 op/s 2024-03-04T09:05:53.156 INFO:tasks.workunit.client.0.smithi071.stdout: 1 743990 11.24 MB/sec execute 220 sec latency 25.601 ms 2024-03-04T09:05:54.156 INFO:tasks.workunit.client.0.smithi071.stdout: 1 746365 11.26 MB/sec execute 221 sec latency 23.464 ms 2024-03-04T09:05:54.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:54 smithi150 ceph-mon[36807]: pgmap v212: 129 pgs: 129 active+clean; 405 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 252 op/s 2024-03-04T09:05:54.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:54 smithi158 ceph-mon[37241]: pgmap v212: 129 pgs: 129 active+clean; 405 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 252 op/s 2024-03-04T09:05:54.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:54 smithi071 ceph-mon[32789]: pgmap v212: 129 pgs: 129 active+clean; 405 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 252 op/s 2024-03-04T09:05:55.156 INFO:tasks.workunit.client.0.smithi071.stdout: 1 748978 11.27 MB/sec execute 222 sec latency 17.952 ms 2024-03-04T09:05:55.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:55 smithi071 ceph-mon[32789]: pgmap v213: 129 pgs: 129 active+clean; 405 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 225 op/s 2024-03-04T09:05:56.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:55 smithi150 ceph-mon[36807]: pgmap v213: 129 pgs: 129 active+clean; 405 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 225 op/s 2024-03-04T09:05:56.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:55 smithi158 ceph-mon[37241]: pgmap v213: 129 pgs: 129 active+clean; 405 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 225 op/s 2024-03-04T09:05:56.156 INFO:tasks.workunit.client.0.smithi071.stdout: 1 750749 11.26 MB/sec execute 223 sec latency 15.524 ms 2024-03-04T09:05:57.156 INFO:tasks.workunit.client.0.smithi071.stdout: 1 753042 11.27 MB/sec execute 224 sec latency 94.165 ms 2024-03-04T09:05:58.156 INFO:tasks.workunit.client.0.smithi071.stdout: 1 754428 11.25 MB/sec execute 225 sec latency 26.040 ms 2024-03-04T09:05:58.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:05:58 smithi150 ceph-mon[36807]: pgmap v214: 129 pgs: 129 active+clean; 414 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 231 op/s 2024-03-04T09:05:58.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:05:58 smithi158 ceph-mon[37241]: pgmap v214: 129 pgs: 129 active+clean; 414 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 231 op/s 2024-03-04T09:05:58.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:05:58 smithi071 ceph-mon[32789]: pgmap v214: 129 pgs: 129 active+clean; 414 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 231 op/s 2024-03-04T09:05:59.157 INFO:tasks.workunit.client.0.smithi071.stdout: 1 756823 11.26 MB/sec execute 226 sec latency 22.477 ms 2024-03-04T09:06:00.157 INFO:tasks.workunit.client.0.smithi071.stdout: 1 759250 11.26 MB/sec execute 227 sec latency 16.427 ms 2024-03-04T09:06:00.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:00 smithi150 ceph-mon[36807]: pgmap v215: 129 pgs: 129 active+clean; 414 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 204 op/s 2024-03-04T09:06:00.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:00 smithi158 ceph-mon[37241]: pgmap v215: 129 pgs: 129 active+clean; 414 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 204 op/s 2024-03-04T09:06:00.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:00 smithi071 ceph-mon[32789]: pgmap v215: 129 pgs: 129 active+clean; 414 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 204 op/s 2024-03-04T09:06:01.157 INFO:tasks.workunit.client.0.smithi071.stdout: 1 760875 11.25 MB/sec execute 228 sec latency 51.632 ms 2024-03-04T09:06:02.157 INFO:tasks.workunit.client.0.smithi071.stdout: 1 761525 11.22 MB/sec execute 229 sec latency 116.481 ms 2024-03-04T09:06:02.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:02 smithi150 ceph-mon[36807]: pgmap v216: 129 pgs: 129 active+clean; 418 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 267 op/s 2024-03-04T09:06:02.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:02 smithi158 ceph-mon[37241]: pgmap v216: 129 pgs: 129 active+clean; 418 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 267 op/s 2024-03-04T09:06:02.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:02 smithi071 ceph-mon[32789]: pgmap v216: 129 pgs: 129 active+clean; 418 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 267 op/s 2024-03-04T09:06:03.157 INFO:tasks.workunit.client.0.smithi071.stdout: 1 763810 11.23 MB/sec execute 230 sec latency 61.498 ms 2024-03-04T09:06:04.157 INFO:tasks.workunit.client.0.smithi071.stdout: 1 765851 11.21 MB/sec execute 231 sec latency 15.665 ms 2024-03-04T09:06:04.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:04 smithi150 ceph-mon[36807]: pgmap v217: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 197 op/s 2024-03-04T09:06:04.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:04 smithi158 ceph-mon[37241]: pgmap v217: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 197 op/s 2024-03-04T09:06:04.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:04 smithi071 ceph-mon[32789]: pgmap v217: 129 pgs: 129 active+clean; 422 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 197 op/s 2024-03-04T09:06:05.157 INFO:tasks.workunit.client.0.smithi071.stdout: 1 767935 11.22 MB/sec execute 232 sec latency 34.376 ms 2024-03-04T09:06:06.157 INFO:tasks.workunit.client.0.smithi071.stdout: 1 770595 11.24 MB/sec execute 233 sec latency 18.748 ms 2024-03-04T09:06:06.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:06 smithi150 ceph-mon[36807]: pgmap v218: 129 pgs: 129 active+clean; 425 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 233 op/s 2024-03-04T09:06:06.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:06 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:06:06.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:06 smithi158 ceph-mon[37241]: pgmap v218: 129 pgs: 129 active+clean; 425 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 233 op/s 2024-03-04T09:06:06.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:06 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:06:06.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:06 smithi071 ceph-mon[32789]: pgmap v218: 129 pgs: 129 active+clean; 425 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 233 op/s 2024-03-04T09:06:06.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:06 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:06:07.157 INFO:tasks.workunit.client.0.smithi071.stdout: 1 772018 11.22 MB/sec execute 234 sec latency 71.009 ms 2024-03-04T09:06:07.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:07 smithi150 ceph-mon[36807]: osdmap e97: 12 total, 12 up, 12 in 2024-03-04T09:06:07.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:07 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:06:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:07 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:06:07.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:07 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:06:07.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:07 smithi158 ceph-mon[37241]: osdmap e97: 12 total, 12 up, 12 in 2024-03-04T09:06:07.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:07 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:06:07.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:07 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:06:07.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:07 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:06:07.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:07 smithi071 ceph-mon[32789]: osdmap e97: 12 total, 12 up, 12 in 2024-03-04T09:06:07.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:07 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:06:07.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:07 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:06:07.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:07 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:06:08.157 INFO:tasks.workunit.client.0.smithi071.stdout: 1 774372 11.23 MB/sec execute 235 sec latency 72.374 ms 2024-03-04T09:06:08.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:08 smithi150 ceph-mon[36807]: pgmap v220: 129 pgs: 129 active+clean; 429 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 260 op/s 2024-03-04T09:06:08.301 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:08 smithi158 ceph-mon[37241]: pgmap v220: 129 pgs: 129 active+clean; 429 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 260 op/s 2024-03-04T09:06:08.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:08 smithi071 ceph-mon[32789]: pgmap v220: 129 pgs: 129 active+clean; 429 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 260 op/s 2024-03-04T09:06:09.157 INFO:tasks.workunit.client.0.smithi071.stdout: 1 776497 11.22 MB/sec execute 236 sec latency 16.169 ms 2024-03-04T09:06:10.157 INFO:tasks.workunit.client.0.smithi071.stdout: 1 777027 11.19 MB/sec execute 237 sec latency 841.024 ms 2024-03-04T09:06:10.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:10 smithi150 ceph-mon[36807]: pgmap v221: 129 pgs: 129 active+clean; 429 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 260 op/s 2024-03-04T09:06:10.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:10 smithi158 ceph-mon[37241]: pgmap v221: 129 pgs: 129 active+clean; 429 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 260 op/s 2024-03-04T09:06:10.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:10 smithi071 ceph-mon[32789]: pgmap v221: 129 pgs: 129 active+clean; 429 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 260 op/s 2024-03-04T09:06:11.157 INFO:tasks.workunit.client.0.smithi071.stdout: 1 777027 11.15 MB/sec execute 238 sec latency 1841.145 ms 2024-03-04T09:06:12.157 INFO:tasks.workunit.client.0.smithi071.stdout: 1 777027 11.10 MB/sec execute 239 sec latency 2841.250 ms 2024-03-04T09:06:13.157 INFO:tasks.workunit.client.0.smithi071.stdout: 1 777027 11.05 MB/sec execute 240 sec latency 3841.308 ms 2024-03-04T09:06:13.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:13 smithi071 ceph-mon[32789]: pgmap v222: 129 pgs: 129 active+clean; 435 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.3 MiB/s wr, 265 op/s 2024-03-04T09:06:14.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:13 smithi150 ceph-mon[36807]: pgmap v222: 129 pgs: 129 active+clean; 435 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.3 MiB/s wr, 265 op/s 2024-03-04T09:06:14.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:13 smithi158 ceph-mon[37241]: pgmap v222: 129 pgs: 129 active+clean; 435 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.3 MiB/s wr, 265 op/s 2024-03-04T09:06:14.157 INFO:tasks.workunit.client.0.smithi071.stdout: 1 777986 11.03 MB/sec execute 241 sec latency 4327.849 ms 2024-03-04T09:06:14.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:14 smithi071 ceph-mon[32789]: pgmap v223: 129 pgs: 129 active+clean; 436 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 239 op/s 2024-03-04T09:06:15.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:14 smithi150 ceph-mon[36807]: pgmap v223: 129 pgs: 129 active+clean; 436 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 239 op/s 2024-03-04T09:06:15.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:14 smithi158 ceph-mon[37241]: pgmap v223: 129 pgs: 129 active+clean; 436 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 239 op/s 2024-03-04T09:06:15.158 INFO:tasks.workunit.client.0.smithi071.stdout: 1 779992 11.02 MB/sec execute 242 sec latency 23.711 ms 2024-03-04T09:06:15.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:15 smithi071 ceph-mon[32789]: osdmap e98: 12 total, 12 up, 12 in 2024-03-04T09:06:15.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:15 smithi071 ceph-mon[32789]: pgmap v225: 129 pgs: 129 active+clean; 438 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 118 op/s 2024-03-04T09:06:16.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:15 smithi150 ceph-mon[36807]: osdmap e98: 12 total, 12 up, 12 in 2024-03-04T09:06:16.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:15 smithi150 ceph-mon[36807]: pgmap v225: 129 pgs: 129 active+clean; 438 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 118 op/s 2024-03-04T09:06:16.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:15 smithi158 ceph-mon[37241]: osdmap e98: 12 total, 12 up, 12 in 2024-03-04T09:06:16.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:15 smithi158 ceph-mon[37241]: pgmap v225: 129 pgs: 129 active+clean; 438 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 118 op/s 2024-03-04T09:06:16.157 INFO:tasks.workunit.client.0.smithi071.stdout: 1 781758 11.02 MB/sec execute 243 sec latency 156.688 ms 2024-03-04T09:06:17.158 INFO:tasks.workunit.client.0.smithi071.stdout: 1 783747 11.01 MB/sec execute 244 sec latency 56.475 ms 2024-03-04T09:06:18.158 INFO:tasks.workunit.client.0.smithi071.stdout: 1 785608 11.01 MB/sec execute 245 sec latency 33.964 ms 2024-03-04T09:06:18.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:18 smithi150 ceph-mon[36807]: pgmap v226: 129 pgs: 129 active+clean; 441 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 148 op/s 2024-03-04T09:06:18.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:18 smithi158 ceph-mon[37241]: pgmap v226: 129 pgs: 129 active+clean; 441 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 148 op/s 2024-03-04T09:06:18.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:18 smithi071 ceph-mon[32789]: pgmap v226: 129 pgs: 129 active+clean; 441 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 148 op/s 2024-03-04T09:06:19.158 INFO:tasks.workunit.client.0.smithi071.stdout: 1 788365 11.04 MB/sec execute 246 sec latency 34.317 ms 2024-03-04T09:06:20.158 INFO:tasks.workunit.client.0.smithi071.stdout: 1 790123 11.02 MB/sec execute 247 sec latency 16.217 ms 2024-03-04T09:06:20.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:20 smithi150 ceph-mon[36807]: pgmap v227: 129 pgs: 129 active+clean; 441 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 148 op/s 2024-03-04T09:06:20.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:20 smithi158 ceph-mon[37241]: pgmap v227: 129 pgs: 129 active+clean; 441 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 148 op/s 2024-03-04T09:06:20.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:20 smithi071 ceph-mon[32789]: pgmap v227: 129 pgs: 129 active+clean; 441 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 148 op/s 2024-03-04T09:06:21.158 INFO:tasks.workunit.client.0.smithi071.stdout: 1 792449 11.03 MB/sec execute 248 sec latency 45.266 ms 2024-03-04T09:06:22.158 INFO:tasks.workunit.client.0.smithi071.stdout: 1 794531 11.03 MB/sec execute 249 sec latency 49.672 ms 2024-03-04T09:06:22.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:22 smithi150 ceph-mon[36807]: pgmap v228: 129 pgs: 129 active+clean; 446 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 169 op/s 2024-03-04T09:06:22.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:22 smithi158 ceph-mon[37241]: pgmap v228: 129 pgs: 129 active+clean; 446 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 169 op/s 2024-03-04T09:06:22.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:22 smithi071 ceph-mon[32789]: pgmap v228: 129 pgs: 129 active+clean; 446 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 169 op/s 2024-03-04T09:06:23.158 INFO:tasks.workunit.client.0.smithi071.stdout: 1 796247 11.02 MB/sec execute 250 sec latency 29.643 ms 2024-03-04T09:06:24.158 INFO:tasks.workunit.client.0.smithi071.stdout: 1 799002 11.05 MB/sec execute 251 sec latency 16.288 ms 2024-03-04T09:06:24.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:24 smithi150 ceph-mon[36807]: pgmap v229: 129 pgs: 129 active+clean; 446 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 162 op/s 2024-03-04T09:06:24.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:24 smithi158 ceph-mon[37241]: pgmap v229: 129 pgs: 129 active+clean; 446 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 162 op/s 2024-03-04T09:06:24.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:24 smithi071 ceph-mon[32789]: pgmap v229: 129 pgs: 129 active+clean; 446 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 162 op/s 2024-03-04T09:06:25.158 INFO:tasks.workunit.client.0.smithi071.stdout: 1 800557 11.03 MB/sec execute 252 sec latency 31.941 ms 2024-03-04T09:06:26.158 INFO:tasks.workunit.client.0.smithi071.stdout: 1 802851 11.04 MB/sec execute 253 sec latency 42.899 ms 2024-03-04T09:06:26.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:26 smithi150 ceph-mon[36807]: pgmap v230: 129 pgs: 129 active+clean; 454 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.2 MiB/s wr, 227 op/s 2024-03-04T09:06:26.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:26 smithi158 ceph-mon[37241]: pgmap v230: 129 pgs: 129 active+clean; 454 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.2 MiB/s wr, 227 op/s 2024-03-04T09:06:26.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:26 smithi071 ceph-mon[32789]: pgmap v230: 129 pgs: 129 active+clean; 454 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.2 MiB/s wr, 227 op/s 2024-03-04T09:06:27.158 INFO:tasks.workunit.client.0.smithi071.stdout: 1 804552 11.02 MB/sec execute 254 sec latency 133.918 ms 2024-03-04T09:06:28.158 INFO:tasks.workunit.client.0.smithi071.stdout: 1 806526 11.03 MB/sec execute 255 sec latency 143.165 ms 2024-03-04T09:06:28.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:28 smithi150 ceph-mon[36807]: pgmap v231: 129 pgs: 129 active+clean; 446 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.3 MiB/s wr, 224 op/s 2024-03-04T09:06:28.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:28 smithi158 ceph-mon[37241]: pgmap v231: 129 pgs: 129 active+clean; 446 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.3 MiB/s wr, 224 op/s 2024-03-04T09:06:28.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:28 smithi071 ceph-mon[32789]: pgmap v231: 129 pgs: 129 active+clean; 446 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.3 MiB/s wr, 224 op/s 2024-03-04T09:06:29.158 INFO:tasks.workunit.client.0.smithi071.stdout: 1 808713 11.03 MB/sec execute 256 sec latency 57.831 ms 2024-03-04T09:06:30.158 INFO:tasks.workunit.client.0.smithi071.stdout: 1 810665 11.02 MB/sec execute 257 sec latency 28.007 ms 2024-03-04T09:06:30.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:30 smithi150 ceph-mon[36807]: pgmap v232: 129 pgs: 129 active+clean; 446 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 188 op/s 2024-03-04T09:06:30.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:30 smithi158 ceph-mon[37241]: pgmap v232: 129 pgs: 129 active+clean; 446 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 188 op/s 2024-03-04T09:06:30.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:30 smithi071 ceph-mon[32789]: pgmap v232: 129 pgs: 129 active+clean; 446 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 188 op/s 2024-03-04T09:06:31.158 INFO:tasks.workunit.client.0.smithi071.stdout: 1 813273 11.05 MB/sec execute 258 sec latency 22.377 ms 2024-03-04T09:06:32.158 INFO:tasks.workunit.client.0.smithi071.stdout: 1 814905 11.03 MB/sec execute 259 sec latency 45.247 ms 2024-03-04T09:06:32.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:32 smithi150 ceph-mon[36807]: pgmap v233: 129 pgs: 129 active+clean; 427 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.6 MiB/s wr, 255 op/s 2024-03-04T09:06:32.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:32 smithi158 ceph-mon[37241]: pgmap v233: 129 pgs: 129 active+clean; 427 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.6 MiB/s wr, 255 op/s 2024-03-04T09:06:32.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:32 smithi071 ceph-mon[32789]: pgmap v233: 129 pgs: 129 active+clean; 427 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.6 MiB/s wr, 255 op/s 2024-03-04T09:06:33.158 INFO:tasks.workunit.client.0.smithi071.stdout: 1 817067 11.04 MB/sec execute 260 sec latency 99.728 ms 2024-03-04T09:06:34.159 INFO:tasks.workunit.client.0.smithi071.stdout: 1 819202 11.03 MB/sec execute 261 sec latency 48.000 ms 2024-03-04T09:06:34.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:34 smithi150 ceph-mon[36807]: pgmap v234: 129 pgs: 129 active+clean; 431 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 199 op/s 2024-03-04T09:06:34.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:34 smithi158 ceph-mon[37241]: pgmap v234: 129 pgs: 129 active+clean; 431 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 199 op/s 2024-03-04T09:06:34.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:34 smithi071 ceph-mon[32789]: pgmap v234: 129 pgs: 129 active+clean; 431 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 199 op/s 2024-03-04T09:06:35.159 INFO:tasks.workunit.client.0.smithi071.stdout: 1 821116 11.03 MB/sec execute 262 sec latency 31.725 ms 2024-03-04T09:06:36.159 INFO:tasks.workunit.client.0.smithi071.stdout: 1 823844 11.05 MB/sec execute 263 sec latency 16.366 ms 2024-03-04T09:06:36.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:36 smithi150 ceph-mon[36807]: pgmap v235: 129 pgs: 129 active+clean; 438 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.4 MiB/s wr, 249 op/s 2024-03-04T09:06:36.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:36 smithi158 ceph-mon[37241]: pgmap v235: 129 pgs: 129 active+clean; 438 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.4 MiB/s wr, 249 op/s 2024-03-04T09:06:36.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:36 smithi071 ceph-mon[32789]: pgmap v235: 129 pgs: 129 active+clean; 438 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.4 MiB/s wr, 249 op/s 2024-03-04T09:06:37.160 INFO:tasks.workunit.client.0.smithi071.stdout: 1 825460 11.03 MB/sec execute 264 sec latency 21.975 ms 2024-03-04T09:06:38.159 INFO:tasks.workunit.client.0.smithi071.stdout: 1 827622 11.04 MB/sec execute 265 sec latency 52.475 ms 2024-03-04T09:06:38.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:38 smithi150 ceph-mon[36807]: pgmap v236: 129 pgs: 129 active+clean; 443 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 235 op/s 2024-03-04T09:06:38.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:38 smithi158 ceph-mon[37241]: pgmap v236: 129 pgs: 129 active+clean; 443 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 235 op/s 2024-03-04T09:06:38.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:38 smithi071 ceph-mon[32789]: pgmap v236: 129 pgs: 129 active+clean; 443 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 235 op/s 2024-03-04T09:06:39.159 INFO:tasks.workunit.client.0.smithi071.stdout: 1 829567 11.03 MB/sec execute 266 sec latency 38.468 ms 2024-03-04T09:06:40.159 INFO:tasks.workunit.client.0.smithi071.stdout: 1 831765 11.04 MB/sec execute 267 sec latency 11.337 ms 2024-03-04T09:06:40.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:40 smithi150 ceph-mon[36807]: pgmap v237: 129 pgs: 129 active+clean; 443 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.5 MiB/s wr, 206 op/s 2024-03-04T09:06:40.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:40 smithi158 ceph-mon[37241]: pgmap v237: 129 pgs: 129 active+clean; 443 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.5 MiB/s wr, 206 op/s 2024-03-04T09:06:40.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:40 smithi071 ceph-mon[32789]: pgmap v237: 129 pgs: 129 active+clean; 443 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.5 MiB/s wr, 206 op/s 2024-03-04T09:06:41.159 INFO:tasks.workunit.client.0.smithi071.stdout: 1 834486 11.06 MB/sec execute 268 sec latency 29.658 ms 2024-03-04T09:06:42.159 INFO:tasks.workunit.client.0.smithi071.stdout: 1 836247 11.04 MB/sec execute 269 sec latency 23.756 ms 2024-03-04T09:06:42.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:42 smithi158 ceph-mon[37241]: pgmap v238: 129 pgs: 129 active+clean; 451 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 285 op/s 2024-03-04T09:06:42.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:42 smithi071 ceph-mon[32789]: pgmap v238: 129 pgs: 129 active+clean; 451 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 285 op/s 2024-03-04T09:06:42.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:42 smithi150 ceph-mon[36807]: pgmap v238: 129 pgs: 129 active+clean; 451 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 285 op/s 2024-03-04T09:06:43.159 INFO:tasks.workunit.client.0.smithi071.stdout: 1 838495 11.05 MB/sec execute 270 sec latency 55.815 ms 2024-03-04T09:06:44.159 INFO:tasks.workunit.client.0.smithi071.stdout: 1 841093 11.06 MB/sec execute 271 sec latency 21.070 ms 2024-03-04T09:06:44.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:44 smithi158 ceph-mon[37241]: pgmap v239: 129 pgs: 129 active+clean; 451 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.9 MiB/s wr, 228 op/s 2024-03-04T09:06:44.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:44 smithi071 ceph-mon[32789]: pgmap v239: 129 pgs: 129 active+clean; 451 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.9 MiB/s wr, 228 op/s 2024-03-04T09:06:44.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:44 smithi150 ceph-mon[36807]: pgmap v239: 129 pgs: 129 active+clean; 451 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.9 MiB/s wr, 228 op/s 2024-03-04T09:06:45.159 INFO:tasks.workunit.client.0.smithi071.stdout: 1 843107 11.06 MB/sec execute 272 sec latency 20.349 ms 2024-03-04T09:06:46.159 INFO:tasks.workunit.client.0.smithi071.stdout: 1 845614 11.07 MB/sec execute 273 sec latency 14.713 ms 2024-03-04T09:06:46.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:46 smithi158 ceph-mon[37241]: pgmap v240: 129 pgs: 129 active+clean; 453 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 221 op/s 2024-03-04T09:06:46.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:46 smithi071 ceph-mon[32789]: pgmap v240: 129 pgs: 129 active+clean; 453 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 221 op/s 2024-03-04T09:06:46.534 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:46 smithi150 ceph-mon[36807]: pgmap v240: 129 pgs: 129 active+clean; 453 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 221 op/s 2024-03-04T09:06:47.159 INFO:tasks.workunit.client.0.smithi071.stdout: 1 848017 11.08 MB/sec execute 274 sec latency 21.730 ms 2024-03-04T09:06:48.159 INFO:tasks.workunit.client.0.smithi071.stdout: 1 849718 11.07 MB/sec execute 275 sec latency 42.736 ms 2024-03-04T09:06:48.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:48 smithi158 ceph-mon[37241]: pgmap v241: 129 pgs: 129 active+clean; 462 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.0 MiB/s wr, 240 op/s 2024-03-04T09:06:48.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:48 smithi071 ceph-mon[32789]: pgmap v241: 129 pgs: 129 active+clean; 462 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.0 MiB/s wr, 240 op/s 2024-03-04T09:06:48.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:48 smithi150 ceph-mon[36807]: pgmap v241: 129 pgs: 129 active+clean; 462 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.0 MiB/s wr, 240 op/s 2024-03-04T09:06:49.159 INFO:tasks.workunit.client.0.smithi071.stdout: 1 852223 11.09 MB/sec execute 276 sec latency 59.952 ms 2024-03-04T09:06:50.159 INFO:tasks.workunit.client.0.smithi071.stdout: 1 854005 11.07 MB/sec execute 277 sec latency 16.517 ms 2024-03-04T09:06:50.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:50 smithi158 ceph-mon[37241]: pgmap v242: 129 pgs: 129 active+clean; 462 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 187 op/s 2024-03-04T09:06:50.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:50 smithi071 ceph-mon[32789]: pgmap v242: 129 pgs: 129 active+clean; 462 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 187 op/s 2024-03-04T09:06:50.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:50 smithi150 ceph-mon[36807]: pgmap v242: 129 pgs: 129 active+clean; 462 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 187 op/s 2024-03-04T09:06:51.159 INFO:tasks.workunit.client.0.smithi071.stdout: 1 856393 11.08 MB/sec execute 278 sec latency 15.797 ms 2024-03-04T09:06:52.159 INFO:tasks.workunit.client.0.smithi071.stdout: 1 859012 11.09 MB/sec execute 279 sec latency 23.833 ms 2024-03-04T09:06:52.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:52 smithi158 ceph-mon[37241]: pgmap v243: 129 pgs: 129 active+clean; 464 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.8 MiB/s wr, 221 op/s 2024-03-04T09:06:52.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:52 smithi071 ceph-mon[32789]: pgmap v243: 129 pgs: 129 active+clean; 464 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.8 MiB/s wr, 221 op/s 2024-03-04T09:06:52.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:52 smithi150 ceph-mon[36807]: pgmap v243: 129 pgs: 129 active+clean; 464 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.8 MiB/s wr, 221 op/s 2024-03-04T09:06:53.159 INFO:tasks.workunit.client.0.smithi071.stdout: 1 860434 11.08 MB/sec execute 280 sec latency 46.173 ms 2024-03-04T09:06:54.159 INFO:tasks.workunit.client.0.smithi071.stdout: 1 862973 11.10 MB/sec execute 281 sec latency 52.938 ms 2024-03-04T09:06:54.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:54 smithi158 ceph-mon[37241]: pgmap v244: 129 pgs: 129 active+clean; 472 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 201 op/s 2024-03-04T09:06:54.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:54 smithi071 ceph-mon[32789]: pgmap v244: 129 pgs: 129 active+clean; 472 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 201 op/s 2024-03-04T09:06:54.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:54 smithi150 ceph-mon[36807]: pgmap v244: 129 pgs: 129 active+clean; 472 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 201 op/s 2024-03-04T09:06:55.159 INFO:tasks.workunit.client.0.smithi071.stdout: 1 864885 11.08 MB/sec execute 282 sec latency 16.234 ms 2024-03-04T09:06:56.159 INFO:tasks.workunit.client.0.smithi071.stdout: 1 866912 11.09 MB/sec execute 283 sec latency 72.016 ms 2024-03-04T09:06:56.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:56 smithi158 ceph-mon[37241]: pgmap v245: 129 pgs: 129 active+clean; 472 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 210 op/s 2024-03-04T09:06:56.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:56 smithi071 ceph-mon[32789]: pgmap v245: 129 pgs: 129 active+clean; 472 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 210 op/s 2024-03-04T09:06:56.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:56 smithi150 ceph-mon[36807]: pgmap v245: 129 pgs: 129 active+clean; 472 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 210 op/s 2024-03-04T09:06:57.159 INFO:tasks.workunit.client.0.smithi071.stdout: 1 869272 11.09 MB/sec execute 284 sec latency 30.819 ms 2024-03-04T09:06:58.159 INFO:tasks.workunit.client.0.smithi071.stdout: 1 870904 11.08 MB/sec execute 285 sec latency 59.775 ms 2024-03-04T09:06:58.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:06:58 smithi158 ceph-mon[37241]: pgmap v246: 129 pgs: 129 active+clean; 481 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.5 MiB/s wr, 269 op/s 2024-03-04T09:06:58.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:06:58 smithi071 ceph-mon[32789]: pgmap v246: 129 pgs: 129 active+clean; 481 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.5 MiB/s wr, 269 op/s 2024-03-04T09:06:58.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:06:58 smithi150 ceph-mon[36807]: pgmap v246: 129 pgs: 129 active+clean; 481 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.5 MiB/s wr, 269 op/s 2024-03-04T09:06:59.160 INFO:tasks.workunit.client.0.smithi071.stdout: 1 873545 11.10 MB/sec execute 286 sec latency 53.746 ms 2024-03-04T09:07:00.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T09:07:00.146 INFO:tasks.workunit.client.0.smithi071.stdout:[875434] write failed on handle 9905 (Resource temporarily unavailable) 2024-03-04T09:07:00.146 INFO:tasks.workunit.client.0.smithi071.stdout:Child failed with status 1 2024-03-04T09:07:00.147 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-03-04T09:07:00.147 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-04T09:07:00.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:00 smithi158 ceph-mon[37241]: pgmap v247: 129 pgs: 129 active+clean; 481 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2024-03-04T09:07:00.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:00 smithi071 ceph-mon[32789]: pgmap v247: 129 pgs: 129 active+clean; 481 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2024-03-04T09:07:00.458 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi071 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-03-04T09:07:00.678 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=8abd448fa5d64856a6110068fbefbf37 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi071 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-03-04T09:07:00.681 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-04T09:07:00.690 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-04T09:07:00.691 INFO:teuthology.task.exec:Executing custom commands... 2024-03-04T09:07:00.691 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi071.front.sepia.ceph.com 2024-03-04T09:07:00.691 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-04T09:07:00.713 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:00 smithi150 ceph-mon[36807]: pgmap v247: 129 pgs: 129 active+clean; 481 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2024-03-04T09:07:00.742 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-04T09:07:02.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:02 smithi158 ceph-mon[37241]: pgmap v248: 129 pgs: 129 active+clean; 485 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 256 op/s 2024-03-04T09:07:02.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:02 smithi071 ceph-mon[32789]: pgmap v248: 129 pgs: 129 active+clean; 485 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 256 op/s 2024-03-04T09:07:02.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:02 smithi150 ceph-mon[36807]: pgmap v248: 129 pgs: 129 active+clean; 485 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 256 op/s 2024-03-04T09:07:04.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:04 smithi071 ceph-mon[32789]: pgmap v249: 129 pgs: 129 active+clean; 488 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 250 op/s 2024-03-04T09:07:04.532 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:04 smithi150 ceph-mon[36807]: pgmap v249: 129 pgs: 129 active+clean; 488 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 250 op/s 2024-03-04T09:07:04.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:04 smithi158 ceph-mon[37241]: pgmap v249: 129 pgs: 129 active+clean; 488 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 250 op/s 2024-03-04T09:07:06.331 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:06 smithi150 ceph-mon[36807]: pgmap v250: 129 pgs: 129 active+clean; 494 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 209 op/s 2024-03-04T09:07:06.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:06 smithi071 ceph-mon[32789]: pgmap v250: 129 pgs: 129 active+clean; 494 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 209 op/s 2024-03-04T09:07:06.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:06 smithi158 ceph-mon[37241]: pgmap v250: 129 pgs: 129 active+clean; 494 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 209 op/s 2024-03-04T09:07:07.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:07 smithi071 ceph-mon[32789]: osdmap e99: 12 total, 12 up, 12 in 2024-03-04T09:07:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:07 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:07:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:07 smithi071 ceph-mon[32789]: pgmap v252: 129 pgs: 129 active+clean; 497 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 145 op/s 2024-03-04T09:07:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:07 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:07:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:07 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:07:07.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:07 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:07:07.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:07 smithi150 ceph-mon[36807]: osdmap e99: 12 total, 12 up, 12 in 2024-03-04T09:07:07.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:07 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:07:07.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:07 smithi150 ceph-mon[36807]: pgmap v252: 129 pgs: 129 active+clean; 497 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 145 op/s 2024-03-04T09:07:07.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:07 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:07:07.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:07 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:07:07.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:07 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:07:07.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:07 smithi158 ceph-mon[37241]: osdmap e99: 12 total, 12 up, 12 in 2024-03-04T09:07:07.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:07 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:07:07.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:07 smithi158 ceph-mon[37241]: pgmap v252: 129 pgs: 129 active+clean; 497 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 145 op/s 2024-03-04T09:07:07.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:07 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:07:07.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:07 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:07:07.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:07 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:07:10.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:10 smithi150 ceph-mon[36807]: pgmap v253: 129 pgs: 129 active+clean; 497 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 145 op/s 2024-03-04T09:07:10.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:10 smithi158 ceph-mon[37241]: pgmap v253: 129 pgs: 129 active+clean; 497 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 145 op/s 2024-03-04T09:07:10.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:10 smithi071 ceph-mon[32789]: pgmap v253: 129 pgs: 129 active+clean; 497 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 145 op/s 2024-03-04T09:07:12.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:12 smithi150 ceph-mon[36807]: pgmap v254: 129 pgs: 129 active+clean; 494 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 2.1 MiB/s wr, 94 op/s 2024-03-04T09:07:12.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:12 smithi158 ceph-mon[37241]: pgmap v254: 129 pgs: 129 active+clean; 494 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 2.1 MiB/s wr, 94 op/s 2024-03-04T09:07:12.429 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:12 smithi071 ceph-mon[32789]: pgmap v254: 129 pgs: 129 active+clean; 494 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 2.1 MiB/s wr, 94 op/s 2024-03-04T09:07:14.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:14 smithi150 ceph-mon[36807]: pgmap v255: 129 pgs: 129 active+clean; 494 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.9 MiB/s wr, 69 op/s 2024-03-04T09:07:14.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:14 smithi158 ceph-mon[37241]: pgmap v255: 129 pgs: 129 active+clean; 494 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.9 MiB/s wr, 69 op/s 2024-03-04T09:07:14.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:14 smithi071 ceph-mon[32789]: pgmap v255: 129 pgs: 129 active+clean; 494 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.9 MiB/s wr, 69 op/s 2024-03-04T09:07:15.921 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:15 smithi071 ceph-mon[32789]: osdmap e100: 12 total, 12 up, 12 in 2024-03-04T09:07:15.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:15 smithi071 ceph-mon[32789]: pgmap v257: 129 pgs: 129 active+clean; 493 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 1.7 MiB/s wr, 61 op/s 2024-03-04T09:07:16.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:15 smithi150 ceph-mon[36807]: osdmap e100: 12 total, 12 up, 12 in 2024-03-04T09:07:16.029 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:15 smithi150 ceph-mon[36807]: pgmap v257: 129 pgs: 129 active+clean; 493 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 1.7 MiB/s wr, 61 op/s 2024-03-04T09:07:16.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:15 smithi158 ceph-mon[37241]: osdmap e100: 12 total, 12 up, 12 in 2024-03-04T09:07:16.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:15 smithi158 ceph-mon[37241]: pgmap v257: 129 pgs: 129 active+clean; 493 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 1.7 MiB/s wr, 61 op/s 2024-03-04T09:07:18.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:18 smithi150 ceph-mon[36807]: pgmap v258: 129 pgs: 129 active+clean; 486 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 2.0 MiB/s wr, 83 op/s 2024-03-04T09:07:18.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:18 smithi158 ceph-mon[37241]: pgmap v258: 129 pgs: 129 active+clean; 486 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 2.0 MiB/s wr, 83 op/s 2024-03-04T09:07:18.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:18 smithi071 ceph-mon[32789]: pgmap v258: 129 pgs: 129 active+clean; 486 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 2.0 MiB/s wr, 83 op/s 2024-03-04T09:07:20.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:20 smithi150 ceph-mon[36807]: pgmap v259: 129 pgs: 129 active+clean; 486 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 2.0 MiB/s wr, 83 op/s 2024-03-04T09:07:20.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:20 smithi158 ceph-mon[37241]: pgmap v259: 129 pgs: 129 active+clean; 486 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 2.0 MiB/s wr, 83 op/s 2024-03-04T09:07:20.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:20 smithi071 ceph-mon[32789]: pgmap v259: 129 pgs: 129 active+clean; 486 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 2.0 MiB/s wr, 83 op/s 2024-03-04T09:07:22.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:22 smithi150 ceph-mon[36807]: pgmap v260: 129 pgs: 129 active+clean; 493 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 2.2 MiB/s wr, 73 op/s 2024-03-04T09:07:22.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:22 smithi158 ceph-mon[37241]: pgmap v260: 129 pgs: 129 active+clean; 493 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 2.2 MiB/s wr, 73 op/s 2024-03-04T09:07:22.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:22 smithi071 ceph-mon[32789]: pgmap v260: 129 pgs: 129 active+clean; 493 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 2.2 MiB/s wr, 73 op/s 2024-03-04T09:07:24.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:24 smithi158 ceph-mon[37241]: pgmap v261: 129 pgs: 129 active+clean; 494 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 2.1 MiB/s wr, 69 op/s 2024-03-04T09:07:24.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:24 smithi071 ceph-mon[32789]: pgmap v261: 129 pgs: 129 active+clean; 494 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 2.1 MiB/s wr, 69 op/s 2024-03-04T09:07:24.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:24 smithi150 ceph-mon[36807]: pgmap v261: 129 pgs: 129 active+clean; 494 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 2.1 MiB/s wr, 69 op/s 2024-03-04T09:07:26.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:26 smithi158 ceph-mon[37241]: pgmap v262: 129 pgs: 129 active+clean; 494 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.8 MiB/s wr, 59 op/s 2024-03-04T09:07:26.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:26 smithi071 ceph-mon[32789]: pgmap v262: 129 pgs: 129 active+clean; 494 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.8 MiB/s wr, 59 op/s 2024-03-04T09:07:26.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:26 smithi150 ceph-mon[36807]: pgmap v262: 129 pgs: 129 active+clean; 494 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.8 MiB/s wr, 59 op/s 2024-03-04T09:07:28.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:28 smithi158 ceph-mon[37241]: pgmap v263: 129 pgs: 129 active+clean; 493 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 2.1 MiB/s wr, 71 op/s 2024-03-04T09:07:28.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:28 smithi071 ceph-mon[32789]: pgmap v263: 129 pgs: 129 active+clean; 493 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 2.1 MiB/s wr, 71 op/s 2024-03-04T09:07:28.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:28 smithi150 ceph-mon[36807]: pgmap v263: 129 pgs: 129 active+clean; 493 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 2.1 MiB/s wr, 71 op/s 2024-03-04T09:07:30.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:30 smithi158 ceph-mon[37241]: pgmap v264: 129 pgs: 129 active+clean; 493 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 1.6 MiB/s wr, 47 op/s 2024-03-04T09:07:30.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:30 smithi071 ceph-mon[32789]: pgmap v264: 129 pgs: 129 active+clean; 493 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 1.6 MiB/s wr, 47 op/s 2024-03-04T09:07:30.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:30 smithi150 ceph-mon[36807]: pgmap v264: 129 pgs: 129 active+clean; 493 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 1.6 MiB/s wr, 47 op/s 2024-03-04T09:07:32.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:32 smithi158 ceph-mon[37241]: pgmap v265: 129 pgs: 129 active+clean; 494 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 2.4 MiB/s wr, 68 op/s 2024-03-04T09:07:32.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:32 smithi071 ceph-mon[32789]: pgmap v265: 129 pgs: 129 active+clean; 494 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 2.4 MiB/s wr, 68 op/s 2024-03-04T09:07:32.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:32 smithi150 ceph-mon[36807]: pgmap v265: 129 pgs: 129 active+clean; 494 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 2.4 MiB/s wr, 68 op/s 2024-03-04T09:07:34.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:34 smithi158 ceph-mon[37241]: pgmap v266: 129 pgs: 129 active+clean; 492 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 1.7 MiB/s wr, 61 op/s 2024-03-04T09:07:34.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:34 smithi071 ceph-mon[32789]: pgmap v266: 129 pgs: 129 active+clean; 492 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 1.7 MiB/s wr, 61 op/s 2024-03-04T09:07:34.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:34 smithi150 ceph-mon[36807]: pgmap v266: 129 pgs: 129 active+clean; 492 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 1.7 MiB/s wr, 61 op/s 2024-03-04T09:07:36.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:36 smithi158 ceph-mon[37241]: pgmap v267: 129 pgs: 129 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 1.7 MiB/s wr, 64 op/s 2024-03-04T09:07:36.331 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:36 smithi150 ceph-mon[36807]: pgmap v267: 129 pgs: 129 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 1.7 MiB/s wr, 64 op/s 2024-03-04T09:07:36.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:36 smithi071 ceph-mon[32789]: pgmap v267: 129 pgs: 129 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 1.7 MiB/s wr, 64 op/s 2024-03-04T09:07:38.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:38 smithi158 ceph-mon[37241]: pgmap v268: 129 pgs: 129 active+clean; 486 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.5 MiB/s wr, 68 op/s 2024-03-04T09:07:38.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:38 smithi071 ceph-mon[32789]: pgmap v268: 129 pgs: 129 active+clean; 486 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.5 MiB/s wr, 68 op/s 2024-03-04T09:07:38.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:38 smithi150 ceph-mon[36807]: pgmap v268: 129 pgs: 129 active+clean; 486 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.5 MiB/s wr, 68 op/s 2024-03-04T09:07:40.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:40 smithi158 ceph-mon[37241]: pgmap v269: 129 pgs: 129 active+clean; 486 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 1.1 MiB/s wr, 47 op/s 2024-03-04T09:07:40.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:40 smithi071 ceph-mon[32789]: pgmap v269: 129 pgs: 129 active+clean; 486 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 1.1 MiB/s wr, 47 op/s 2024-03-04T09:07:40.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:40 smithi150 ceph-mon[36807]: pgmap v269: 129 pgs: 129 active+clean; 486 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 1.1 MiB/s wr, 47 op/s 2024-03-04T09:07:42.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:42 smithi071 ceph-mon[32789]: pgmap v270: 129 pgs: 129 active+clean; 483 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.1 MiB/s wr, 53 op/s 2024-03-04T09:07:42.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:42 smithi150 ceph-mon[36807]: pgmap v270: 129 pgs: 129 active+clean; 483 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.1 MiB/s wr, 53 op/s 2024-03-04T09:07:42.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:42 smithi158 ceph-mon[37241]: pgmap v270: 129 pgs: 129 active+clean; 483 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.1 MiB/s wr, 53 op/s 2024-03-04T09:07:44.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:44 smithi071 ceph-mon[32789]: pgmap v271: 129 pgs: 129 active+clean; 483 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 347 KiB/s wr, 33 op/s 2024-03-04T09:07:44.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:44 smithi150 ceph-mon[36807]: pgmap v271: 129 pgs: 129 active+clean; 483 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 347 KiB/s wr, 33 op/s 2024-03-04T09:07:44.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:44 smithi158 ceph-mon[37241]: pgmap v271: 129 pgs: 129 active+clean; 483 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 347 KiB/s wr, 33 op/s 2024-03-04T09:07:46.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:46 smithi071 ceph-mon[32789]: pgmap v272: 129 pgs: 129 active+clean; 483 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 309 KiB/s wr, 27 op/s 2024-03-04T09:07:46.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:46 smithi150 ceph-mon[36807]: pgmap v272: 129 pgs: 129 active+clean; 483 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 309 KiB/s wr, 27 op/s 2024-03-04T09:07:46.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:46 smithi158 ceph-mon[37241]: pgmap v272: 129 pgs: 129 active+clean; 483 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 309 KiB/s wr, 27 op/s 2024-03-04T09:07:48.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:48 smithi071 ceph-mon[32789]: pgmap v273: 129 pgs: 129 active+clean; 483 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 203 KiB/s wr, 20 op/s 2024-03-04T09:07:48.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:48 smithi150 ceph-mon[36807]: pgmap v273: 129 pgs: 129 active+clean; 483 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 203 KiB/s wr, 20 op/s 2024-03-04T09:07:48.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:48 smithi158 ceph-mon[37241]: pgmap v273: 129 pgs: 129 active+clean; 483 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 203 KiB/s wr, 20 op/s 2024-03-04T09:07:50.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:50 smithi071 ceph-mon[32789]: pgmap v274: 129 pgs: 129 active+clean; 483 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-03-04T09:07:50.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:50 smithi150 ceph-mon[36807]: pgmap v274: 129 pgs: 129 active+clean; 483 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-03-04T09:07:50.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:50 smithi158 ceph-mon[37241]: pgmap v274: 129 pgs: 129 active+clean; 483 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-03-04T09:07:52.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:52 smithi071 ceph-mon[32789]: pgmap v275: 129 pgs: 129 active+clean; 483 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-03-04T09:07:52.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:52 smithi150 ceph-mon[36807]: pgmap v275: 129 pgs: 129 active+clean; 483 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-03-04T09:07:52.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:52 smithi158 ceph-mon[37241]: pgmap v275: 129 pgs: 129 active+clean; 483 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 16 KiB/s wr, 8 op/s 2024-03-04T09:07:53.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:53 smithi071 ceph-mon[32789]: pgmap v276: 129 pgs: 129 active+clean; 483 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.6 KiB/s wr, 3 op/s 2024-03-04T09:07:53.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:53 smithi150 ceph-mon[36807]: pgmap v276: 129 pgs: 129 active+clean; 483 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.6 KiB/s wr, 3 op/s 2024-03-04T09:07:53.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:53 smithi158 ceph-mon[37241]: pgmap v276: 129 pgs: 129 active+clean; 483 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.6 KiB/s wr, 3 op/s 2024-03-04T09:07:56.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:56 smithi158 ceph-mon[37241]: pgmap v277: 129 pgs: 129 active+clean; 467 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 KiB/s wr, 2 op/s 2024-03-04T09:07:56.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:56 smithi071 ceph-mon[32789]: pgmap v277: 129 pgs: 129 active+clean; 467 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 KiB/s wr, 2 op/s 2024-03-04T09:07:56.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:56 smithi150 ceph-mon[36807]: pgmap v277: 129 pgs: 129 active+clean; 467 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 KiB/s wr, 2 op/s 2024-03-04T09:07:58.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:07:58 smithi158 ceph-mon[37241]: pgmap v278: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.4 KiB/s wr, 4 op/s 2024-03-04T09:07:58.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:07:58 smithi071 ceph-mon[32789]: pgmap v278: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.4 KiB/s wr, 4 op/s 2024-03-04T09:07:58.538 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:07:58 smithi150 ceph-mon[36807]: pgmap v278: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.4 KiB/s wr, 4 op/s 2024-03-04T09:08:00.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:00 smithi158 ceph-mon[37241]: pgmap v279: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.4 KiB/s wr, 3 op/s 2024-03-04T09:08:00.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:00 smithi071 ceph-mon[32789]: pgmap v279: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.4 KiB/s wr, 3 op/s 2024-03-04T09:08:00.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:00 smithi150 ceph-mon[36807]: pgmap v279: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.4 KiB/s wr, 3 op/s 2024-03-04T09:08:02.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:02 smithi158 ceph-mon[37241]: pgmap v280: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 37 KiB/s wr, 18 op/s 2024-03-04T09:08:02.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:02 smithi071 ceph-mon[32789]: pgmap v280: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 37 KiB/s wr, 18 op/s 2024-03-04T09:08:02.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:02 smithi150 ceph-mon[36807]: pgmap v280: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 37 KiB/s wr, 18 op/s 2024-03-04T09:08:04.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:04 smithi158 ceph-mon[37241]: pgmap v281: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-03-04T09:08:04.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:04 smithi071 ceph-mon[32789]: pgmap v281: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-03-04T09:08:04.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:04 smithi150 ceph-mon[36807]: pgmap v281: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-03-04T09:08:06.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:06 smithi158 ceph-mon[37241]: pgmap v282: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 65 KiB/s wr, 21 op/s 2024-03-04T09:08:06.331 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:06 smithi150 ceph-mon[36807]: pgmap v282: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 65 KiB/s wr, 21 op/s 2024-03-04T09:08:06.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:06 smithi071 ceph-mon[32789]: pgmap v282: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 65 KiB/s wr, 21 op/s 2024-03-04T09:08:07.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:07 smithi071 ceph-mon[32789]: osdmap e101: 12 total, 12 up, 12 in 2024-03-04T09:08:07.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:07 smithi150 ceph-mon[36807]: osdmap e101: 12 total, 12 up, 12 in 2024-03-04T09:08:07.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:07 smithi158 ceph-mon[37241]: osdmap e101: 12 total, 12 up, 12 in 2024-03-04T09:08:08.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:08 smithi071 ceph-mon[32789]: pgmap v284: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 76 KiB/s wr, 23 op/s 2024-03-04T09:08:08.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:08 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:08:08.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:08 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:08:08.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:08 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:08:08.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:08 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:08:08.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:08 smithi150 ceph-mon[36807]: pgmap v284: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 76 KiB/s wr, 23 op/s 2024-03-04T09:08:08.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:08 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:08:08.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:08 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:08:08.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:08 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:08:08.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:08 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:08:08.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:08 smithi158 ceph-mon[37241]: pgmap v284: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 76 KiB/s wr, 23 op/s 2024-03-04T09:08:08.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:08 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:08:08.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:08 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:08:08.554 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:08 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:08:08.555 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:08 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:08:10.355 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:10 smithi158 ceph-mon[37241]: pgmap v285: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 76 KiB/s wr, 23 op/s 2024-03-04T09:08:10.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:10 smithi071 ceph-mon[32789]: pgmap v285: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 76 KiB/s wr, 23 op/s 2024-03-04T09:08:10.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:10 smithi150 ceph-mon[36807]: pgmap v285: 129 pgs: 129 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 76 KiB/s wr, 23 op/s 2024-03-04T09:08:12.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:12 smithi071 ceph-mon[32789]: pgmap v286: 129 pgs: 129 active+clean; 441 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 58 KiB/s wr, 6 op/s 2024-03-04T09:08:12.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:12 smithi150 ceph-mon[36807]: pgmap v286: 129 pgs: 129 active+clean; 441 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 58 KiB/s wr, 6 op/s 2024-03-04T09:08:12.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:12 smithi158 ceph-mon[37241]: pgmap v286: 129 pgs: 129 active+clean; 441 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 58 KiB/s wr, 6 op/s 2024-03-04T09:08:14.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:14 smithi071 ceph-mon[32789]: pgmap v287: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 58 KiB/s wr, 6 op/s 2024-03-04T09:08:14.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:14 smithi150 ceph-mon[36807]: pgmap v287: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 58 KiB/s wr, 6 op/s 2024-03-04T09:08:14.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:14 smithi158 ceph-mon[37241]: pgmap v287: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 58 KiB/s wr, 6 op/s 2024-03-04T09:08:16.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:15 smithi150 ceph-mon[36807]: osdmap e102: 12 total, 12 up, 12 in 2024-03-04T09:08:16.030 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:15 smithi150 ceph-mon[36807]: pgmap v289: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 2 op/s 2024-03-04T09:08:16.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:15 smithi158 ceph-mon[37241]: osdmap e102: 12 total, 12 up, 12 in 2024-03-04T09:08:16.052 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:15 smithi158 ceph-mon[37241]: pgmap v289: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 2 op/s 2024-03-04T09:08:16.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:15 smithi071 ceph-mon[32789]: osdmap e102: 12 total, 12 up, 12 in 2024-03-04T09:08:16.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:15 smithi071 ceph-mon[32789]: pgmap v289: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 2 op/s 2024-03-04T09:08:18.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:18 smithi158 ceph-mon[37241]: pgmap v290: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 3 op/s 2024-03-04T09:08:18.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:18 smithi071 ceph-mon[32789]: pgmap v290: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 3 op/s 2024-03-04T09:08:18.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:18 smithi150 ceph-mon[36807]: pgmap v290: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 3 op/s 2024-03-04T09:08:20.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:20 smithi158 ceph-mon[37241]: pgmap v291: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 3 op/s 2024-03-04T09:08:20.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:20 smithi071 ceph-mon[32789]: pgmap v291: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 3 op/s 2024-03-04T09:08:20.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:20 smithi150 ceph-mon[36807]: pgmap v291: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 3 op/s 2024-03-04T09:08:22.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:22 smithi158 ceph-mon[37241]: pgmap v292: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.7 KiB/s wr, 2 op/s 2024-03-04T09:08:22.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:22 smithi071 ceph-mon[32789]: pgmap v292: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.7 KiB/s wr, 2 op/s 2024-03-04T09:08:22.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:22 smithi150 ceph-mon[36807]: pgmap v292: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.7 KiB/s wr, 2 op/s 2024-03-04T09:08:24.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:24 smithi158 ceph-mon[37241]: pgmap v293: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.7 KiB/s wr, 2 op/s 2024-03-04T09:08:24.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:24 smithi071 ceph-mon[32789]: pgmap v293: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.7 KiB/s wr, 2 op/s 2024-03-04T09:08:24.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:24 smithi150 ceph-mon[36807]: pgmap v293: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.7 KiB/s wr, 2 op/s 2024-03-04T09:08:26.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:26 smithi158 ceph-mon[37241]: pgmap v294: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.5 KiB/s wr, 2 op/s 2024-03-04T09:08:26.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:26 smithi071 ceph-mon[32789]: pgmap v294: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.5 KiB/s wr, 2 op/s 2024-03-04T09:08:26.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:26 smithi150 ceph-mon[36807]: pgmap v294: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.5 KiB/s wr, 2 op/s 2024-03-04T09:08:28.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:28 smithi071 ceph-mon[32789]: pgmap v295: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.0 KiB/s wr, 2 op/s 2024-03-04T09:08:28.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:28 smithi150 ceph-mon[36807]: pgmap v295: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.0 KiB/s wr, 2 op/s 2024-03-04T09:08:28.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:28 smithi158 ceph-mon[37241]: pgmap v295: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.0 KiB/s wr, 2 op/s 2024-03-04T09:08:30.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:30 smithi071 ceph-mon[32789]: pgmap v296: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-04T09:08:30.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:30 smithi150 ceph-mon[36807]: pgmap v296: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-04T09:08:30.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:30 smithi158 ceph-mon[37241]: pgmap v296: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-04T09:08:32.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:32 smithi071 ceph-mon[32789]: pgmap v297: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.0 KiB/s wr, 2 op/s 2024-03-04T09:08:32.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:32 smithi150 ceph-mon[36807]: pgmap v297: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.0 KiB/s wr, 2 op/s 2024-03-04T09:08:32.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:32 smithi158 ceph-mon[37241]: pgmap v297: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.0 KiB/s wr, 2 op/s 2024-03-04T09:08:34.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:34 smithi071 ceph-mon[32789]: pgmap v298: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-04T09:08:34.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:34 smithi150 ceph-mon[36807]: pgmap v298: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-04T09:08:34.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:34 smithi158 ceph-mon[37241]: pgmap v298: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-04T09:08:36.331 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:36 smithi150 ceph-mon[36807]: pgmap v299: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.0 KiB/s wr, 2 op/s 2024-03-04T09:08:36.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:36 smithi071 ceph-mon[32789]: pgmap v299: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.0 KiB/s wr, 2 op/s 2024-03-04T09:08:36.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:36 smithi158 ceph-mon[37241]: pgmap v299: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.0 KiB/s wr, 2 op/s 2024-03-04T09:08:38.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:38 smithi071 ceph-mon[32789]: pgmap v300: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T09:08:38.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:38 smithi150 ceph-mon[36807]: pgmap v300: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T09:08:38.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:38 smithi158 ceph-mon[37241]: pgmap v300: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T09:08:40.354 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:40 smithi158 ceph-mon[37241]: pgmap v301: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:08:40.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:40 smithi071 ceph-mon[32789]: pgmap v301: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:08:40.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:40 smithi150 ceph-mon[36807]: pgmap v301: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:08:42.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:42 smithi071 ceph-mon[32789]: pgmap v302: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T09:08:42.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:42 smithi150 ceph-mon[36807]: pgmap v302: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T09:08:42.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:42 smithi158 ceph-mon[37241]: pgmap v302: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T09:08:43.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:43 smithi071 ceph-mon[32789]: pgmap v303: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:08:43.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:43 smithi150 ceph-mon[36807]: pgmap v303: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:08:43.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:43 smithi158 ceph-mon[37241]: pgmap v303: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:08:46.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:46 smithi158 ceph-mon[37241]: pgmap v304: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-03-04T09:08:46.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:46 smithi071 ceph-mon[32789]: pgmap v304: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-03-04T09:08:46.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:46 smithi150 ceph-mon[36807]: pgmap v304: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-03-04T09:08:48.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:48 smithi071 ceph-mon[32789]: pgmap v305: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T09:08:48.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:48 smithi150 ceph-mon[36807]: pgmap v305: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T09:08:48.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:48 smithi158 ceph-mon[37241]: pgmap v305: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T09:08:50.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:50 smithi071 ceph-mon[32789]: pgmap v306: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:08:50.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:50 smithi150 ceph-mon[36807]: pgmap v306: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:08:50.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:50 smithi158 ceph-mon[37241]: pgmap v306: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:08:52.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:52 smithi071 ceph-mon[32789]: pgmap v307: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T09:08:52.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:52 smithi150 ceph-mon[36807]: pgmap v307: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T09:08:52.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:52 smithi158 ceph-mon[37241]: pgmap v307: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T09:08:54.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:54 smithi071 ceph-mon[32789]: pgmap v308: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:08:54.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:54 smithi150 ceph-mon[36807]: pgmap v308: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:08:54.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:54 smithi158 ceph-mon[37241]: pgmap v308: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:08:56.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:56 smithi071 ceph-mon[32789]: pgmap v309: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-03-04T09:08:56.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:56 smithi150 ceph-mon[36807]: pgmap v309: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-03-04T09:08:56.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:56 smithi158 ceph-mon[37241]: pgmap v309: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-03-04T09:08:57.813 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-04T09:08:57.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.931+0000 7f86fcfac640 1 -- 172.21.15.71:0/3289527244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f8100720 msgr2=0x7f86f8100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:57.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.931+0000 7f86fcfac640 1 --2- 172.21.15.71:0/3289527244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f8100720 0x7f86f8100ba0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009900 tx=0x7f86ec02bc30 comp rx=0 tx=0).stop 2024-03-04T09:08:57.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.931+0000 7f86fcfac640 1 -- 172.21.15.71:0/3289527244 shutdown_connections 2024-03-04T09:08:57.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.931+0000 7f86fcfac640 1 --2- 172.21.15.71:0/3289527244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86f81010e0 0x7f86f810d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:57.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.931+0000 7f86fcfac640 1 --2- 172.21.15.71:0/3289527244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f8100720 0x7f86f8100ba0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:57.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.931+0000 7f86fcfac640 1 --2- 172.21.15.71:0/3289527244 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86f8105470 0x7f86f8105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:57.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.931+0000 7f86fcfac640 1 -- 172.21.15.71:0/3289527244 >> 172.21.15.71:0/3289527244 conn(0x7f86f8077320 msgr2=0x7f86f8077730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:08:57.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.931+0000 7f86fcfac640 1 -- 172.21.15.71:0/3289527244 shutdown_connections 2024-03-04T09:08:57.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.931+0000 7f86fcfac640 1 -- 172.21.15.71:0/3289527244 wait complete. 2024-03-04T09:08:57.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.932+0000 7f86fcfac640 1 Processor -- start 2024-03-04T09:08:57.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.932+0000 7f86fcfac640 1 -- start start 2024-03-04T09:08:57.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.932+0000 7f86fcfac640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f8100720 0x7f86f8197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:57.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.932+0000 7f86fcfac640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86f81010e0 0x7f86f8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:57.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.932+0000 7f86fcfac640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86f8105470 0x7f86f819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:57.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.932+0000 7f86fcfac640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f8111ab0 con 0x7f86f8100720 2024-03-04T09:08:57.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.932+0000 7f86fcfac640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f8111930 con 0x7f86f81010e0 2024-03-04T09:08:57.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.932+0000 7f86fcfac640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f8111c30 con 0x7f86f8105470 2024-03-04T09:08:57.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.932+0000 7f86f659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f8100720 0x7f86f8197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:57.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.932+0000 7f86f659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f8100720 0x7f86f8197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57162/0 (socket says 172.21.15.71:57162) 2024-03-04T09:08:57.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.932+0000 7f86f659c640 1 -- 172.21.15.71:0/106382891 learned_addr learned my addr 172.21.15.71:0/106382891 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T09:08:57.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.932+0000 7f86f5d9b640 1 --2- 172.21.15.71:0/106382891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86f81010e0 0x7f86f8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:57.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.932+0000 7f86f6d9d640 1 --2- 172.21.15.71:0/106382891 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86f8105470 0x7f86f819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:57.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.932+0000 7f86f659c640 1 -- 172.21.15.71:0/106382891 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86f8105470 msgr2=0x7f86f819c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:57.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.932+0000 7f86f659c640 1 --2- 172.21.15.71:0/106382891 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86f8105470 0x7f86f819c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:57.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.932+0000 7f86f659c640 1 -- 172.21.15.71:0/106382891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86f81010e0 msgr2=0x7f86f8198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:57.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.932+0000 7f86f659c640 1 --2- 172.21.15.71:0/106382891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86f81010e0 0x7f86f8198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:57.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.932+0000 7f86f659c640 1 -- 172.21.15.71:0/106382891 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f819cec0 con 0x7f86f8100720 2024-03-04T09:08:57.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.932+0000 7f86f659c640 1 --2- 172.21.15.71:0/106382891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f8100720 0x7f86f8197ed0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f86e800cce0 tx=0x7f86e800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:08:57.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.933+0000 7f86e77fe640 1 -- 172.21.15.71:0/106382891 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8004b20 con 0x7f86f8100720 2024-03-04T09:08:57.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.933+0000 7f86fcfac640 1 -- 172.21.15.71:0/106382891 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f819d1b0 con 0x7f86f8100720 2024-03-04T09:08:57.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.933+0000 7f86e77fe640 1 -- 172.21.15.71:0/106382891 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86e8004cc0 con 0x7f86f8100720 2024-03-04T09:08:57.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.933+0000 7f86e77fe640 1 -- 172.21.15.71:0/106382891 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e800f8c0 con 0x7f86f8100720 2024-03-04T09:08:57.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.933+0000 7f86fcfac640 1 -- 172.21.15.71:0/106382891 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f81a4a40 con 0x7f86f8100720 2024-03-04T09:08:57.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.934+0000 7f86fcfac640 1 -- 172.21.15.71:0/106382891 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f804f330 con 0x7f86f8100720 2024-03-04T09:08:57.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.935+0000 7f86e77fe640 1 -- 172.21.15.71:0/106382891 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f86e800fa60 con 0x7f86f8100720 2024-03-04T09:08:57.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.935+0000 7f86e77fe640 1 --2- 172.21.15.71:0/106382891 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f86e00a1f90 0x7f86e00a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:57.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.936+0000 7f86f5d9b640 1 --2- 172.21.15.71:0/106382891 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f86e00a1f90 0x7f86e00a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:57.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.936+0000 7f86e77fe640 1 -- 172.21.15.71:0/106382891 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9467+0+0 (secure 0 0 0) 0x7f86e80ce5e0 con 0x7f86f8100720 2024-03-04T09:08:57.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.936+0000 7f86f5d9b640 1 --2- 172.21.15.71:0/106382891 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f86e00a1f90 0x7f86e00a4450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009230 tx=0x7f86ec005fd0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:08:57.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:57.940+0000 7f86e77fe640 1 -- 172.21.15.71:0/106382891 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f86e8014030 con 0x7f86f8100720 2024-03-04T09:08:58.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.137+0000 7f86fcfac640 1 -- 172.21.15.71:0/106382891 --> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f86f80fede0 con 0x7f86e00a1f90 2024-03-04T09:08:58.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.145+0000 7f86e77fe640 1 -- 172.21.15.71:0/106382891 <== mgr.15399 v2:172.21.15.71:6800/217968316 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+350 (secure 0 0 0) 0x7f86f80fede0 con 0x7f86e00a1f90 2024-03-04T09:08:58.142 INFO:teuthology.orchestra.run.smithi071.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-04T00:00:00", "created": "2024-03-04T08:58:55", "first": "2024-03-04T08:59:00", "last": "2024-03-04T09:08:00", "last_pruned": "2024-03-04T09:08:00", "created_count": 10, "pruned_count": 3, "active": true} 2024-03-04T09:08:58.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.147+0000 7f86fcfac640 1 -- 172.21.15.71:0/106382891 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f86e00a1f90 msgr2=0x7f86e00a4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:58.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.148+0000 7f86fcfac640 1 --2- 172.21.15.71:0/106382891 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f86e00a1f90 0x7f86e00a4450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009230 tx=0x7f86ec005fd0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.148+0000 7f86fcfac640 1 -- 172.21.15.71:0/106382891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f8100720 msgr2=0x7f86f8197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:58.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.148+0000 7f86fcfac640 1 --2- 172.21.15.71:0/106382891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f8100720 0x7f86f8197ed0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f86e800cce0 tx=0x7f86e800a620 comp rx=0 tx=0).stop 2024-03-04T09:08:58.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.148+0000 7f86fcfac640 1 -- 172.21.15.71:0/106382891 shutdown_connections 2024-03-04T09:08:58.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.148+0000 7f86fcfac640 1 --2- 172.21.15.71:0/106382891 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f86f8105470 0x7f86f819c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.148+0000 7f86fcfac640 1 --2- 172.21.15.71:0/106382891 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86f81010e0 0x7f86f8198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.148+0000 7f86fcfac640 1 --2- 172.21.15.71:0/106382891 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f86e00a1f90 0x7f86e00a4450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.148+0000 7f86fcfac640 1 --2- 172.21.15.71:0/106382891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f8100720 0x7f86f8197ed0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.148+0000 7f86fcfac640 1 -- 172.21.15.71:0/106382891 >> 172.21.15.71:0/106382891 conn(0x7f86f8077320 msgr2=0x7f86f80fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:08:58.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.148+0000 7f86fcfac640 1 -- 172.21.15.71:0/106382891 shutdown_connections 2024-03-04T09:08:58.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.148+0000 7f86fcfac640 1 -- 172.21.15.71:0/106382891 wait complete. 2024-03-04T09:08:58.154 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-04T09:08:58.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.261+0000 7f3f9f7ac640 1 -- 172.21.15.71:0/1617118976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f98105470 msgr2=0x7f3f98105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:58.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.261+0000 7f3f9f7ac640 1 --2- 172.21.15.71:0/1617118976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f98105470 0x7f3f98105850 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009ca0 tx=0x7f3f8802c0f0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.261+0000 7f3f9f7ac640 1 -- 172.21.15.71:0/1617118976 shutdown_connections 2024-03-04T09:08:58.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.261+0000 7f3f9f7ac640 1 --2- 172.21.15.71:0/1617118976 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f981010e0 0x7f3f9810d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.261+0000 7f3f9f7ac640 1 --2- 172.21.15.71:0/1617118976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f98100720 0x7f3f98100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.261+0000 7f3f9f7ac640 1 --2- 172.21.15.71:0/1617118976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f98105470 0x7f3f98105850 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.261+0000 7f3f9f7ac640 1 -- 172.21.15.71:0/1617118976 >> 172.21.15.71:0/1617118976 conn(0x7f3f98077320 msgr2=0x7f3f98077730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:08:58.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.261+0000 7f3f9f7ac640 1 -- 172.21.15.71:0/1617118976 shutdown_connections 2024-03-04T09:08:58.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.261+0000 7f3f9f7ac640 1 -- 172.21.15.71:0/1617118976 wait complete. 2024-03-04T09:08:58.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.261+0000 7f3f9f7ac640 1 Processor -- start 2024-03-04T09:08:58.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.261+0000 7f3f9f7ac640 1 -- start start 2024-03-04T09:08:58.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.261+0000 7f3f9f7ac640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f98100720 0x7f3f98197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:58.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.261+0000 7f3f9f7ac640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f981010e0 0x7f3f98198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:58.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.261+0000 7f3f9f7ac640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f98105470 0x7f3f9819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:58.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.261+0000 7f3f9f7ac640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f98111ab0 con 0x7f3f98105470 2024-03-04T09:08:58.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.261+0000 7f3f9f7ac640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f98111930 con 0x7f3f981010e0 2024-03-04T09:08:58.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.261+0000 7f3f9f7ac640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f98111c30 con 0x7f3f98100720 2024-03-04T09:08:58.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.262+0000 7f3f9dd49640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f98105470 0x7f3f9819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:58.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.262+0000 7f3f9dd49640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f98105470 0x7f3f9819c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57178/0 (socket says 172.21.15.71:57178) 2024-03-04T09:08:58.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.262+0000 7f3f9d548640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f98100720 0x7f3f98197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:58.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.262+0000 7f3f9dd49640 1 -- 172.21.15.71:0/1334373841 learned_addr learned my addr 172.21.15.71:0/1334373841 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T09:08:58.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.262+0000 7f3f9cd47640 1 --2- 172.21.15.71:0/1334373841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f981010e0 0x7f3f98198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:58.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.262+0000 7f3f9dd49640 1 -- 172.21.15.71:0/1334373841 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f98100720 msgr2=0x7f3f98197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:58.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.262+0000 7f3f9dd49640 1 --2- 172.21.15.71:0/1334373841 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f98100720 0x7f3f98197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.262+0000 7f3f9dd49640 1 -- 172.21.15.71:0/1334373841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f981010e0 msgr2=0x7f3f98198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:58.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.262+0000 7f3f9dd49640 1 --2- 172.21.15.71:0/1334373841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f981010e0 0x7f3f98198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.262+0000 7f3f9dd49640 1 -- 172.21.15.71:0/1334373841 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f9819cec0 con 0x7f3f98105470 2024-03-04T09:08:58.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.262+0000 7f3f9dd49640 1 --2- 172.21.15.71:0/1334373841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f98105470 0x7f3f9819c790 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000b3e0 tx=0x7f3f90009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:08:58.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.262+0000 7f3f967fc640 1 -- 172.21.15.71:0/1334373841 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f90015020 con 0x7f3f98105470 2024-03-04T09:08:58.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.262+0000 7f3f9f7ac640 1 -- 172.21.15.71:0/1334373841 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f9819d1b0 con 0x7f3f98105470 2024-03-04T09:08:58.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.262+0000 7f3f967fc640 1 -- 172.21.15.71:0/1334373841 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f90011d10 con 0x7f3f98105470 2024-03-04T09:08:58.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.262+0000 7f3f967fc640 1 -- 172.21.15.71:0/1334373841 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9000f2e0 con 0x7f3f98105470 2024-03-04T09:08:58.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.262+0000 7f3f9f7ac640 1 -- 172.21.15.71:0/1334373841 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f981a4a40 con 0x7f3f98105470 2024-03-04T09:08:58.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.263+0000 7f3f9f7ac640 1 -- 172.21.15.71:0/1334373841 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f9804f330 con 0x7f3f98105470 2024-03-04T09:08:58.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.265+0000 7f3f967fc640 1 -- 172.21.15.71:0/1334373841 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f3f90018020 con 0x7f3f98105470 2024-03-04T09:08:58.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.265+0000 7f3f967fc640 1 --2- 172.21.15.71:0/1334373841 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f3f800a1f90 0x7f3f800a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:58.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.266+0000 7f3f9d548640 1 --2- 172.21.15.71:0/1334373841 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f3f800a1f90 0x7f3f800a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:58.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.266+0000 7f3f967fc640 1 -- 172.21.15.71:0/1334373841 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9467+0+0 (secure 0 0 0) 0x7f3f900ce090 con 0x7f3f98105470 2024-03-04T09:08:58.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.266+0000 7f3f9d548640 1 --2- 172.21.15.71:0/1334373841 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f3f800a1f90 0x7f3f800a4450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3f88004af0 tx=0x7f3f88005c50 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:08:58.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.269+0000 7f3f967fc640 1 -- 172.21.15.71:0/1334373841 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3f9001f020 con 0x7f3f98105470 2024-03-04T09:08:58.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:58 smithi071 ceph-mon[32789]: pgmap v310: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T09:08:58.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.462+0000 7f3f9f7ac640 1 -- 172.21.15.71:0/1334373841 --> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f3f981089d0 con 0x7f3f800a1f90 2024-03-04T09:08:58.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.471+0000 7f3f967fc640 1 -- 172.21.15.71:0/1334373841 <== mgr.15399 v2:172.21.15.71:6800/217968316 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f3f981089d0 con 0x7f3f800a1f90 2024-03-04T09:08:58.468 INFO:teuthology.orchestra.run.smithi071.stdout:/ 1m 6m3h 2024-03-04T09:08:58.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.473+0000 7f3f9f7ac640 1 -- 172.21.15.71:0/1334373841 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f3f800a1f90 msgr2=0x7f3f800a4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:58.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.473+0000 7f3f9f7ac640 1 --2- 172.21.15.71:0/1334373841 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f3f800a1f90 0x7f3f800a4450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3f88004af0 tx=0x7f3f88005c50 comp rx=0 tx=0).stop 2024-03-04T09:08:58.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.473+0000 7f3f9f7ac640 1 -- 172.21.15.71:0/1334373841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f98105470 msgr2=0x7f3f9819c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:58.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.473+0000 7f3f9f7ac640 1 --2- 172.21.15.71:0/1334373841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f98105470 0x7f3f9819c790 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000b3e0 tx=0x7f3f90009f90 comp rx=0 tx=0).stop 2024-03-04T09:08:58.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.473+0000 7f3f9f7ac640 1 -- 172.21.15.71:0/1334373841 shutdown_connections 2024-03-04T09:08:58.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.473+0000 7f3f9f7ac640 1 --2- 172.21.15.71:0/1334373841 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f3f800a1f90 0x7f3f800a4450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.473+0000 7f3f9f7ac640 1 --2- 172.21.15.71:0/1334373841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f98105470 0x7f3f9819c790 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.473+0000 7f3f9f7ac640 1 --2- 172.21.15.71:0/1334373841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f981010e0 0x7f3f98198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.473+0000 7f3f9f7ac640 1 --2- 172.21.15.71:0/1334373841 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f3f98100720 0x7f3f98197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.473+0000 7f3f9f7ac640 1 -- 172.21.15.71:0/1334373841 >> 172.21.15.71:0/1334373841 conn(0x7f3f98077320 msgr2=0x7f3f980fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:08:58.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.473+0000 7f3f9f7ac640 1 -- 172.21.15.71:0/1334373841 shutdown_connections 2024-03-04T09:08:58.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.473+0000 7f3f9f7ac640 1 -- 172.21.15.71:0/1334373841 wait complete. 2024-03-04T09:08:58.480 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-04T09:08:58.491 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T09:08:58.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:58 smithi150 ceph-mon[36807]: pgmap v310: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T09:08:58.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:58 smithi158 ceph-mon[37241]: pgmap v310: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T09:08:58.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.597+0000 7f8d9a050640 1 -- 172.21.15.71:0/3484133387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d94107640 msgr2=0x7f8d94107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:58.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.597+0000 7f8d9a050640 1 --2- 172.21.15.71:0/3484133387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d94107640 0x7f8d94107a20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009900 tx=0x7f8d8402bc30 comp rx=0 tx=0).stop 2024-03-04T09:08:58.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.597+0000 7f8d9a050640 1 -- 172.21.15.71:0/3484133387 shutdown_connections 2024-03-04T09:08:58.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.597+0000 7f8d9a050640 1 --2- 172.21.15.71:0/3484133387 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d94101f40 0x7f8d9410e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.597+0000 7f8d9a050640 1 --2- 172.21.15.71:0/3484133387 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d94101580 0x7f8d94101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.597+0000 7f8d9a050640 1 --2- 172.21.15.71:0/3484133387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d94107640 0x7f8d94107a20 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.597+0000 7f8d9a050640 1 -- 172.21.15.71:0/3484133387 >> 172.21.15.71:0/3484133387 conn(0x7f8d940fd070 msgr2=0x7f8d940ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:08:58.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.597+0000 7f8d9a050640 1 -- 172.21.15.71:0/3484133387 shutdown_connections 2024-03-04T09:08:58.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.597+0000 7f8d9a050640 1 -- 172.21.15.71:0/3484133387 wait complete. 2024-03-04T09:08:58.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.598+0000 7f8d9a050640 1 Processor -- start 2024-03-04T09:08:58.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.598+0000 7f8d9a050640 1 -- start start 2024-03-04T09:08:58.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.598+0000 7f8d9a050640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d94101580 0x7f8d94197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:58.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.598+0000 7f8d9a050640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d94101f40 0x7f8d94198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:58.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.598+0000 7f8d9a050640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d94107640 0x7f8d9419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:58.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.598+0000 7f8d9a050640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d94111900 con 0x7f8d94101f40 2024-03-04T09:08:58.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.598+0000 7f8d9a050640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d94111780 con 0x7f8d94101580 2024-03-04T09:08:58.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.598+0000 7f8d9a050640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d94111a80 con 0x7f8d94107640 2024-03-04T09:08:58.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.598+0000 7f8d92ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d94101f40 0x7f8d94198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:58.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.598+0000 7f8d92ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d94101f40 0x7f8d94198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57190/0 (socket says 172.21.15.71:57190) 2024-03-04T09:08:58.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.598+0000 7f8d92ffd640 1 -- 172.21.15.71:0/928112593 learned_addr learned my addr 172.21.15.71:0/928112593 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T09:08:58.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.598+0000 7f8d937fe640 1 --2- 172.21.15.71:0/928112593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d94101580 0x7f8d94197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:58.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.598+0000 7f8d92ffd640 1 -- 172.21.15.71:0/928112593 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d94107640 msgr2=0x7f8d9419c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:58.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.598+0000 7f8d92ffd640 1 --2- 172.21.15.71:0/928112593 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d94107640 0x7f8d9419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.598+0000 7f8d92ffd640 1 -- 172.21.15.71:0/928112593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d94101580 msgr2=0x7f8d94197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:58.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.598+0000 7f8d92ffd640 1 --2- 172.21.15.71:0/928112593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d94101580 0x7f8d94197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.598+0000 7f8d92ffd640 1 -- 172.21.15.71:0/928112593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9419cdd0 con 0x7f8d94101f40 2024-03-04T09:08:58.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.598+0000 7f8d92ffd640 1 --2- 172.21.15.71:0/928112593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d94101f40 0x7f8d94198320 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8d8800d950 tx=0x7f8d8800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:08:58.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.599+0000 7f8d90ff9640 1 -- 172.21.15.71:0/928112593 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88017070 con 0x7f8d94101f40 2024-03-04T09:08:58.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.599+0000 7f8d90ff9640 1 -- 172.21.15.71:0/928112593 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d88009c30 con 0x7f8d94101f40 2024-03-04T09:08:58.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.599+0000 7f8d90ff9640 1 -- 172.21.15.71:0/928112593 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8800f6b0 con 0x7f8d94101f40 2024-03-04T09:08:58.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.599+0000 7f8d9a050640 1 -- 172.21.15.71:0/928112593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9419d0c0 con 0x7f8d94101f40 2024-03-04T09:08:58.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.599+0000 7f8d9a050640 1 -- 172.21.15.71:0/928112593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d941a4950 con 0x7f8d94101f40 2024-03-04T09:08:58.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.600+0000 7f8d9a050640 1 -- 172.21.15.71:0/928112593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d94068810 con 0x7f8d94101f40 2024-03-04T09:08:58.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.601+0000 7f8d90ff9640 1 -- 172.21.15.71:0/928112593 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f8d880109d0 con 0x7f8d94101f40 2024-03-04T09:08:58.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.601+0000 7f8d90ff9640 1 --2- 172.21.15.71:0/928112593 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f8d7c0a1ef0 0x7f8d7c0a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:58.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.601+0000 7f8d937fe640 1 --2- 172.21.15.71:0/928112593 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f8d7c0a1ef0 0x7f8d7c0a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:58.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.602+0000 7f8d90ff9640 1 -- 172.21.15.71:0/928112593 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9467+0+0 (secure 0 0 0) 0x7f8d880ce140 con 0x7f8d94101f40 2024-03-04T09:08:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.602+0000 7f8d937fe640 1 --2- 172.21.15.71:0/928112593 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f8d7c0a1ef0 0x7f8d7c0a43b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009230 tx=0x7f8d84005fd0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:08:58.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.605+0000 7f8d90ff9640 1 -- 172.21.15.71:0/928112593 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8d88009de0 con 0x7f8d94101f40 2024-03-04T09:08:58.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.781+0000 7f8d9a050640 1 -- 172.21.15.71:0/928112593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d9402cff0 con 0x7f8d94101f40 2024-03-04T09:08:58.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.782+0000 7f8d90ff9640 1 -- 172.21.15.71:0/928112593 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+25291 (secure 0 0 0) 0x7f8d8809ae80 con 0x7f8d94101f40 2024-03-04T09:08:58.779 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T09:08:58.779 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":102,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","created":"2024-03-04T08:47:28.546102+0000","modified":"2024-03-04T09:08:14.689181+0000","last_up_change":"2024-03-04T08:55:30.744961+0000","last_in_change":"2024-03-04T08:55:11.570990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T08:51:14.933487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T08:56:28.253074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T08:56:29.543622+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":14,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T08:56:32.884220+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":14,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7e43b47d-8888-4631-bc96-8a96fcf8cd46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6803","nonce":1173641530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6805","nonce":1173641530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6809","nonce":1173641530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6807","nonce":1173641530}]},"public_addr":"172.21.15.71:6803/1173641530","cluster_addr":"172.21.15.71:6805/1173641530","heartbeat_back_addr":"172.21.15.71:6809/1173641530","heartbeat_front_addr":"172.21.15.71:6807/1173641530","state":["exists","up"]},{"osd":1,"uuid":"1209f214-f2e2-4209-bf69-4528a43a2123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6801","nonce":3383948110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6803","nonce":3383948110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6807","nonce":3383948110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6805","nonce":3383948110}]},"public_addr":"172.21.15.150:6801/3383948110","cluster_addr":"172.21.15.150:6803/3383948110","heartbeat_back_addr":"172.21.15.150:6807/3383948110","heartbeat_front_addr":"172.21.15.150:6805/3383948110","state":["exists","up"]},{"osd":2,"uuid":"b88c3082-93a9-4c9b-9221-a7105a44c91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6801","nonce":586944144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6803","nonce":586944144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6807","nonce":586944144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6805","nonce":586944144}]},"public_addr":"172.21.15.158:6801/586944144","cluster_addr":"172.21.15.158:6803/586944144","heartbeat_back_addr":"172.21.15.158:6807/586944144","heartbeat_front_addr":"172.21.15.158:6805/586944144","state":["exists","up"]},{"osd":3,"uuid":"e4fc7723-fd73-40b2-b9d0-7f9bda282ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6811","nonce":1311903262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6813","nonce":1311903262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6817","nonce":1311903262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6815","nonce":1311903262}]},"public_addr":"172.21.15.71:6811/1311903262","cluster_addr":"172.21.15.71:6813/1311903262","heartbeat_back_addr":"172.21.15.71:6817/1311903262","heartbeat_front_addr":"172.21.15.71:6815/1311903262","state":["exists","up"]},{"osd":4,"uuid":"e7eb72ac-f066-4ccd-b2bf-22a6a26daf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6809","nonce":2300837573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6811","nonce":2300837573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6815","nonce":2300837573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6813","nonce":2300837573}]},"public_addr":"172.21.15.150:6809/2300837573","cluster_addr":"172.21.15.150:6811/2300837573","heartbeat_back_addr":"172.21.15.150:6815/2300837573","heartbeat_front_addr":"172.21.15.150:6813/2300837573","state":["exists","up"]},{"osd":5,"uuid":"9b295840-e104-4a66-9cc4-7020627c1765","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6809","nonce":1922694045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6811","nonce":1922694045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6815","nonce":1922694045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6813","nonce":1922694045}]},"public_addr":"172.21.15.158:6809/1922694045","cluster_addr":"172.21.15.158:6811/1922694045","heartbeat_back_addr":"172.21.15.158:6815/1922694045","heartbeat_front_addr":"172.21.15.158:6813/1922694045","state":["exists","up"]},{"osd":6,"uuid":"6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6819","nonce":3848842707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6821","nonce":3848842707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6825","nonce":3848842707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6823","nonce":3848842707}]},"public_addr":"172.21.15.71:6819/3848842707","cluster_addr":"172.21.15.71:6821/3848842707","heartbeat_back_addr":"172.21.15.71:6825/3848842707","heartbeat_front_addr":"172.21.15.71:6823/3848842707","state":["exists","up"]},{"osd":7,"uuid":"6160a40a-d30f-4569-949c-0afb1bb3d133","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6817","nonce":1328330305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6819","nonce":1328330305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6823","nonce":1328330305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6821","nonce":1328330305}]},"public_addr":"172.21.15.150:6817/1328330305","cluster_addr":"172.21.15.150:6819/1328330305","heartbeat_back_addr":"172.21.15.150:6823/1328330305","heartbeat_front_addr":"172.21.15.150:6821/1328330305","state":["exists","up"]},{"osd":8,"uuid":"95ed2341-84a4-465c-9948-ac667680ce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6817","nonce":2195304577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6819","nonce":2195304577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6823","nonce":2195304577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6821","nonce":2195304577}]},"public_addr":"172.21.15.158:6817/2195304577","cluster_addr":"172.21.15.158:6819/2195304577","heartbeat_back_addr":"172.21.15.158:6823/2195304577","heartbeat_front_addr":"172.21.15.158:6821/2195304577","state":["exists","up"]},{"osd":9,"uuid":"088f6419-63a0-4768-afae-dbb8a5f5a565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6827","nonce":1913779347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6829","nonce":1913779347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6833","nonce":1913779347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6831","nonce":1913779347}]},"public_addr":"172.21.15.71:6827/1913779347","cluster_addr":"172.21.15.71:6829/1913779347","heartbeat_back_addr":"172.21.15.71:6833/1913779347","heartbeat_front_addr":"172.21.15.71:6831/1913779347","state":["exists","up"]},{"osd":10,"uuid":"18c85e24-068d-4f1e-a3b8-f7915b363234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6825","nonce":3295420996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6827","nonce":3295420996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6831","nonce":3295420996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6829","nonce":3295420996}]},"public_addr":"172.21.15.150:6825/3295420996","cluster_addr":"172.21.15.150:6827/3295420996","heartbeat_back_addr":"172.21.15.150:6831/3295420996","heartbeat_front_addr":"172.21.15.150:6829/3295420996","state":["exists","up"]},{"osd":11,"uuid":"4f6b3ddf-c482-4740-b1fd-de836e047c2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6825","nonce":3332083865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6827","nonce":3332083865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6831","nonce":3332083865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6829","nonce":3332083865}]},"public_addr":"172.21.15.158:6825/3332083865","cluster_addr":"172.21.15.158:6827/3332083865","heartbeat_back_addr":"172.21.15.158:6831/3332083865","heartbeat_front_addr":"172.21.15.158:6829/3332083865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:05.997832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:29.314235+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:08.819465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:41.194275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:06.032479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:38.814498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:13.067960+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:38.568479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:03.416713+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:38.349479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:04.096945+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:29.425250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/2792362717":"2024-03-05T08:58:54.820040+0000","172.21.15.71:0/1289352375":"2024-03-05T08:58:54.820040+0000","172.21.15.71:6801/2901868399":"2024-03-05T08:58:54.820040+0000","172.21.15.71:6800/2901868399":"2024-03-05T08:58:54.820040+0000","172.21.15.71:0/1087047482":"2024-03-05T08:58:54.820040+0000","172.21.15.71:0/4186644246":"2024-03-05T08:58:54.820040+0000","172.21.15.71:0/1287712358":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/3770452860":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/718743094":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/1904717532":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:6800/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6800/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/2862184313":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/3621754630":"2024-03-05T08:48:22.593931+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":14,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":14,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:08:58.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.784+0000 7f8d9a050640 1 -- 172.21.15.71:0/928112593 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f8d7c0a1ef0 msgr2=0x7f8d7c0a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:58.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.784+0000 7f8d9a050640 1 --2- 172.21.15.71:0/928112593 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f8d7c0a1ef0 0x7f8d7c0a43b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009230 tx=0x7f8d84005fd0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.784+0000 7f8d9a050640 1 -- 172.21.15.71:0/928112593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d94101f40 msgr2=0x7f8d94198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:58.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.784+0000 7f8d9a050640 1 --2- 172.21.15.71:0/928112593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d94101f40 0x7f8d94198320 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8d8800d950 tx=0x7f8d8800de10 comp rx=0 tx=0).stop 2024-03-04T09:08:58.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.784+0000 7f8d9a050640 1 -- 172.21.15.71:0/928112593 shutdown_connections 2024-03-04T09:08:58.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.785+0000 7f8d9a050640 1 --2- 172.21.15.71:0/928112593 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f8d94107640 0x7f8d9419c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.785+0000 7f8d9a050640 1 --2- 172.21.15.71:0/928112593 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f8d7c0a1ef0 0x7f8d7c0a43b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.785+0000 7f8d9a050640 1 --2- 172.21.15.71:0/928112593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d94101f40 0x7f8d94198320 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.785+0000 7f8d9a050640 1 --2- 172.21.15.71:0/928112593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d94101580 0x7f8d94197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.785+0000 7f8d9a050640 1 -- 172.21.15.71:0/928112593 >> 172.21.15.71:0/928112593 conn(0x7f8d940fd070 msgr2=0x7f8d940fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:08:58.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.785+0000 7f8d9a050640 1 -- 172.21.15.71:0/928112593 shutdown_connections 2024-03-04T09:08:58.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.785+0000 7f8d9a050640 1 -- 172.21.15.71:0/928112593 wait complete. 2024-03-04T09:08:58.792 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T08:51:14.933487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T08:56:28.253074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T08:56:29.543622+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 14, 'snap_epoch': 101, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T08:56:32.884220+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 14, 'snap_epoch': 101, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T09:08:58.793 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T09:08:58.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.901+0000 7f5feb9f7640 1 -- 172.21.15.71:0/471015902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fe41064e0 msgr2=0x7f5fe41068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:58.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.901+0000 7f5feb9f7640 1 --2- 172.21.15.71:0/471015902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fe41064e0 0x7f5fe41068c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009cd0 tx=0x7f5fe002c0f0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.901+0000 7f5feb9f7640 1 -- 172.21.15.71:0/471015902 shutdown_connections 2024-03-04T09:08:58.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.901+0000 7f5feb9f7640 1 --2- 172.21.15.71:0/471015902 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fe4100e00 0x7f5fe410d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.901+0000 7f5feb9f7640 1 --2- 172.21.15.71:0/471015902 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5fe4100440 0x7f5fe41008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.901+0000 7f5feb9f7640 1 --2- 172.21.15.71:0/471015902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fe41064e0 0x7f5fe41068c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.901+0000 7f5feb9f7640 1 -- 172.21.15.71:0/471015902 >> 172.21.15.71:0/471015902 conn(0x7f5fe40775d0 msgr2=0x7f5fe40779e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:08:58.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.901+0000 7f5feb9f7640 1 -- 172.21.15.71:0/471015902 shutdown_connections 2024-03-04T09:08:58.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.901+0000 7f5feb9f7640 1 -- 172.21.15.71:0/471015902 wait complete. 2024-03-04T09:08:58.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.902+0000 7f5feb9f7640 1 Processor -- start 2024-03-04T09:08:58.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.902+0000 7f5feb9f7640 1 -- start start 2024-03-04T09:08:58.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.902+0000 7f5feb9f7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fe4100440 0x7f5fe4198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:58.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.902+0000 7f5feb9f7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fe4100e00 0x7f5fe4199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:58.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.902+0000 7f5feb9f7640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5fe41064e0 0x7f5fe41a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:58.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.902+0000 7f5feb9f7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe4110800 con 0x7f5fe4100e00 2024-03-04T09:08:58.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.902+0000 7f5feb9f7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe4110680 con 0x7f5fe4100440 2024-03-04T09:08:58.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.902+0000 7f5feb9f7640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe4110980 con 0x7f5fe41064e0 2024-03-04T09:08:58.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.902+0000 7f5fe8f92640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fe4100e00 0x7f5fe4199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:58.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.902+0000 7f5fe8f92640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fe4100e00 0x7f5fe4199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57218/0 (socket says 172.21.15.71:57218) 2024-03-04T09:08:58.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.902+0000 7f5fe8f92640 1 -- 172.21.15.71:0/3627395402 learned_addr learned my addr 172.21.15.71:0/3627395402 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T09:08:58.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.902+0000 7f5fe9793640 1 --2- 172.21.15.71:0/3627395402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fe4100440 0x7f5fe4198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:58.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.903+0000 7f5fe8f92640 1 -- 172.21.15.71:0/3627395402 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5fe41064e0 msgr2=0x7f5fe41a3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:58.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.903+0000 7f5fe8f92640 1 --2- 172.21.15.71:0/3627395402 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5fe41064e0 0x7f5fe41a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.903+0000 7f5fe8f92640 1 -- 172.21.15.71:0/3627395402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fe4100440 msgr2=0x7f5fe4198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:58.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.903+0000 7f5fe8f92640 1 --2- 172.21.15.71:0/3627395402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fe4100440 0x7f5fe4198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:58.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.903+0000 7f5fe8f92640 1 -- 172.21.15.71:0/3627395402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe41a4280 con 0x7f5fe4100e00 2024-03-04T09:08:58.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.903+0000 7f5fe8f92640 1 --2- 172.21.15.71:0/3627395402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fe4100e00 0x7f5fe4199200 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400d770 tx=0x7f5fd400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:08:58.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.903+0000 7f5fda7fc640 1 -- 172.21.15.71:0/3627395402 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4017070 con 0x7f5fe4100e00 2024-03-04T09:08:58.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.903+0000 7f5fda7fc640 1 -- 172.21.15.71:0/3627395402 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fd4005500 con 0x7f5fe4100e00 2024-03-04T09:08:58.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.903+0000 7f5feb9f7640 1 -- 172.21.15.71:0/3627395402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe41a4450 con 0x7f5fe4100e00 2024-03-04T09:08:58.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.903+0000 7f5fda7fc640 1 -- 172.21.15.71:0/3627395402 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4004e00 con 0x7f5fe4100e00 2024-03-04T09:08:58.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.903+0000 7f5feb9f7640 1 -- 172.21.15.71:0/3627395402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe41a4950 con 0x7f5fe4100e00 2024-03-04T09:08:58.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.904+0000 7f5feb9f7640 1 -- 172.21.15.71:0/3627395402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe40683c0 con 0x7f5fe4100e00 2024-03-04T09:08:58.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.905+0000 7f5fda7fc640 1 -- 172.21.15.71:0/3627395402 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f5fd4004fa0 con 0x7f5fe4100e00 2024-03-04T09:08:58.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.906+0000 7f5fda7fc640 1 --2- 172.21.15.71:0/3627395402 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f5fd00a1ef0 0x7f5fd00a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:58.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.906+0000 7f5fe9793640 1 --2- 172.21.15.71:0/3627395402 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f5fd00a1ef0 0x7f5fd00a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:58.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.906+0000 7f5fda7fc640 1 -- 172.21.15.71:0/3627395402 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9467+0+0 (secure 0 0 0) 0x7f5fd40cdfc0 con 0x7f5fe4100e00 2024-03-04T09:08:58.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.906+0000 7f5fe9793640 1 --2- 172.21.15.71:0/3627395402 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f5fd00a1ef0 0x7f5fd00a43b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0004af0 tx=0x7f5fe0005c50 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:08:58.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:58.910+0000 7f5fda7fc640 1 -- 172.21.15.71:0/3627395402 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5fd409ad00 con 0x7f5fe4100e00 2024-03-04T09:08:59.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.085+0000 7f5feb9f7640 1 -- 172.21.15.71:0/3627395402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5fe41089e0 con 0x7f5fe4100e00 2024-03-04T09:08:59.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.086+0000 7f5fda7fc640 1 -- 172.21.15.71:0/3627395402 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5fd409a600 con 0x7f5fe4100e00 2024-03-04T09:08:59.082 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-03-04T09:08:59.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.088+0000 7f5feb9f7640 1 -- 172.21.15.71:0/3627395402 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f5fd00a1ef0 msgr2=0x7f5fd00a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:59.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.088+0000 7f5feb9f7640 1 --2- 172.21.15.71:0/3627395402 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f5fd00a1ef0 0x7f5fd00a43b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0004af0 tx=0x7f5fe0005c50 comp rx=0 tx=0).stop 2024-03-04T09:08:59.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.088+0000 7f5feb9f7640 1 -- 172.21.15.71:0/3627395402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fe4100e00 msgr2=0x7f5fe4199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:59.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.088+0000 7f5feb9f7640 1 --2- 172.21.15.71:0/3627395402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fe4100e00 0x7f5fe4199200 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400d770 tx=0x7f5fd400dc30 comp rx=0 tx=0).stop 2024-03-04T09:08:59.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.088+0000 7f5feb9f7640 1 -- 172.21.15.71:0/3627395402 shutdown_connections 2024-03-04T09:08:59.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.088+0000 7f5feb9f7640 1 --2- 172.21.15.71:0/3627395402 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f5fe41064e0 0x7f5fe41a3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.088+0000 7f5feb9f7640 1 --2- 172.21.15.71:0/3627395402 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f5fd00a1ef0 0x7f5fd00a43b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.088+0000 7f5feb9f7640 1 --2- 172.21.15.71:0/3627395402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fe4100e00 0x7f5fe4199200 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.088+0000 7f5feb9f7640 1 --2- 172.21.15.71:0/3627395402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5fe4100440 0x7f5fe4198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.088+0000 7f5feb9f7640 1 -- 172.21.15.71:0/3627395402 >> 172.21.15.71:0/3627395402 conn(0x7f5fe40775d0 msgr2=0x7f5fe4102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:08:59.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.088+0000 7f5feb9f7640 1 -- 172.21.15.71:0/3627395402 shutdown_connections 2024-03-04T09:08:59.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.088+0000 7f5feb9f7640 1 -- 172.21.15.71:0/3627395402 wait complete. 2024-03-04T09:08:59.095 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T09:08:59.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.203+0000 7f01d9439640 1 -- 172.21.15.71:0/2484451924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01d4101580 msgr2=0x7f01d4101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:59.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.203+0000 7f01d9439640 1 --2- 172.21.15.71:0/2484451924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01d4101580 0x7f01d4101a00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f01bc009900 tx=0x7f01bc02bc30 comp rx=0 tx=0).stop 2024-03-04T09:08:59.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.203+0000 7f01d9439640 1 -- 172.21.15.71:0/2484451924 shutdown_connections 2024-03-04T09:08:59.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.203+0000 7f01d9439640 1 --2- 172.21.15.71:0/2484451924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01d4101f40 0x7f01d410e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.203+0000 7f01d9439640 1 --2- 172.21.15.71:0/2484451924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01d4101580 0x7f01d4101a00 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.203+0000 7f01d9439640 1 --2- 172.21.15.71:0/2484451924 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f01d4107640 0x7f01d4107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.203+0000 7f01d9439640 1 -- 172.21.15.71:0/2484451924 >> 172.21.15.71:0/2484451924 conn(0x7f01d40789d0 msgr2=0x7f01d4078de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:08:59.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.203+0000 7f01d9439640 1 -- 172.21.15.71:0/2484451924 shutdown_connections 2024-03-04T09:08:59.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.203+0000 7f01d9439640 1 -- 172.21.15.71:0/2484451924 wait complete. 2024-03-04T09:08:59.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.203+0000 7f01d9439640 1 Processor -- start 2024-03-04T09:08:59.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.203+0000 7f01d9439640 1 -- start start 2024-03-04T09:08:59.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.204+0000 7f01d9439640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01d4101580 0x7f01d40738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:59.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.204+0000 7f01d9439640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f01d4101f40 0x7f01d4073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:59.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.204+0000 7f01d9439640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01d4107640 0x7f01d4074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:59.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.204+0000 7f01d9439640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d4111980 con 0x7f01d4101580 2024-03-04T09:08:59.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.204+0000 7f01d9439640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d4111800 con 0x7f01d4107640 2024-03-04T09:08:59.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.204+0000 7f01d9439640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d4111b00 con 0x7f01d4101f40 2024-03-04T09:08:59.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.204+0000 7f01d2ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01d4101580 0x7f01d40738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:59.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.204+0000 7f01d2ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01d4101580 0x7f01d40738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57242/0 (socket says 172.21.15.71:57242) 2024-03-04T09:08:59.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.204+0000 7f01d2ffd640 1 -- 172.21.15.71:0/382274372 learned_addr learned my addr 172.21.15.71:0/382274372 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T09:08:59.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.204+0000 7f01d37fe640 1 --2- 172.21.15.71:0/382274372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01d4107640 0x7f01d4074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:59.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.204+0000 7f01ca7fc640 1 --2- 172.21.15.71:0/382274372 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f01d4101f40 0x7f01d4073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:59.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.204+0000 7f01d2ffd640 1 -- 172.21.15.71:0/382274372 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f01d4101f40 msgr2=0x7f01d4073e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:59.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.204+0000 7f01d2ffd640 1 --2- 172.21.15.71:0/382274372 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f01d4101f40 0x7f01d4073e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.204+0000 7f01d2ffd640 1 -- 172.21.15.71:0/382274372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01d4107640 msgr2=0x7f01d4074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:59.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.204+0000 7f01d2ffd640 1 --2- 172.21.15.71:0/382274372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01d4107640 0x7f01d4074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.204+0000 7f01d2ffd640 1 -- 172.21.15.71:0/382274372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d41a6340 con 0x7f01d4101580 2024-03-04T09:08:59.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.204+0000 7f01d2ffd640 1 --2- 172.21.15.71:0/382274372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01d4101580 0x7f01d40738d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f01c400b4a0 tx=0x7f01c400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:08:59.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.204+0000 7f01d0ff9640 1 -- 172.21.15.71:0/382274372 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c4014070 con 0x7f01d4101580 2024-03-04T09:08:59.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.204+0000 7f01d0ff9640 1 -- 172.21.15.71:0/382274372 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f01c4010af0 con 0x7f01d4101580 2024-03-04T09:08:59.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.204+0000 7f01d0ff9640 1 -- 172.21.15.71:0/382274372 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c4017d10 con 0x7f01d4101580 2024-03-04T09:08:59.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.204+0000 7f01d9439640 1 -- 172.21.15.71:0/382274372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d41a6630 con 0x7f01d4101580 2024-03-04T09:08:59.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.204+0000 7f01d9439640 1 -- 172.21.15.71:0/382274372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d41a6b80 con 0x7f01d4101580 2024-03-04T09:08:59.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.205+0000 7f01d9439640 1 -- 172.21.15.71:0/382274372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d4068810 con 0x7f01d4101580 2024-03-04T09:08:59.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.206+0000 7f01d0ff9640 1 -- 172.21.15.71:0/382274372 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f01c4017070 con 0x7f01d4101580 2024-03-04T09:08:59.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.207+0000 7f01d0ff9640 1 --2- 172.21.15.71:0/382274372 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f01c00a1ef0 0x7f01c00a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:59.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.207+0000 7f01ca7fc640 1 --2- 172.21.15.71:0/382274372 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f01c00a1ef0 0x7f01c00a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:59.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.207+0000 7f01d0ff9640 1 -- 172.21.15.71:0/382274372 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9467+0+0 (secure 0 0 0) 0x7f01c40cdc10 con 0x7f01d4101580 2024-03-04T09:08:59.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.207+0000 7f01ca7fc640 1 --2- 172.21.15.71:0/382274372 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f01c00a1ef0 0x7f01c00a43b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f01d4075360 tx=0x7f01bc02c750 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:08:59.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.211+0000 7f01d0ff9640 1 -- 172.21.15.71:0/382274372 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f01c409aa60 con 0x7f01d4101580 2024-03-04T09:08:59.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.385+0000 7f01d9439640 1 -- 172.21.15.71:0/382274372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f01d4109a60 con 0x7f01d4101580 2024-03-04T09:08:59.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.386+0000 7f01d0ff9640 1 -- 172.21.15.71:0/382274372 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) v1 ==== 104+0+11 (secure 0 0 0) 0x7f01c409a360 con 0x7f01d4101580 2024-03-04T09:08:59.382 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-03-04T09:08:59.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.387+0000 7f01d9439640 1 -- 172.21.15.71:0/382274372 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f01c00a1ef0 msgr2=0x7f01c00a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:59.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.387+0000 7f01d9439640 1 --2- 172.21.15.71:0/382274372 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f01c00a1ef0 0x7f01c00a43b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f01d4075360 tx=0x7f01bc02c750 comp rx=0 tx=0).stop 2024-03-04T09:08:59.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.387+0000 7f01d9439640 1 -- 172.21.15.71:0/382274372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01d4101580 msgr2=0x7f01d40738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:59.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.387+0000 7f01d9439640 1 --2- 172.21.15.71:0/382274372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01d4101580 0x7f01d40738d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f01c400b4a0 tx=0x7f01c400b960 comp rx=0 tx=0).stop 2024-03-04T09:08:59.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.387+0000 7f01d9439640 1 -- 172.21.15.71:0/382274372 shutdown_connections 2024-03-04T09:08:59.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.387+0000 7f01d9439640 1 --2- 172.21.15.71:0/382274372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01d4107640 0x7f01d4074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.387+0000 7f01d9439640 1 --2- 172.21.15.71:0/382274372 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f01d4101f40 0x7f01d4073e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.387+0000 7f01d9439640 1 --2- 172.21.15.71:0/382274372 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f01c00a1ef0 0x7f01c00a43b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.387+0000 7f01d9439640 1 --2- 172.21.15.71:0/382274372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01d4101580 0x7f01d40738d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.387+0000 7f01d9439640 1 -- 172.21.15.71:0/382274372 >> 172.21.15.71:0/382274372 conn(0x7f01d40789d0 msgr2=0x7f01d4100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:08:59.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.388+0000 7f01d9439640 1 -- 172.21.15.71:0/382274372 shutdown_connections 2024-03-04T09:08:59.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.388+0000 7f01d9439640 1 -- 172.21.15.71:0/382274372 wait complete. 2024-03-04T09:08:59.393 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:08:59 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/928112593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:08:59.394 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T09:08:59.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.503+0000 7fc68ff5c640 1 -- 172.21.15.71:0/1620517783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6881077d0 msgr2=0x7fc688107bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:59.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.503+0000 7fc68ff5c640 1 --2- 172.21.15.71:0/1620517783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6881077d0 0x7fc688107bb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc684009930 tx=0x7fc68402bcd0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.503+0000 7fc68ff5c640 1 -- 172.21.15.71:0/1620517783 shutdown_connections 2024-03-04T09:08:59.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.503+0000 7fc68ff5c640 1 --2- 172.21.15.71:0/1620517783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc688100850 0x7fc68810e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.503+0000 7fc68ff5c640 1 --2- 172.21.15.71:0/1620517783 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6880ffe90 0x7fc688100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.503+0000 7fc68ff5c640 1 --2- 172.21.15.71:0/1620517783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6881077d0 0x7fc688107bb0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.503+0000 7fc68ff5c640 1 -- 172.21.15.71:0/1620517783 >> 172.21.15.71:0/1620517783 conn(0x7fc688075810 msgr2=0x7fc688075c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:08:59.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.503+0000 7fc68ff5c640 1 -- 172.21.15.71:0/1620517783 shutdown_connections 2024-03-04T09:08:59.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.503+0000 7fc68ff5c640 1 -- 172.21.15.71:0/1620517783 wait complete. 2024-03-04T09:08:59.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.503+0000 7fc68ff5c640 1 Processor -- start 2024-03-04T09:08:59.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.503+0000 7fc68ff5c640 1 -- start start 2024-03-04T09:08:59.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.504+0000 7fc68ff5c640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6880ffe90 0x7fc688197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:59.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.504+0000 7fc68ff5c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc688100850 0x7fc688198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:59.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.504+0000 7fc68ff5c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6881077d0 0x7fc68819c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:59.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.504+0000 7fc68ff5c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc688111a30 con 0x7fc688100850 2024-03-04T09:08:59.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.504+0000 7fc68ff5c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6881118b0 con 0x7fc6881077d0 2024-03-04T09:08:59.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.504+0000 7fc68ff5c640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc688111bb0 con 0x7fc6880ffe90 2024-03-04T09:08:59.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.504+0000 7fc68d4f7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc688100850 0x7fc688198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:59.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.504+0000 7fc68d4f7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc688100850 0x7fc688198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57264/0 (socket says 172.21.15.71:57264) 2024-03-04T09:08:59.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.504+0000 7fc68d4f7640 1 -- 172.21.15.71:0/75774603 learned_addr learned my addr 172.21.15.71:0/75774603 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T09:08:59.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.504+0000 7fc68e4f9640 1 --2- 172.21.15.71:0/75774603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6881077d0 0x7fc68819c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:59.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.504+0000 7fc68dcf8640 1 --2- 172.21.15.71:0/75774603 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6880ffe90 0x7fc688197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:59.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.504+0000 7fc68d4f7640 1 -- 172.21.15.71:0/75774603 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6880ffe90 msgr2=0x7fc688197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:59.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.504+0000 7fc68d4f7640 1 --2- 172.21.15.71:0/75774603 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6880ffe90 0x7fc688197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.504+0000 7fc68d4f7640 1 -- 172.21.15.71:0/75774603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6881077d0 msgr2=0x7fc68819c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:59.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.504+0000 7fc68d4f7640 1 --2- 172.21.15.71:0/75774603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6881077d0 0x7fc68819c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.504+0000 7fc68d4f7640 1 -- 172.21.15.71:0/75774603 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68819cf00 con 0x7fc688100850 2024-03-04T09:08:59.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.504+0000 7fc68d4f7640 1 --2- 172.21.15.71:0/75774603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc688100850 0x7fc688198450 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc67800d770 tx=0x7fc67800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:08:59.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.504+0000 7fc67effd640 1 -- 172.21.15.71:0/75774603 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678017070 con 0x7fc688100850 2024-03-04T09:08:59.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.504+0000 7fc67effd640 1 -- 172.21.15.71:0/75774603 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc678005500 con 0x7fc688100850 2024-03-04T09:08:59.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.504+0000 7fc68ff5c640 1 -- 172.21.15.71:0/75774603 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc68819d1f0 con 0x7fc688100850 2024-03-04T09:08:59.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.504+0000 7fc67effd640 1 -- 172.21.15.71:0/75774603 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678004e00 con 0x7fc688100850 2024-03-04T09:08:59.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.504+0000 7fc68ff5c640 1 -- 172.21.15.71:0/75774603 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6881a4a80 con 0x7fc688100850 2024-03-04T09:08:59.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.505+0000 7fc68ff5c640 1 -- 172.21.15.71:0/75774603 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6880696e0 con 0x7fc688100850 2024-03-04T09:08:59.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.506+0000 7fc67effd640 1 -- 172.21.15.71:0/75774603 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7fc678004fa0 con 0x7fc688100850 2024-03-04T09:08:59.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.507+0000 7fc67effd640 1 --2- 172.21.15.71:0/75774603 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fc6740a1ef0 0x7fc6740a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:59.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.507+0000 7fc68dcf8640 1 --2- 172.21.15.71:0/75774603 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fc6740a1ef0 0x7fc6740a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:59.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.507+0000 7fc67effd640 1 -- 172.21.15.71:0/75774603 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9467+0+0 (secure 0 0 0) 0x7fc6780cdef0 con 0x7fc688100850 2024-03-04T09:08:59.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.508+0000 7fc68dcf8640 1 --2- 172.21.15.71:0/75774603 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fc6740a1ef0 0x7fc6740a43b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc684005ab0 tx=0x7fc684005fd0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:08:59.508 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.512+0000 7fc67effd640 1 -- 172.21.15.71:0/75774603 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc67809a5e0 con 0x7fc688100850 2024-03-04T09:08:59.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:08:59 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/928112593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:08:59.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:08:59 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/928112593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:08:59.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.687+0000 7fc68ff5c640 1 -- 172.21.15.71:0/75774603 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc688109af0 con 0x7fc688100850 2024-03-04T09:08:59.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.687+0000 7fc67effd640 1 -- 172.21.15.71:0/75774603 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc67809e680 con 0x7fc688100850 2024-03-04T09:08:59.684 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2024-03-04T09:08:59.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.689+0000 7fc68ff5c640 1 -- 172.21.15.71:0/75774603 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fc6740a1ef0 msgr2=0x7fc6740a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:59.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.689+0000 7fc68ff5c640 1 --2- 172.21.15.71:0/75774603 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fc6740a1ef0 0x7fc6740a43b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc684005ab0 tx=0x7fc684005fd0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.689+0000 7fc68ff5c640 1 -- 172.21.15.71:0/75774603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc688100850 msgr2=0x7fc688198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:59.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.689+0000 7fc68ff5c640 1 --2- 172.21.15.71:0/75774603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc688100850 0x7fc688198450 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc67800d770 tx=0x7fc67800dc30 comp rx=0 tx=0).stop 2024-03-04T09:08:59.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.689+0000 7fc68ff5c640 1 -- 172.21.15.71:0/75774603 shutdown_connections 2024-03-04T09:08:59.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.689+0000 7fc68ff5c640 1 --2- 172.21.15.71:0/75774603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6881077d0 0x7fc68819c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.689+0000 7fc68ff5c640 1 --2- 172.21.15.71:0/75774603 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fc6740a1ef0 0x7fc6740a43b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.689+0000 7fc68ff5c640 1 --2- 172.21.15.71:0/75774603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc688100850 0x7fc688198450 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.689+0000 7fc68ff5c640 1 --2- 172.21.15.71:0/75774603 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc6880ffe90 0x7fc688197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.689+0000 7fc68ff5c640 1 -- 172.21.15.71:0/75774603 >> 172.21.15.71:0/75774603 conn(0x7fc688075810 msgr2=0x7fc6880fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:08:59.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.689+0000 7fc68ff5c640 1 -- 172.21.15.71:0/75774603 shutdown_connections 2024-03-04T09:08:59.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.689+0000 7fc68ff5c640 1 -- 172.21.15.71:0/75774603 wait complete. 2024-03-04T09:08:59.696 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T09:08:59.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.807+0000 7f696fcdf640 1 -- 172.21.15.71:0/190821411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6968103130 msgr2=0x7f696810b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:59.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.807+0000 7f696fcdf640 1 --2- 172.21.15.71:0/190821411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6968103130 0x7f696810b610 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6958009900 tx=0x7f695802bcd0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.807+0000 7f696fcdf640 1 -- 172.21.15.71:0/190821411 shutdown_connections 2024-03-04T09:08:59.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.807+0000 7f696fcdf640 1 --2- 172.21.15.71:0/190821411 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f696810bbe0 0x7f696810e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.807+0000 7f696fcdf640 1 --2- 172.21.15.71:0/190821411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6968103130 0x7f696810b610 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.807+0000 7f696fcdf640 1 --2- 172.21.15.71:0/190821411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6968102810 0x7f6968102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.807+0000 7f696fcdf640 1 -- 172.21.15.71:0/190821411 >> 172.21.15.71:0/190821411 conn(0x7f6968076850 msgr2=0x7f6968076c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:08:59.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.807+0000 7f696fcdf640 1 -- 172.21.15.71:0/190821411 shutdown_connections 2024-03-04T09:08:59.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.807+0000 7f696fcdf640 1 -- 172.21.15.71:0/190821411 wait complete. 2024-03-04T09:08:59.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.807+0000 7f696fcdf640 1 Processor -- start 2024-03-04T09:08:59.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.807+0000 7f696fcdf640 1 -- start start 2024-03-04T09:08:59.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.807+0000 7f696fcdf640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6968102810 0x7f6968100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:59.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.807+0000 7f696fcdf640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6968103130 0x7f69681009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:59.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.807+0000 7f696fcdf640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f696810bbe0 0x7f6968100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:59.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.807+0000 7f696fcdf640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69681119d0 con 0x7f696810bbe0 2024-03-04T09:08:59.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.807+0000 7f696fcdf640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6968111850 con 0x7f6968103130 2024-03-04T09:08:59.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.807+0000 7f696fcdf640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6968111b50 con 0x7f6968102810 2024-03-04T09:08:59.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.808+0000 7f696e27c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f696810bbe0 0x7f6968100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:59.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.808+0000 7f696e27c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f696810bbe0 0x7f6968100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57282/0 (socket says 172.21.15.71:57282) 2024-03-04T09:08:59.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.808+0000 7f696e27c640 1 -- 172.21.15.71:0/4262240804 learned_addr learned my addr 172.21.15.71:0/4262240804 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T09:08:59.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.808+0000 7f696d27a640 1 --2- 172.21.15.71:0/4262240804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6968103130 0x7f69681009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:59.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.808+0000 7f696da7b640 1 --2- 172.21.15.71:0/4262240804 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6968102810 0x7f6968100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:59.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.808+0000 7f696e27c640 1 -- 172.21.15.71:0/4262240804 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6968102810 msgr2=0x7f6968100460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:59.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.808+0000 7f696e27c640 1 --2- 172.21.15.71:0/4262240804 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6968102810 0x7f6968100460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.808+0000 7f696e27c640 1 -- 172.21.15.71:0/4262240804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6968103130 msgr2=0x7f69681009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:59.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.808+0000 7f696e27c640 1 --2- 172.21.15.71:0/4262240804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6968103130 0x7f69681009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.808+0000 7f696e27c640 1 -- 172.21.15.71:0/4262240804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69681a63f0 con 0x7f696810bbe0 2024-03-04T09:08:59.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.808+0000 7f696da7b640 1 --2- 172.21.15.71:0/4262240804 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6968102810 0x7f6968100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:08:59.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.808+0000 7f696e27c640 1 --2- 172.21.15.71:0/4262240804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f696810bbe0 0x7f6968100ee0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f696000b9e0 tx=0x7f696000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:08:59.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.808+0000 7f695effd640 1 -- 172.21.15.71:0/4262240804 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6960019070 con 0x7f696810bbe0 2024-03-04T09:08:59.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.808+0000 7f695effd640 1 -- 172.21.15.71:0/4262240804 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6960014440 con 0x7f696810bbe0 2024-03-04T09:08:59.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.808+0000 7f696fcdf640 1 -- 172.21.15.71:0/4262240804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69681a66e0 con 0x7f696810bbe0 2024-03-04T09:08:59.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.808+0000 7f695effd640 1 -- 172.21.15.71:0/4262240804 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69600125e0 con 0x7f696810bbe0 2024-03-04T09:08:59.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.808+0000 7f696fcdf640 1 -- 172.21.15.71:0/4262240804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69681a6c00 con 0x7f696810bbe0 2024-03-04T09:08:59.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.809+0000 7f696fcdf640 1 -- 172.21.15.71:0/4262240804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f696806a3c0 con 0x7f696810bbe0 2024-03-04T09:08:59.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.810+0000 7f695effd640 1 -- 172.21.15.71:0/4262240804 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f696001c070 con 0x7f696810bbe0 2024-03-04T09:08:59.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.811+0000 7f695effd640 1 --2- 172.21.15.71:0/4262240804 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f69540a1ef0 0x7f69540a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:08:59.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.811+0000 7f696da7b640 1 --2- 172.21.15.71:0/4262240804 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f69540a1ef0 0x7f69540a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:08:59.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.811+0000 7f695effd640 1 -- 172.21.15.71:0/4262240804 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9467+0+0 (secure 0 0 0) 0x7f69600cda00 con 0x7f696810bbe0 2024-03-04T09:08:59.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.811+0000 7f696da7b640 1 --2- 172.21.15.71:0/4262240804 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f69540a1ef0 0x7f69540a43b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6964009c20 tx=0x7f6964005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:08:59.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.815+0000 7f695effd640 1 -- 172.21.15.71:0/4262240804 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f696009a200 con 0x7f696810bbe0 2024-03-04T09:08:59.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.988+0000 7f696fcdf640 1 -- 172.21.15.71:0/4262240804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6968106cd0 con 0x7f696810bbe0 2024-03-04T09:08:59.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.989+0000 7f695effd640 1 -- 172.21.15.71:0/4262240804 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v102) v1 ==== 103+0+11 (secure 0 0 0) 0x7f696009e2a0 con 0x7f696810bbe0 2024-03-04T09:08:59.986 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-03-04T09:08:59.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.991+0000 7f696fcdf640 1 -- 172.21.15.71:0/4262240804 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f69540a1ef0 msgr2=0x7f69540a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:59.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.991+0000 7f696fcdf640 1 --2- 172.21.15.71:0/4262240804 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f69540a1ef0 0x7f69540a43b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6964009c20 tx=0x7f6964005f90 comp rx=0 tx=0).stop 2024-03-04T09:08:59.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.991+0000 7f696fcdf640 1 -- 172.21.15.71:0/4262240804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f696810bbe0 msgr2=0x7f6968100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:08:59.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.991+0000 7f696fcdf640 1 --2- 172.21.15.71:0/4262240804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f696810bbe0 0x7f6968100ee0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f696000b9e0 tx=0x7f696000bea0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.991+0000 7f696fcdf640 1 -- 172.21.15.71:0/4262240804 shutdown_connections 2024-03-04T09:08:59.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.991+0000 7f696fcdf640 1 --2- 172.21.15.71:0/4262240804 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f69540a1ef0 0x7f69540a43b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.991+0000 7f696fcdf640 1 --2- 172.21.15.71:0/4262240804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f696810bbe0 0x7f6968100ee0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.991+0000 7f696fcdf640 1 --2- 172.21.15.71:0/4262240804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6968103130 0x7f69681009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.991+0000 7f696fcdf640 1 --2- 172.21.15.71:0/4262240804 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f6968102810 0x7f6968100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:08:59.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.991+0000 7f696fcdf640 1 -- 172.21.15.71:0/4262240804 >> 172.21.15.71:0/4262240804 conn(0x7f6968076850 msgr2=0x7f69680fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:08:59.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.991+0000 7f696fcdf640 1 -- 172.21.15.71:0/4262240804 shutdown_connections 2024-03-04T09:08:59.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:08:59.991+0000 7f696fcdf640 1 -- 172.21.15.71:0/4262240804 wait complete. 2024-03-04T09:08:59.998 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-04T09:09:00.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.104+0000 7f9591483640 1 -- 172.21.15.71:0/4284614108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f958c101580 msgr2=0x7f958c101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:00.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.104+0000 7f9591483640 1 --2- 172.21.15.71:0/4284614108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f958c101580 0x7f958c101a00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9580009890 tx=0x7f958002bb70 comp rx=0 tx=0).stop 2024-03-04T09:09:00.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.104+0000 7f9591483640 1 -- 172.21.15.71:0/4284614108 shutdown_connections 2024-03-04T09:09:00.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.104+0000 7f9591483640 1 --2- 172.21.15.71:0/4284614108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f958c101f40 0x7f958c10e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:00.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.104+0000 7f9591483640 1 --2- 172.21.15.71:0/4284614108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f958c101580 0x7f958c101a00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:00.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.104+0000 7f9591483640 1 --2- 172.21.15.71:0/4284614108 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f958c107640 0x7f958c107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:00.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.104+0000 7f9591483640 1 -- 172.21.15.71:0/4284614108 >> 172.21.15.71:0/4284614108 conn(0x7f958c0789d0 msgr2=0x7f958c078de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:00.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.104+0000 7f9591483640 1 -- 172.21.15.71:0/4284614108 shutdown_connections 2024-03-04T09:09:00.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.104+0000 7f9591483640 1 -- 172.21.15.71:0/4284614108 wait complete. 2024-03-04T09:09:00.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.104+0000 7f9591483640 1 Processor -- start 2024-03-04T09:09:00.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.105+0000 7f9591483640 1 -- start start 2024-03-04T09:09:00.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.105+0000 7f9591483640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f958c101580 0x7f958c197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:00.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.105+0000 7f9591483640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f958c101f40 0x7f958c1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:00.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.105+0000 7f9591483640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f958c107640 0x7f958c19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:00.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.105+0000 7f9591483640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958c111900 con 0x7f958c101580 2024-03-04T09:09:00.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.105+0000 7f9591483640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958c111780 con 0x7f958c101f40 2024-03-04T09:09:00.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.105+0000 7f9591483640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958c111a80 con 0x7f958c107640 2024-03-04T09:09:00.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.105+0000 7f958affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f958c101580 0x7f958c197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:00.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.105+0000 7f958affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f958c101580 0x7f958c197d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57308/0 (socket says 172.21.15.71:57308) 2024-03-04T09:09:00.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.105+0000 7f958affd640 1 -- 172.21.15.71:0/3246742560 learned_addr learned my addr 172.21.15.71:0/3246742560 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T09:09:00.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.105+0000 7f958a7fc640 1 --2- 172.21.15.71:0/3246742560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f958c101f40 0x7f958c1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:00.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.105+0000 7f958b7fe640 1 --2- 172.21.15.71:0/3246742560 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f958c107640 0x7f958c19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:00.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.105+0000 7f958affd640 1 -- 172.21.15.71:0/3246742560 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f958c107640 msgr2=0x7f958c19c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:00.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.105+0000 7f958affd640 1 --2- 172.21.15.71:0/3246742560 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f958c107640 0x7f958c19c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:00.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.105+0000 7f958affd640 1 -- 172.21.15.71:0/3246742560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f958c101f40 msgr2=0x7f958c1982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:00.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.105+0000 7f958affd640 1 --2- 172.21.15.71:0/3246742560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f958c101f40 0x7f958c1982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:00.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.105+0000 7f958affd640 1 -- 172.21.15.71:0/3246742560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958c19cd70 con 0x7f958c101580 2024-03-04T09:09:00.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.105+0000 7f958a7fc640 1 --2- 172.21.15.71:0/3246742560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f958c101f40 0x7f958c1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:09:00.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.105+0000 7f958affd640 1 --2- 172.21.15.71:0/3246742560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f958c101580 0x7f958c197d80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f957c00c920 tx=0x7f957c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:00.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.105+0000 7f9573fff640 1 -- 172.21.15.71:0/3246742560 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c007c70 con 0x7f958c101580 2024-03-04T09:09:00.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.105+0000 7f9591483640 1 -- 172.21.15.71:0/3246742560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f958c19d060 con 0x7f958c101580 2024-03-04T09:09:00.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.105+0000 7f9573fff640 1 -- 172.21.15.71:0/3246742560 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f957c007e10 con 0x7f958c101580 2024-03-04T09:09:00.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.106+0000 7f9573fff640 1 -- 172.21.15.71:0/3246742560 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c018890 con 0x7f958c101580 2024-03-04T09:09:00.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.106+0000 7f9591483640 1 -- 172.21.15.71:0/3246742560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f958c1a48f0 con 0x7f958c101580 2024-03-04T09:09:00.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.106+0000 7f9591483640 1 -- 172.21.15.71:0/3246742560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958c068810 con 0x7f958c101580 2024-03-04T09:09:00.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.108+0000 7f9573fff640 1 -- 172.21.15.71:0/3246742560 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f957c018a50 con 0x7f958c101580 2024-03-04T09:09:00.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.108+0000 7f9573fff640 1 --2- 172.21.15.71:0/3246742560 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f95740a1ef0 0x7f95740a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:00.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.108+0000 7f958a7fc640 1 --2- 172.21.15.71:0/3246742560 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f95740a1ef0 0x7f95740a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:00.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.108+0000 7f9573fff640 1 -- 172.21.15.71:0/3246742560 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9467+0+0 (secure 0 0 0) 0x7f957c0ce920 con 0x7f958c101580 2024-03-04T09:09:00.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.109+0000 7f958a7fc640 1 --2- 172.21.15.71:0/3246742560 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f95740a1ef0 0x7f95740a43b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9580000c00 tx=0x7f958002e040 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:00.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.112+0000 7f9573fff640 1 -- 172.21.15.71:0/3246742560 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f957c014030 con 0x7f958c101580 2024-03-04T09:09:00.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.350+0000 7f9591483640 1 -- 172.21.15.71:0/3246742560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f958c1990d0 con 0x7f958c101580 2024-03-04T09:09:00.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.358+0000 7f9573fff640 1 -- 172.21.15.71:0/3246742560 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) v1 ==== 81+0+736924 (secure 0 0 0) 0x7f957c09b6f0 con 0x7f958c101580 2024-03-04T09:09:00.365 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T09:09:00.366 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-03-04T09:09:00.366 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 25, 2024-03-04T09:09:00.366 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.366 INFO:teuthology.orchestra.run.smithi071.stdout: "active_gid": 15399, 2024-03-04T09:09:00.366 INFO:teuthology.orchestra.run.smithi071.stdout: "active_name": "x", 2024-03-04T09:09:00.366 INFO:teuthology.orchestra.run.smithi071.stdout: "active_addrs": { 2024-03-04T09:09:00.366 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-03-04T09:09:00.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.367 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-03-04T09:09:00.367 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2024-03-04T09:09:00.367 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 217968316 2024-03-04T09:09:00.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.367 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-03-04T09:09:00.367 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2024-03-04T09:09:00.368 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 217968316 2024-03-04T09:09:00.368 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.368 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-03-04T09:09:00.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.368 INFO:teuthology.orchestra.run.smithi071.stdout: "active_addr": "172.21.15.71:6801/217968316", 2024-03-04T09:09:00.368 INFO:teuthology.orchestra.run.smithi071.stdout: "active_change": "2024-03-04T08:58:54.820260+0000", 2024-03-04T09:09:00.368 INFO:teuthology.orchestra.run.smithi071.stdout: "active_mgr_features": 4540701547738038271, 2024-03-04T09:09:00.368 INFO:teuthology.orchestra.run.smithi071.stdout: "available": true, 2024-03-04T09:09:00.369 INFO:teuthology.orchestra.run.smithi071.stdout: "standbys": [ 2024-03-04T09:09:00.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.369 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 25033, 2024-03-04T09:09:00.369 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "z", 2024-03-04T09:09:00.369 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_features": 4540701547738038271, 2024-03-04T09:09:00.369 INFO:teuthology.orchestra.run.smithi071.stdout: "available_modules": [ 2024-03-04T09:09:00.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.369 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "alerts", 2024-03-04T09:09:00.369 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.370 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.370 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.370 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-03-04T09:09:00.370 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-03-04T09:09:00.370 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:00.370 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.370 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.370 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-03-04T09:09:00.370 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.370 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.371 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.371 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "How frequently to reexamine health status", 2024-03-04T09:09:00.371 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.371 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.371 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.371 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.371 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.371 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.372 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.372 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.372 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.372 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.372 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.372 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.372 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.372 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.373 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.373 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.373 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.373 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.373 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.373 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.373 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.373 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.373 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.374 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.374 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.374 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.374 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.374 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.374 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.374 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.375 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.375 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.375 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.375 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.375 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.375 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.375 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.376 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.376 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.376 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.376 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.376 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.376 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.376 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.376 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.376 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.377 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.377 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.377 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.377 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.377 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.377 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.377 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.377 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.378 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.378 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.378 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.378 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.378 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.378 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.378 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.379 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.379 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.379 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.379 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.379 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.379 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.379 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.379 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.380 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_destination": { 2024-03-04T09:09:00.380 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_destination", 2024-03-04T09:09:00.380 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.380 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.380 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.380 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.380 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.381 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.381 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.381 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Email address to send alerts to", 2024-03-04T09:09:00.381 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.381 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.381 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.381 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_from_name": { 2024-03-04T09:09:00.381 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_from_name", 2024-03-04T09:09:00.382 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.382 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.382 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.382 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "Ceph", 2024-03-04T09:09:00.382 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.382 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.382 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.382 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Email From: name", 2024-03-04T09:09:00.383 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.383 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.383 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.383 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_host": { 2024-03-04T09:09:00.383 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_host", 2024-03-04T09:09:00.383 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.383 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.384 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.384 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.384 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.384 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.384 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.384 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "SMTP server", 2024-03-04T09:09:00.384 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.384 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.384 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.385 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_password": { 2024-03-04T09:09:00.385 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_password", 2024-03-04T09:09:00.385 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.385 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.385 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.385 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.385 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.386 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.386 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.386 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Password to authenticate with", 2024-03-04T09:09:00.386 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.386 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.386 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.386 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_port": { 2024-03-04T09:09:00.387 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_port", 2024-03-04T09:09:00.387 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.387 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.387 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.387 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "465", 2024-03-04T09:09:00.387 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.387 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.387 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.387 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "SMTP port", 2024-03-04T09:09:00.388 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.388 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.388 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.388 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_sender": { 2024-03-04T09:09:00.388 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_sender", 2024-03-04T09:09:00.388 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.388 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.389 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.389 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.389 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.389 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.389 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.389 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "SMTP envelope sender", 2024-03-04T09:09:00.389 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.389 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.390 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.390 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_ssl": { 2024-03-04T09:09:00.390 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_ssl", 2024-03-04T09:09:00.390 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.390 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.390 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.390 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.391 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.391 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.391 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.391 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-04T09:09:00.391 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.391 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.391 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.392 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_user": { 2024-03-04T09:09:00.392 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_user", 2024-03-04T09:09:00.392 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.392 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.392 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.392 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.392 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.392 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.393 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.393 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "User to authenticate as", 2024-03-04T09:09:00.393 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.393 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.393 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.393 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.393 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.394 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "balancer", 2024-03-04T09:09:00.394 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.394 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.394 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.394 INFO:teuthology.orchestra.run.smithi071.stdout: "active": { 2024-03-04T09:09:00.394 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "active", 2024-03-04T09:09:00.394 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.394 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.395 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.395 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.395 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.395 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.395 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.395 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "automatically balance PGs across cluster", 2024-03-04T09:09:00.395 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.395 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.395 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.396 INFO:teuthology.orchestra.run.smithi071.stdout: "begin_time": { 2024-03-04T09:09:00.396 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "begin_time", 2024-03-04T09:09:00.396 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.396 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.396 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.396 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0000", 2024-03-04T09:09:00.396 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.397 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.397 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.397 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "beginning time of day to automatically balance", 2024-03-04T09:09:00.397 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-04T09:09:00.397 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.397 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.397 INFO:teuthology.orchestra.run.smithi071.stdout: "begin_weekday": { 2024-03-04T09:09:00.397 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "begin_weekday", 2024-03-04T09:09:00.398 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-03-04T09:09:00.398 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.398 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.398 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-03-04T09:09:00.398 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "0", 2024-03-04T09:09:00.398 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "6", 2024-03-04T09:09:00.398 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.398 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-04T09:09:00.399 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-04T09:09:00.399 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.399 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.399 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_compat_max_iterations": { 2024-03-04T09:09:00.399 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crush_compat_max_iterations", 2024-03-04T09:09:00.399 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-03-04T09:09:00.399 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.399 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.400 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "25", 2024-03-04T09:09:00.400 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1", 2024-03-04T09:09:00.400 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "250", 2024-03-04T09:09:00.400 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.400 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-04T09:09:00.400 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.400 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.400 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.401 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_compat_metrics": { 2024-03-04T09:09:00.401 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crush_compat_metrics", 2024-03-04T09:09:00.401 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.401 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.401 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.401 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "pgs,objects,bytes", 2024-03-04T09:09:00.401 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.402 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.402 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.402 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-04T09:09:00.402 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-04T09:09:00.402 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.402 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.402 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_compat_step": { 2024-03-04T09:09:00.402 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crush_compat_step", 2024-03-04T09:09:00.403 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-03-04T09:09:00.403 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.403 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.403 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0.5", 2024-03-04T09:09:00.403 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "0.001", 2024-03-04T09:09:00.403 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "0.999", 2024-03-04T09:09:00.403 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.403 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "aggressiveness of optimization", 2024-03-04T09:09:00.403 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-04T09:09:00.404 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.404 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.404 INFO:teuthology.orchestra.run.smithi071.stdout: "end_time": { 2024-03-04T09:09:00.404 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "end_time", 2024-03-04T09:09:00.404 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.404 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.404 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.405 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "2359", 2024-03-04T09:09:00.405 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.405 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.405 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.405 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "ending time of day to automatically balance", 2024-03-04T09:09:00.405 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-04T09:09:00.405 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.405 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.406 INFO:teuthology.orchestra.run.smithi071.stdout: "end_weekday": { 2024-03-04T09:09:00.406 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "end_weekday", 2024-03-04T09:09:00.406 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-03-04T09:09:00.406 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.406 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.406 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-03-04T09:09:00.406 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "0", 2024-03-04T09:09:00.406 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "6", 2024-03-04T09:09:00.407 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.407 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-04T09:09:00.407 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-04T09:09:00.407 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.407 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.407 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.407 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.407 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.408 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.408 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.408 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.408 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.408 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.408 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.408 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.408 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.409 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.409 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.409 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.409 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.409 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.409 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.409 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.409 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.409 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.410 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.411 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.411 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.411 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.411 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.411 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.412 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.412 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.412 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.412 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.412 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.412 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.412 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.413 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.413 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.413 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.413 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.413 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.413 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.413 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.414 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.414 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.414 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.414 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.414 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.414 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.414 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.414 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.415 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.415 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.415 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.415 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.415 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.415 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.415 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.415 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.416 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.416 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.416 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.416 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.416 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.416 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.416 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.416 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.417 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.417 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.417 INFO:teuthology.orchestra.run.smithi071.stdout: "min_score": { 2024-03-04T09:09:00.417 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "min_score", 2024-03-04T09:09:00.418 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-03-04T09:09:00.418 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.418 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.418 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-03-04T09:09:00.418 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.418 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.418 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.419 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-04T09:09:00.419 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.419 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.419 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.419 INFO:teuthology.orchestra.run.smithi071.stdout: "mode": { 2024-03-04T09:09:00.419 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mode", 2024-03-04T09:09:00.419 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.420 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.420 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.420 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "upmap", 2024-03-04T09:09:00.420 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.420 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.420 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.420 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-compat", 2024-03-04T09:09:00.420 INFO:teuthology.orchestra.run.smithi071.stdout: "none", 2024-03-04T09:09:00.421 INFO:teuthology.orchestra.run.smithi071.stdout: "read", 2024-03-04T09:09:00.421 INFO:teuthology.orchestra.run.smithi071.stdout: "upmap", 2024-03-04T09:09:00.421 INFO:teuthology.orchestra.run.smithi071.stdout: "upmap-read" 2024-03-04T09:09:00.421 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.421 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Balancer mode", 2024-03-04T09:09:00.421 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.421 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.421 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.422 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_ids": { 2024-03-04T09:09:00.422 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "pool_ids", 2024-03-04T09:09:00.422 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.422 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.422 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.422 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.422 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.422 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.422 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.423 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-04T09:09:00.423 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.423 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.423 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.423 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-03-04T09:09:00.424 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-03-04T09:09:00.424 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:00.424 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.424 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.425 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-03-04T09:09:00.425 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.425 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.425 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.425 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-04T09:09:00.425 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.425 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.425 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.426 INFO:teuthology.orchestra.run.smithi071.stdout: "upmap_max_deviation": { 2024-03-04T09:09:00.426 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "upmap_max_deviation", 2024-03-04T09:09:00.426 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.426 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.426 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.426 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5", 2024-03-04T09:09:00.426 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1", 2024-03-04T09:09:00.426 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.427 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.427 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-04T09:09:00.427 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-04T09:09:00.427 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.427 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.427 INFO:teuthology.orchestra.run.smithi071.stdout: "upmap_max_optimizations": { 2024-03-04T09:09:00.427 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "upmap_max_optimizations", 2024-03-04T09:09:00.427 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-03-04T09:09:00.428 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.428 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.428 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-03-04T09:09:00.428 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.428 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.428 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.428 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-04T09:09:00.428 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.429 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.429 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.429 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.429 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.430 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephadm", 2024-03-04T09:09:00.430 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.430 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.430 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.430 INFO:teuthology.orchestra.run.smithi071.stdout: "agent_down_multiplier": { 2024-03-04T09:09:00.430 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "agent_down_multiplier", 2024-03-04T09:09:00.431 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-03-04T09:09:00.431 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.431 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.431 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "3.0", 2024-03-04T09:09:00.431 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.431 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.431 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.431 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-04T09:09:00.431 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.432 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.432 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.432 INFO:teuthology.orchestra.run.smithi071.stdout: "agent_refresh_rate": { 2024-03-04T09:09:00.432 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "agent_refresh_rate", 2024-03-04T09:09:00.432 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:00.432 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.432 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.433 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "20", 2024-03-04T09:09:00.433 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.433 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.433 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.433 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-04T09:09:00.433 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.433 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.433 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.434 INFO:teuthology.orchestra.run.smithi071.stdout: "agent_starting_port": { 2024-03-04T09:09:00.434 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "agent_starting_port", 2024-03-04T09:09:00.434 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.434 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.434 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.434 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "4721", 2024-03-04T09:09:00.434 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.435 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.435 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.435 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-04T09:09:00.435 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.435 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.435 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.435 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_ptrace": { 2024-03-04T09:09:00.436 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "allow_ptrace", 2024-03-04T09:09:00.436 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.436 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.436 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.436 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.436 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.436 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.436 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.437 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-04T09:09:00.437 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-04T09:09:00.437 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.437 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.437 INFO:teuthology.orchestra.run.smithi071.stdout: "autotune_interval": { 2024-03-04T09:09:00.437 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "autotune_interval", 2024-03-04T09:09:00.437 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:00.438 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.438 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.438 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-03-04T09:09:00.438 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.438 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.438 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.438 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-04T09:09:00.438 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.438 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.439 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.439 INFO:teuthology.orchestra.run.smithi071.stdout: "autotune_memory_target_ratio": { 2024-03-04T09:09:00.439 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "autotune_memory_target_ratio", 2024-03-04T09:09:00.439 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-03-04T09:09:00.439 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.439 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.439 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0.7", 2024-03-04T09:09:00.440 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.440 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.440 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.440 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-04T09:09:00.440 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.440 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.440 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.441 INFO:teuthology.orchestra.run.smithi071.stdout: "cephadm_log_destination": { 2024-03-04T09:09:00.441 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephadm_log_destination", 2024-03-04T09:09:00.441 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.441 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.441 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.441 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.441 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.441 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.441 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.442 INFO:teuthology.orchestra.run.smithi071.stdout: "file", 2024-03-04T09:09:00.442 INFO:teuthology.orchestra.run.smithi071.stdout: "file,syslog", 2024-03-04T09:09:00.442 INFO:teuthology.orchestra.run.smithi071.stdout: "syslog" 2024-03-04T09:09:00.442 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.442 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-04T09:09:00.442 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.442 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.442 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.443 INFO:teuthology.orchestra.run.smithi071.stdout: "cgroups_split": { 2024-03-04T09:09:00.443 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cgroups_split", 2024-03-04T09:09:00.443 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.443 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.443 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.443 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.443 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.444 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.444 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.444 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-04T09:09:00.444 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.444 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.444 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.444 INFO:teuthology.orchestra.run.smithi071.stdout: "config_checks_enabled": { 2024-03-04T09:09:00.445 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "config_checks_enabled", 2024-03-04T09:09:00.445 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.445 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.445 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.445 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.445 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.445 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.445 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.446 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-04T09:09:00.446 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.446 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.446 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.446 INFO:teuthology.orchestra.run.smithi071.stdout: "config_dashboard": { 2024-03-04T09:09:00.446 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "config_dashboard", 2024-03-04T09:09:00.446 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.446 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.447 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.447 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.447 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.447 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.447 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.447 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-04T09:09:00.447 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.447 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.448 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.448 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_alertmanager": { 2024-03-04T09:09:00.448 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_alertmanager", 2024-03-04T09:09:00.448 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.448 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.448 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.448 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-04T09:09:00.449 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.449 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.449 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.449 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Prometheus container image", 2024-03-04T09:09:00.449 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.449 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.449 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.449 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_base": { 2024-03-04T09:09:00.450 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_base", 2024-03-04T09:09:00.450 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.450 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.450 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.450 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-04T09:09:00.450 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.450 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.450 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.451 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Container image name, without the tag", 2024-03-04T09:09:00.451 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.451 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.451 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.451 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_elasticsearch": { 2024-03-04T09:09:00.451 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_elasticsearch", 2024-03-04T09:09:00.451 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.452 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.452 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.452 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-04T09:09:00.452 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.452 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.452 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.452 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "elasticsearch container image", 2024-03-04T09:09:00.452 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.453 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.453 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.453 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_grafana": { 2024-03-04T09:09:00.453 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_grafana", 2024-03-04T09:09:00.453 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.453 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.453 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.453 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-04T09:09:00.454 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.454 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.454 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.454 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Prometheus container image", 2024-03-04T09:09:00.454 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.454 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.454 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.455 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_haproxy": { 2024-03-04T09:09:00.455 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_haproxy", 2024-03-04T09:09:00.455 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.455 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.455 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.455 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-04T09:09:00.455 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.456 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.456 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.456 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "HAproxy container image", 2024-03-04T09:09:00.456 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.456 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.456 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.456 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_jaeger_agent": { 2024-03-04T09:09:00.457 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_jaeger_agent", 2024-03-04T09:09:00.457 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.457 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.457 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.457 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-04T09:09:00.457 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.457 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.457 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.458 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Jaeger agent container image", 2024-03-04T09:09:00.458 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.458 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.458 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.458 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_jaeger_collector": { 2024-03-04T09:09:00.458 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_jaeger_collector", 2024-03-04T09:09:00.458 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.458 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.458 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.459 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-04T09:09:00.459 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.459 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.459 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.459 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Jaeger collector container image", 2024-03-04T09:09:00.459 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.459 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.459 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.460 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_jaeger_query": { 2024-03-04T09:09:00.460 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_jaeger_query", 2024-03-04T09:09:00.460 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.460 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.460 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.460 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-04T09:09:00.460 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.461 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.461 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.461 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Jaeger query container image", 2024-03-04T09:09:00.461 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.461 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.461 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.461 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_keepalived": { 2024-03-04T09:09:00.462 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_keepalived", 2024-03-04T09:09:00.462 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.462 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.462 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.462 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-04T09:09:00.462 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.462 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.462 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.462 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Keepalived container image", 2024-03-04T09:09:00.463 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.463 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.463 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.463 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_loki": { 2024-03-04T09:09:00.463 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_loki", 2024-03-04T09:09:00.463 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.463 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.464 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.464 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-04T09:09:00.464 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.464 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.464 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.464 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Loki container image", 2024-03-04T09:09:00.464 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.464 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.465 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.465 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_node_exporter": { 2024-03-04T09:09:00.465 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_node_exporter", 2024-03-04T09:09:00.465 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.465 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.465 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.465 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-04T09:09:00.465 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.466 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.466 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.466 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Prometheus container image", 2024-03-04T09:09:00.466 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.466 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.466 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.466 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_nvmeof": { 2024-03-04T09:09:00.466 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_nvmeof", 2024-03-04T09:09:00.467 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.467 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.467 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.467 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-04T09:09:00.467 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.467 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.467 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.467 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Nvme-of container image", 2024-03-04T09:09:00.468 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.468 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.468 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.468 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_prometheus": { 2024-03-04T09:09:00.468 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_prometheus", 2024-03-04T09:09:00.468 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.468 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.468 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.469 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-04T09:09:00.469 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.469 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.469 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.469 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Prometheus container image", 2024-03-04T09:09:00.469 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.469 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.469 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.470 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_promtail": { 2024-03-04T09:09:00.470 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_promtail", 2024-03-04T09:09:00.470 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.470 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.470 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.470 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-04T09:09:00.470 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.471 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.471 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.471 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Promtail container image", 2024-03-04T09:09:00.471 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.471 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.471 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.471 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_snmp_gateway": { 2024-03-04T09:09:00.471 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_snmp_gateway", 2024-03-04T09:09:00.472 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.472 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.472 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.472 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-04T09:09:00.472 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.472 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.472 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.472 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "SNMP Gateway container image", 2024-03-04T09:09:00.473 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.473 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.473 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.473 INFO:teuthology.orchestra.run.smithi071.stdout: "container_init": { 2024-03-04T09:09:00.473 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_init", 2024-03-04T09:09:00.473 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.473 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.474 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.474 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.474 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.474 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.474 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.474 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Run podman/docker with `--init`", 2024-03-04T09:09:00.474 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.474 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.474 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.475 INFO:teuthology.orchestra.run.smithi071.stdout: "daemon_cache_timeout": { 2024-03-04T09:09:00.475 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "daemon_cache_timeout", 2024-03-04T09:09:00.475 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:00.475 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.475 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.475 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-03-04T09:09:00.475 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.476 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.476 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.476 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-04T09:09:00.476 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.476 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.476 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.477 INFO:teuthology.orchestra.run.smithi071.stdout: "default_cephadm_command_timeout": { 2024-03-04T09:09:00.477 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "default_cephadm_command_timeout", 2024-03-04T09:09:00.477 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.477 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.477 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.477 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "900", 2024-03-04T09:09:00.477 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.477 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.477 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.478 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-04T09:09:00.478 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.478 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.478 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.478 INFO:teuthology.orchestra.run.smithi071.stdout: "default_registry": { 2024-03-04T09:09:00.478 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "default_registry", 2024-03-04T09:09:00.478 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.479 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.479 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.479 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "docker.io", 2024-03-04T09:09:00.479 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.479 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.479 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.479 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-04T09:09:00.479 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.480 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.480 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.480 INFO:teuthology.orchestra.run.smithi071.stdout: "device_cache_timeout": { 2024-03-04T09:09:00.480 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "device_cache_timeout", 2024-03-04T09:09:00.480 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:00.480 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.480 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.481 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "1800", 2024-03-04T09:09:00.481 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.481 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.481 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.481 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "seconds to cache device inventory", 2024-03-04T09:09:00.481 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.481 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.481 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.482 INFO:teuthology.orchestra.run.smithi071.stdout: "device_enhanced_scan": { 2024-03-04T09:09:00.482 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "device_enhanced_scan", 2024-03-04T09:09:00.482 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.482 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.482 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.482 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.482 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.482 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.483 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.483 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-04T09:09:00.483 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.483 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.483 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.483 INFO:teuthology.orchestra.run.smithi071.stdout: "facts_cache_timeout": { 2024-03-04T09:09:00.483 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "facts_cache_timeout", 2024-03-04T09:09:00.483 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:00.484 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.484 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.484 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-03-04T09:09:00.484 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.484 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.484 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.485 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "seconds to cache host facts data", 2024-03-04T09:09:00.485 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.485 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.485 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.485 INFO:teuthology.orchestra.run.smithi071.stdout: "host_check_interval": { 2024-03-04T09:09:00.485 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "host_check_interval", 2024-03-04T09:09:00.485 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:00.486 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.486 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.486 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-03-04T09:09:00.486 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.486 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.486 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.486 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to perform a host check", 2024-03-04T09:09:00.486 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.487 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.487 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.487 INFO:teuthology.orchestra.run.smithi071.stdout: "hw_monitoring": { 2024-03-04T09:09:00.487 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "hw_monitoring", 2024-03-04T09:09:00.487 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.487 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.488 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.488 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.488 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.488 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.488 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.488 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-04T09:09:00.488 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.488 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.489 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.489 INFO:teuthology.orchestra.run.smithi071.stdout: "inventory_list_all": { 2024-03-04T09:09:00.489 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "inventory_list_all", 2024-03-04T09:09:00.489 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.489 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.489 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.489 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.490 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.490 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.490 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.490 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-04T09:09:00.490 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.490 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.490 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.491 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.491 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.491 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.491 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.491 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.491 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.491 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.491 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.491 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.492 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.492 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.492 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.492 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.492 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.492 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.492 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.493 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.493 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.493 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.493 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.493 INFO:teuthology.orchestra.run.smithi071.stdout: "log_refresh_metadata": { 2024-03-04T09:09:00.493 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_refresh_metadata", 2024-03-04T09:09:00.493 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.494 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.494 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.494 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.494 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.494 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.494 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.494 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-04T09:09:00.494 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.494 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.495 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.495 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.495 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.495 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.495 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.495 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.495 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.496 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.496 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.496 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.496 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-04T09:09:00.496 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.496 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.496 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.497 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.497 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.497 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.497 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.497 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.497 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.497 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.498 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.498 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.498 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.498 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.498 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.498 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.498 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.498 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.498 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.499 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.499 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.499 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.499 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.499 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.499 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.499 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.500 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.500 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.500 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.500 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.500 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.500 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.500 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.500 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.501 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.501 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.501 INFO:teuthology.orchestra.run.smithi071.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-04T09:09:00.501 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-04T09:09:00.501 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.501 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.501 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.501 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.502 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.502 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.502 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.502 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-04T09:09:00.502 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.502 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.502 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.503 INFO:teuthology.orchestra.run.smithi071.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-04T09:09:00.503 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-04T09:09:00.503 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.503 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.503 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.503 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "*", 2024-03-04T09:09:00.503 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.503 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.503 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.504 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-04T09:09:00.504 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.504 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.504 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.504 INFO:teuthology.orchestra.run.smithi071.stdout: "max_count_per_host": { 2024-03-04T09:09:00.504 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_count_per_host", 2024-03-04T09:09:00.504 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.504 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.505 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.505 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-03-04T09:09:00.505 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.505 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.505 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.505 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "max number of daemons per service per host", 2024-03-04T09:09:00.505 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.505 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.506 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.506 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd_draining_count": { 2024-03-04T09:09:00.506 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_osd_draining_count", 2024-03-04T09:09:00.506 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.506 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.506 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.506 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-03-04T09:09:00.506 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.507 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.507 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.507 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-04T09:09:00.507 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.507 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.507 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.507 INFO:teuthology.orchestra.run.smithi071.stdout: "migration_current": { 2024-03-04T09:09:00.507 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "migration_current", 2024-03-04T09:09:00.508 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.508 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.508 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.508 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.508 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.508 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.508 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.508 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "internal - do not modify", 2024-03-04T09:09:00.509 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.509 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.509 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.509 INFO:teuthology.orchestra.run.smithi071.stdout: "mode": { 2024-03-04T09:09:00.509 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mode", 2024-03-04T09:09:00.509 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.509 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.509 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.510 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "root", 2024-03-04T09:09:00.510 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.510 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.510 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.510 INFO:teuthology.orchestra.run.smithi071.stdout: "cephadm-package", 2024-03-04T09:09:00.510 INFO:teuthology.orchestra.run.smithi071.stdout: "root" 2024-03-04T09:09:00.510 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.510 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "mode for remote execution of cephadm", 2024-03-04T09:09:00.511 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.511 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.511 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.511 INFO:teuthology.orchestra.run.smithi071.stdout: "oob_default_addr": { 2024-03-04T09:09:00.511 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "oob_default_addr", 2024-03-04T09:09:00.511 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.511 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.511 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.512 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "169.254.1.1", 2024-03-04T09:09:00.512 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.512 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.512 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.512 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-04T09:09:00.515 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.515 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.515 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.515 INFO:teuthology.orchestra.run.smithi071.stdout: "prometheus_alerts_path": { 2024-03-04T09:09:00.516 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "prometheus_alerts_path", 2024-03-04T09:09:00.516 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.516 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.516 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.516 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-04T09:09:00.516 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.516 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.516 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.517 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-04T09:09:00.517 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.517 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.517 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.517 INFO:teuthology.orchestra.run.smithi071.stdout: "registry_insecure": { 2024-03-04T09:09:00.517 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "registry_insecure", 2024-03-04T09:09:00.517 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.517 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.518 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.518 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.518 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.518 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.518 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.518 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-04T09:09:00.518 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.518 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.518 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.519 INFO:teuthology.orchestra.run.smithi071.stdout: "registry_password": { 2024-03-04T09:09:00.519 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "registry_password", 2024-03-04T09:09:00.519 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.519 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.519 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.519 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.519 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.520 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.520 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.520 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-04T09:09:00.520 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.520 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.520 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.520 INFO:teuthology.orchestra.run.smithi071.stdout: "registry_url": { 2024-03-04T09:09:00.520 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "registry_url", 2024-03-04T09:09:00.521 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.521 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.521 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.521 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.521 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.521 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.521 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.522 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-04T09:09:00.522 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.522 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.522 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.522 INFO:teuthology.orchestra.run.smithi071.stdout: "registry_username": { 2024-03-04T09:09:00.522 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "registry_username", 2024-03-04T09:09:00.522 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.522 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.523 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.523 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.523 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.523 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.523 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.523 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-04T09:09:00.523 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.523 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.524 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.524 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.524 INFO:teuthology.orchestra.run.smithi071.stdout: "secure_monitoring_stack": { 2024-03-04T09:09:00.524 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "secure_monitoring_stack", 2024-03-04T09:09:00.524 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.524 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.524 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.524 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.524 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.525 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.525 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.525 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-04T09:09:00.525 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.525 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.525 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.525 INFO:teuthology.orchestra.run.smithi071.stdout: "service_discovery_port": { 2024-03-04T09:09:00.526 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "service_discovery_port", 2024-03-04T09:09:00.526 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.526 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.526 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.526 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8765", 2024-03-04T09:09:00.526 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.526 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.526 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.526 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "cephadm service discovery port", 2024-03-04T09:09:00.527 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.527 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.527 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.527 INFO:teuthology.orchestra.run.smithi071.stdout: "ssh_config_file": { 2024-03-04T09:09:00.527 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ssh_config_file", 2024-03-04T09:09:00.527 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.527 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.527 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.528 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.528 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.528 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.528 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.528 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-04T09:09:00.528 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.528 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.528 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.529 INFO:teuthology.orchestra.run.smithi071.stdout: "use_agent": { 2024-03-04T09:09:00.529 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "use_agent", 2024-03-04T09:09:00.529 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.529 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.529 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.529 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.529 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.530 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.530 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.530 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-04T09:09:00.530 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.530 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.530 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.530 INFO:teuthology.orchestra.run.smithi071.stdout: "use_repo_digest": { 2024-03-04T09:09:00.530 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "use_repo_digest", 2024-03-04T09:09:00.531 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.531 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.531 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.531 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.531 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.531 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.531 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.531 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-04T09:09:00.531 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.532 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.532 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.532 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_on_failed_host_check": { 2024-03-04T09:09:00.532 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_on_failed_host_check", 2024-03-04T09:09:00.532 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.532 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.532 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.532 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.533 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.533 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.533 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.533 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "raise a health warning if the host check fails", 2024-03-04T09:09:00.533 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.533 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.533 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.534 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_on_stray_daemons": { 2024-03-04T09:09:00.534 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_on_stray_daemons", 2024-03-04T09:09:00.534 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.534 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.534 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.534 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.534 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.534 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.534 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.535 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-04T09:09:00.535 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.535 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.535 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.535 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_on_stray_hosts": { 2024-03-04T09:09:00.535 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_on_stray_hosts", 2024-03-04T09:09:00.535 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.535 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.536 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.536 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.536 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.536 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.536 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.536 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-04T09:09:00.536 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.536 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.537 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.537 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.537 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.537 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crash", 2024-03-04T09:09:00.537 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.537 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.538 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.538 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.538 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.538 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.538 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.538 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.538 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.538 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.538 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.539 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.539 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.539 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.539 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.539 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.539 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.539 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.539 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.539 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.540 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.540 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.540 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.540 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.540 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.540 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.540 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.541 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.541 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.541 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.541 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.541 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.541 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.541 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.541 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.542 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.542 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.542 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.542 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.542 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.542 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.542 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.543 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.543 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.543 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.543 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.543 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.543 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.543 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.543 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.543 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.544 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.544 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.544 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.544 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.544 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.544 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.544 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.545 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.545 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.545 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.545 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.545 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.545 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.545 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.545 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.545 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.546 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.546 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.546 INFO:teuthology.orchestra.run.smithi071.stdout: "retain_interval": { 2024-03-04T09:09:00.546 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "retain_interval", 2024-03-04T09:09:00.546 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:00.546 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.546 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.546 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "31536000", 2024-03-04T09:09:00.547 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.547 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.547 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.547 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-04T09:09:00.547 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.547 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.547 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.548 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_recent_interval": { 2024-03-04T09:09:00.548 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_recent_interval", 2024-03-04T09:09:00.548 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:00.548 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.548 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.548 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "1209600", 2024-03-04T09:09:00.548 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.548 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.548 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.549 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-04T09:09:00.549 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.549 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.549 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.549 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.549 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.550 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "dashboard", 2024-03-04T09:09:00.550 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.550 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.550 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.550 INFO:teuthology.orchestra.run.smithi071.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-04T09:09:00.550 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-04T09:09:00.550 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.550 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.550 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.551 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-03-04T09:09:00.551 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.551 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.551 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.551 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.551 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.551 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.551 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.552 INFO:teuthology.orchestra.run.smithi071.stdout: "ALERTMANAGER_API_HOST": { 2024-03-04T09:09:00.552 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-04T09:09:00.552 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.552 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.552 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.552 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.552 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.552 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.553 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.553 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.553 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.553 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.553 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.553 INFO:teuthology.orchestra.run.smithi071.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-04T09:09:00.553 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-04T09:09:00.553 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.554 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.554 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.554 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.554 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.554 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.554 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.554 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.554 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.555 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.555 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.555 INFO:teuthology.orchestra.run.smithi071.stdout: "AUDIT_API_ENABLED": { 2024-03-04T09:09:00.555 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "AUDIT_API_ENABLED", 2024-03-04T09:09:00.555 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.555 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.555 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.555 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.556 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.556 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.556 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.556 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.556 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.556 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.556 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.557 INFO:teuthology.orchestra.run.smithi071.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-04T09:09:00.557 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-04T09:09:00.557 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.557 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.557 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.557 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.557 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.557 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.557 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.558 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.558 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.558 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.558 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.558 INFO:teuthology.orchestra.run.smithi071.stdout: "ENABLE_BROWSABLE_API": { 2024-03-04T09:09:00.558 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-04T09:09:00.558 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.558 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.559 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.559 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.559 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.559 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.559 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.559 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.559 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.559 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.560 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.560 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-04T09:09:00.560 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-04T09:09:00.560 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.560 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.560 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.560 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.561 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.561 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.561 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.561 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.561 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.561 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.561 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.562 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-04T09:09:00.562 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-04T09:09:00.562 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.562 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.562 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.562 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.562 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.562 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.562 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.563 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.563 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.563 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.563 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.563 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-04T09:09:00.563 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-04T09:09:00.564 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.564 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.564 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.564 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.564 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.564 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.564 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.564 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.565 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.565 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.565 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.565 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-04T09:09:00.565 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-04T09:09:00.565 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.565 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.565 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.566 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.566 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.566 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.566 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.566 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.566 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.566 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.566 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.567 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-04T09:09:00.567 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-04T09:09:00.567 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.567 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.567 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.567 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.567 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.567 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.568 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.568 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.568 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.568 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.568 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.568 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-04T09:09:00.568 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-04T09:09:00.569 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.569 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.569 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.569 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.569 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.569 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.569 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.569 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.569 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.570 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.570 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.570 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-04T09:09:00.570 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-04T09:09:00.570 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.570 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.570 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.571 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.571 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.571 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.571 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.571 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.571 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.571 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.571 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.572 INFO:teuthology.orchestra.run.smithi071.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-04T09:09:00.572 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-04T09:09:00.572 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.572 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.572 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.572 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.572 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.572 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.573 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.573 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.573 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.573 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.573 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.573 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_API_PASSWORD": { 2024-03-04T09:09:00.573 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-04T09:09:00.573 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.574 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.574 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.574 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "admin", 2024-03-04T09:09:00.574 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.574 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.574 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.574 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.574 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.575 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.575 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.575 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-04T09:09:00.575 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-04T09:09:00.575 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.575 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.575 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.575 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.576 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.576 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.576 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.576 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.576 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.576 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.576 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.577 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_API_URL": { 2024-03-04T09:09:00.577 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_API_URL", 2024-03-04T09:09:00.577 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.577 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.577 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.577 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.577 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.577 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.578 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.578 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.578 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.578 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.578 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.578 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_API_USERNAME": { 2024-03-04T09:09:00.578 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-04T09:09:00.579 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.579 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.579 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.579 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "admin", 2024-03-04T09:09:00.579 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.579 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.579 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.579 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.580 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.580 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.580 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.580 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-04T09:09:00.580 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-04T09:09:00.580 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.580 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.581 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.581 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.581 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.581 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.581 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.581 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.581 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.581 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.581 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.582 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-04T09:09:00.582 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-04T09:09:00.582 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.582 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.582 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.582 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.582 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.583 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.583 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.583 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.583 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.583 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.583 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.583 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.583 INFO:teuthology.orchestra.run.smithi071.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-04T09:09:00.583 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-04T09:09:00.584 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.584 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.584 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.584 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.584 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.584 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.584 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.584 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.584 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.585 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.585 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.585 INFO:teuthology.orchestra.run.smithi071.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-04T09:09:00.585 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-04T09:09:00.585 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.585 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.586 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.586 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.586 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.586 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.586 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.586 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.586 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.586 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.586 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.587 INFO:teuthology.orchestra.run.smithi071.stdout: "PROMETHEUS_API_HOST": { 2024-03-04T09:09:00.587 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-04T09:09:00.587 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.587 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.587 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.587 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.587 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.587 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.588 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.588 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.588 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.588 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.588 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.588 INFO:teuthology.orchestra.run.smithi071.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-04T09:09:00.588 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-04T09:09:00.589 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.589 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.589 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.589 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.589 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.589 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.589 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.589 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.589 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.590 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.590 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.590 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.590 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-04T09:09:00.590 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-04T09:09:00.590 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.590 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.590 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.591 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.591 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.591 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.591 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.591 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.591 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.591 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.591 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.591 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.592 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-04T09:09:00.592 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-04T09:09:00.592 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.592 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.592 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.592 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.592 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.592 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.593 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.593 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.593 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.593 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.593 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.593 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.593 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-04T09:09:00.593 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-04T09:09:00.594 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.594 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.594 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.594 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.594 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.594 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.594 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.594 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.595 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.595 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.595 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.595 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.595 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-04T09:09:00.595 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-04T09:09:00.595 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.595 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.595 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.596 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.596 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.596 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.596 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.596 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.596 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.596 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.596 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.597 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.597 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-04T09:09:00.597 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-04T09:09:00.597 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.597 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.597 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.597 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.597 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.598 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.598 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.598 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.598 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.598 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.598 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.598 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.598 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-04T09:09:00.598 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-04T09:09:00.599 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.599 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.599 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.599 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.599 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.599 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.599 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.599 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.600 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.600 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.600 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.600 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.600 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-04T09:09:00.600 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-04T09:09:00.600 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.600 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.600 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.601 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.601 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.601 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.601 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.601 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.601 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.601 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.601 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.602 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.602 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_ENABLED": { 2024-03-04T09:09:00.602 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-04T09:09:00.602 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.602 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.602 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.602 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.602 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.602 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.603 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.603 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.603 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.603 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.603 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.603 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.603 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-04T09:09:00.603 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-04T09:09:00.604 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.604 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.604 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.604 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-04T09:09:00.604 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.604 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.604 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.604 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.604 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.605 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.605 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.605 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.605 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-04T09:09:00.605 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-04T09:09:00.605 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.605 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.606 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.606 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-03-04T09:09:00.606 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.606 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.606 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.606 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.606 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.606 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.606 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.607 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.607 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-04T09:09:00.607 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-04T09:09:00.607 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.607 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.607 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.607 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8", 2024-03-04T09:09:00.608 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.608 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.608 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.608 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.608 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.608 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.608 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.608 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.608 INFO:teuthology.orchestra.run.smithi071.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-04T09:09:00.609 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-04T09:09:00.609 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.609 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.609 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.609 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "45", 2024-03-04T09:09:00.609 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.609 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.609 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.610 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.610 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.610 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.610 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.610 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.610 INFO:teuthology.orchestra.run.smithi071.stdout: "RGW_API_ACCESS_KEY": { 2024-03-04T09:09:00.610 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-04T09:09:00.610 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.610 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.611 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.611 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.611 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.611 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.611 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.611 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.611 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.611 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.612 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.612 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.612 INFO:teuthology.orchestra.run.smithi071.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-04T09:09:00.612 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-04T09:09:00.612 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.612 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.612 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.612 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "admin", 2024-03-04T09:09:00.613 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.613 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.613 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.613 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.613 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.613 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.613 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.613 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.614 INFO:teuthology.orchestra.run.smithi071.stdout: "RGW_API_SECRET_KEY": { 2024-03-04T09:09:00.614 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-04T09:09:00.614 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.614 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.614 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.614 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.614 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.614 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.615 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.615 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.615 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.615 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.615 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.615 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.615 INFO:teuthology.orchestra.run.smithi071.stdout: "RGW_API_SSL_VERIFY": { 2024-03-04T09:09:00.615 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-04T09:09:00.615 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.616 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.616 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.616 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.616 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.616 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.616 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.616 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.616 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.617 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.617 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.617 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.617 INFO:teuthology.orchestra.run.smithi071.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-04T09:09:00.617 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-04T09:09:00.617 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.617 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.617 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.617 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-03-04T09:09:00.618 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.618 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.618 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.618 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.618 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.618 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.618 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.618 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.619 INFO:teuthology.orchestra.run.smithi071.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-04T09:09:00.619 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-04T09:09:00.619 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.619 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.619 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.619 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-03-04T09:09:00.619 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.619 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.619 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.620 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.620 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.620 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.620 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.620 INFO:teuthology.orchestra.run.smithi071.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-04T09:09:00.620 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-04T09:09:00.620 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.621 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.621 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.621 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5", 2024-03-04T09:09:00.621 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.621 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.621 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.621 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.621 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.621 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.622 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.622 INFO:teuthology.orchestra.run.smithi071.stdout: "cross_origin_url": { 2024-03-04T09:09:00.622 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cross_origin_url", 2024-03-04T09:09:00.622 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.622 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.622 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.622 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.623 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.623 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.623 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.623 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.623 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.623 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.623 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.623 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.624 INFO:teuthology.orchestra.run.smithi071.stdout: "crt_file": { 2024-03-04T09:09:00.624 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crt_file", 2024-03-04T09:09:00.624 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.624 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.624 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.624 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.624 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.624 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.624 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.625 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.625 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.625 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.625 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.625 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.625 INFO:teuthology.orchestra.run.smithi071.stdout: "debug": { 2024-03-04T09:09:00.625 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "debug", 2024-03-04T09:09:00.625 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.626 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.626 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.626 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.626 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.626 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.626 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.626 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Enable/disable debug options", 2024-03-04T09:09:00.626 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.627 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.627 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.627 INFO:teuthology.orchestra.run.smithi071.stdout: "jwt_token_ttl": { 2024-03-04T09:09:00.627 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "jwt_token_ttl", 2024-03-04T09:09:00.627 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.627 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.627 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.627 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "28800", 2024-03-04T09:09:00.628 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.628 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.628 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.628 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.628 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.628 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.628 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.628 INFO:teuthology.orchestra.run.smithi071.stdout: "key_file": { 2024-03-04T09:09:00.629 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "key_file", 2024-03-04T09:09:00.629 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.629 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.629 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.629 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.629 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.629 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.630 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.630 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.630 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.630 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.630 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.630 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.630 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.630 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.630 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.631 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.631 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.631 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.631 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.631 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.631 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.631 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.631 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.631 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.632 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.632 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.632 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.632 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.632 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.632 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.632 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.632 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.633 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.633 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.633 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.633 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.633 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.633 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.633 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.633 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.634 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.634 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.634 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.634 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.634 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.634 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.634 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.635 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.635 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.635 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.635 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.635 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.635 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.635 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.635 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.635 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.636 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.636 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.636 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.636 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.636 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.636 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.636 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.636 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.637 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.637 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.637 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.637 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.637 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.637 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.637 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.638 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.638 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.638 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.638 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.638 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.638 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.638 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.638 INFO:teuthology.orchestra.run.smithi071.stdout: "motd": { 2024-03-04T09:09:00.639 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "motd", 2024-03-04T09:09:00.639 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.639 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.639 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.639 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.639 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.639 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.639 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.640 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "The message of the day", 2024-03-04T09:09:00.640 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.640 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.640 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.640 INFO:teuthology.orchestra.run.smithi071.stdout: "redirect_resolve_ip_addr": { 2024-03-04T09:09:00.640 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "redirect_resolve_ip_addr", 2024-03-04T09:09:00.640 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.640 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.641 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.641 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.641 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.641 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.641 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.641 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.641 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.641 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.642 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.642 INFO:teuthology.orchestra.run.smithi071.stdout: "server_addr": { 2024-03-04T09:09:00.642 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_addr", 2024-03-04T09:09:00.642 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.642 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.642 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.642 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "::", 2024-03-04T09:09:00.643 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.643 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.643 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.643 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.643 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.643 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.643 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.643 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.643 INFO:teuthology.orchestra.run.smithi071.stdout: "server_port": { 2024-03-04T09:09:00.644 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_port", 2024-03-04T09:09:00.644 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.644 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.644 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.644 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8080", 2024-03-04T09:09:00.644 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.644 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.644 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.645 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.645 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.645 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.645 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.645 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.645 INFO:teuthology.orchestra.run.smithi071.stdout: "ssl": { 2024-03-04T09:09:00.645 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ssl", 2024-03-04T09:09:00.645 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.645 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.646 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.646 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.646 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.646 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.646 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.646 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.646 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.646 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.647 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.647 INFO:teuthology.orchestra.run.smithi071.stdout: "ssl_server_port": { 2024-03-04T09:09:00.647 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ssl_server_port", 2024-03-04T09:09:00.647 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.647 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.647 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.647 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8443", 2024-03-04T09:09:00.647 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.648 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.648 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.648 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.648 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.648 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.648 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.648 INFO:teuthology.orchestra.run.smithi071.stdout: "standby_behaviour": { 2024-03-04T09:09:00.649 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "standby_behaviour", 2024-03-04T09:09:00.649 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.649 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.649 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.649 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "redirect", 2024-03-04T09:09:00.649 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.649 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.649 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.649 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.650 INFO:teuthology.orchestra.run.smithi071.stdout: "redirect" 2024-03-04T09:09:00.650 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.650 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.650 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.650 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.650 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.650 INFO:teuthology.orchestra.run.smithi071.stdout: "standby_error_status_code": { 2024-03-04T09:09:00.650 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "standby_error_status_code", 2024-03-04T09:09:00.651 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.651 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.651 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.651 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "500", 2024-03-04T09:09:00.651 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "400", 2024-03-04T09:09:00.651 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "599", 2024-03-04T09:09:00.651 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.651 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.651 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.652 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.652 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.652 INFO:teuthology.orchestra.run.smithi071.stdout: "url_prefix": { 2024-03-04T09:09:00.652 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "url_prefix", 2024-03-04T09:09:00.652 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.652 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.653 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.653 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.653 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.653 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.653 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.653 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.653 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.653 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.654 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.654 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.654 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.654 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.654 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.654 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "devicehealth", 2024-03-04T09:09:00.654 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.654 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.654 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.655 INFO:teuthology.orchestra.run.smithi071.stdout: "enable_monitoring": { 2024-03-04T09:09:00.655 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "enable_monitoring", 2024-03-04T09:09:00.655 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.655 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.655 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.655 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.655 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.655 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.655 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.656 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "monitor device health metrics", 2024-03-04T09:09:00.656 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.656 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.656 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.656 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.656 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.657 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.657 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.657 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.657 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.657 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.657 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.657 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.657 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.657 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.658 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.658 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.658 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.658 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.658 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.658 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.658 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.658 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.659 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.659 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.659 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.659 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.659 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.659 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.659 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.659 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.660 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.660 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.660 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.660 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.660 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.660 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.660 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.660 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.661 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.661 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.661 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.661 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.661 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.661 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.661 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.661 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.662 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.662 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.662 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.662 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.662 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.662 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.662 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.662 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.662 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.663 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.663 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.663 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.663 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.663 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.663 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.663 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.664 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.664 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.664 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.664 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.664 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.664 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.664 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.664 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.665 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.665 INFO:teuthology.orchestra.run.smithi071.stdout: "mark_out_threshold": { 2024-03-04T09:09:00.665 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mark_out_threshold", 2024-03-04T09:09:00.665 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:00.665 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.665 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.665 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "2419200", 2024-03-04T09:09:00.665 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.666 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.666 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.666 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-04T09:09:00.666 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.666 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.666 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.666 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": { 2024-03-04T09:09:00.666 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "pool_name", 2024-03-04T09:09:00.667 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.667 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.667 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.667 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "device_health_metrics", 2024-03-04T09:09:00.667 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.667 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.667 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.667 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-04T09:09:00.668 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.668 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.668 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.668 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.668 INFO:teuthology.orchestra.run.smithi071.stdout: "retention_period": { 2024-03-04T09:09:00.668 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "retention_period", 2024-03-04T09:09:00.668 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:00.668 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.668 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.669 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "15552000", 2024-03-04T09:09:00.669 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.669 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.669 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.669 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how long to retain device health metrics", 2024-03-04T09:09:00.669 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.669 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.669 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.670 INFO:teuthology.orchestra.run.smithi071.stdout: "scrape_frequency": { 2024-03-04T09:09:00.670 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "scrape_frequency", 2024-03-04T09:09:00.670 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:00.670 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.670 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.670 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "86400", 2024-03-04T09:09:00.670 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.670 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.671 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.671 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-04T09:09:00.671 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.671 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.671 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.671 INFO:teuthology.orchestra.run.smithi071.stdout: "self_heal": { 2024-03-04T09:09:00.671 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "self_heal", 2024-03-04T09:09:00.672 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.672 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.672 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.672 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.672 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.672 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.672 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.672 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-04T09:09:00.672 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.673 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.673 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.673 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-03-04T09:09:00.673 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-03-04T09:09:00.673 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:00.673 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.673 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.673 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-03-04T09:09:00.674 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.674 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.674 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.674 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to wake up and check device health", 2024-03-04T09:09:00.674 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.674 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.674 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.675 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_threshold": { 2024-03-04T09:09:00.675 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_threshold", 2024-03-04T09:09:00.675 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:00.675 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.675 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.675 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "7257600", 2024-03-04T09:09:00.675 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.675 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.676 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.676 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-04T09:09:00.676 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.676 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.676 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.676 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.676 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.676 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.677 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "diskprediction_local", 2024-03-04T09:09:00.677 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.677 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.677 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.677 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.677 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.677 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.677 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.678 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.678 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.678 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.678 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.678 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.678 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.678 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.678 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.678 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.679 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.679 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.679 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.679 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.679 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.679 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.679 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.679 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.680 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.680 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.680 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.680 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.680 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.680 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.680 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.680 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.680 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.681 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.681 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.681 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.681 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.681 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.681 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.681 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.682 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.682 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.682 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.682 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.682 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.682 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.682 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.682 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.682 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.683 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.683 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.683 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.683 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.683 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.683 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.683 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.683 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.683 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.684 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.684 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.684 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.684 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.684 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.684 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.684 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.684 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.685 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.685 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.685 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.685 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.685 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.685 INFO:teuthology.orchestra.run.smithi071.stdout: "predict_interval": { 2024-03-04T09:09:00.685 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "predict_interval", 2024-03-04T09:09:00.685 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.685 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.686 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.686 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "86400", 2024-03-04T09:09:00.686 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.686 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.686 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.686 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.686 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.686 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.686 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.687 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.687 INFO:teuthology.orchestra.run.smithi071.stdout: "predictor_model": { 2024-03-04T09:09:00.687 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "predictor_model", 2024-03-04T09:09:00.687 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.687 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.687 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.687 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "prophetstor", 2024-03-04T09:09:00.687 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.687 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.688 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.688 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.688 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.688 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.688 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.688 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-03-04T09:09:00.688 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-03-04T09:09:00.689 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.689 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.689 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.689 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-03-04T09:09:00.689 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.689 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.689 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.689 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.690 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.690 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.690 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.690 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.690 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.690 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.690 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.690 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "influx", 2024-03-04T09:09:00.690 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": false, 2024-03-04T09:09:00.691 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "influxdb python module not found", 2024-03-04T09:09:00.691 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.691 INFO:teuthology.orchestra.run.smithi071.stdout: "batch_size": { 2024-03-04T09:09:00.691 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "batch_size", 2024-03-04T09:09:00.691 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.691 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.691 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.691 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5000", 2024-03-04T09:09:00.692 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.692 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.692 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.692 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-04T09:09:00.692 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.692 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.692 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.692 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.692 INFO:teuthology.orchestra.run.smithi071.stdout: "database": { 2024-03-04T09:09:00.693 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "database", 2024-03-04T09:09:00.693 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.693 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.693 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.693 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "ceph", 2024-03-04T09:09:00.693 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.693 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.693 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.694 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-04T09:09:00.694 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.694 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.694 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.694 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.694 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": { 2024-03-04T09:09:00.694 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "hostname", 2024-03-04T09:09:00.694 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.695 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.695 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.695 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.695 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.695 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.695 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.695 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "InfluxDB server hostname", 2024-03-04T09:09:00.695 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.695 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.696 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.696 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.696 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-03-04T09:09:00.696 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-03-04T09:09:00.696 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:00.696 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.696 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.697 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "30", 2024-03-04T09:09:00.697 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "5", 2024-03-04T09:09:00.697 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.697 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.697 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-04T09:09:00.697 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.697 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.697 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.698 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.698 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.698 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.698 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.698 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.698 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.698 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.698 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.698 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.699 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.699 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.699 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.699 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.699 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.699 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.699 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.699 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.700 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.700 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.700 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.700 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.700 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.700 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.700 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.700 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.700 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.701 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.701 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.701 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.701 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.701 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.701 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.701 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.701 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.702 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.702 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.702 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.702 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.702 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.702 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.702 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.702 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.703 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.703 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.703 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.703 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.703 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.703 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.703 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.703 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.703 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.704 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.704 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.704 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.704 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.704 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.704 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.704 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.704 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.704 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.705 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.705 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.705 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.705 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.705 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.705 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.705 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.705 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.706 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.706 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.706 INFO:teuthology.orchestra.run.smithi071.stdout: "password": { 2024-03-04T09:09:00.706 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "password", 2024-03-04T09:09:00.706 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.706 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.706 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.706 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.707 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.707 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.707 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.707 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "password of InfluxDB server user", 2024-03-04T09:09:00.707 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.707 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.707 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.708 INFO:teuthology.orchestra.run.smithi071.stdout: "port": { 2024-03-04T09:09:00.708 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "port", 2024-03-04T09:09:00.708 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.708 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.708 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.708 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8086", 2024-03-04T09:09:00.708 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.708 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.708 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.709 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "InfluxDB server port", 2024-03-04T09:09:00.709 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.709 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.709 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.709 INFO:teuthology.orchestra.run.smithi071.stdout: "ssl": { 2024-03-04T09:09:00.709 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ssl", 2024-03-04T09:09:00.709 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.710 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.710 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.710 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "false", 2024-03-04T09:09:00.710 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.710 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.710 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.710 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-04T09:09:00.710 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.710 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.711 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.711 INFO:teuthology.orchestra.run.smithi071.stdout: "threads": { 2024-03-04T09:09:00.711 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "threads", 2024-03-04T09:09:00.711 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.711 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.711 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.711 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5", 2024-03-04T09:09:00.711 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1", 2024-03-04T09:09:00.712 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "32", 2024-03-04T09:09:00.712 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.712 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-04T09:09:00.712 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.712 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.712 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.712 INFO:teuthology.orchestra.run.smithi071.stdout: "username": { 2024-03-04T09:09:00.713 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "username", 2024-03-04T09:09:00.713 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.713 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.713 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.713 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.713 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.713 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.713 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.714 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "username of InfluxDB server user", 2024-03-04T09:09:00.714 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.714 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.714 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.714 INFO:teuthology.orchestra.run.smithi071.stdout: "verify_ssl": { 2024-03-04T09:09:00.714 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "verify_ssl", 2024-03-04T09:09:00.714 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.714 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.715 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.715 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "true", 2024-03-04T09:09:00.715 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.715 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.715 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.715 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-04T09:09:00.715 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.715 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.716 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.716 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.716 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.716 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.716 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "insights", 2024-03-04T09:09:00.716 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.716 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.716 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.717 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.717 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.717 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.717 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.717 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.717 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.717 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.717 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.718 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.719 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.719 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.719 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.719 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.720 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.720 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.720 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.720 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.720 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.720 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.720 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.721 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.721 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.721 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.721 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.721 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.721 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.721 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.722 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.722 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.722 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.722 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.722 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.722 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.722 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.722 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.723 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.723 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.723 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.723 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.723 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.723 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.723 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.723 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.724 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.724 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.724 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.724 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.724 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.724 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.724 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.724 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.725 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.725 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.725 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.725 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.725 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.725 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.725 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.725 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.725 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.726 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.726 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.726 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.726 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.726 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.726 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.726 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.726 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.726 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.727 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "iostat", 2024-03-04T09:09:00.727 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.727 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.727 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.727 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.728 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.728 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.728 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.728 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.728 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.728 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.728 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.728 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.729 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.729 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.729 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.729 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.729 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.729 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.729 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.729 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.729 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.730 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.730 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.730 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.730 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.730 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.730 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.730 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.730 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.731 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.731 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.731 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.731 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.731 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.731 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.731 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.732 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.732 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.732 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.732 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.732 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.732 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.732 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.732 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.732 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.733 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.733 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.733 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.733 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.733 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.733 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.733 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.733 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.734 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.734 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.734 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.734 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.734 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.734 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.734 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.734 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.735 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.735 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.735 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.735 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.735 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.735 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.735 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.735 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.736 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.736 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.736 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "k8sevents", 2024-03-04T09:09:00.736 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.736 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.736 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.736 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_event_retention_days": { 2024-03-04T09:09:00.737 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ceph_event_retention_days", 2024-03-04T09:09:00.737 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.737 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.737 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.737 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "7", 2024-03-04T09:09:00.737 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.737 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.737 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.738 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-04T09:09:00.738 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.738 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.738 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.738 INFO:teuthology.orchestra.run.smithi071.stdout: "config_check_secs": { 2024-03-04T09:09:00.738 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "config_check_secs", 2024-03-04T09:09:00.738 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.738 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.739 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.739 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-03-04T09:09:00.739 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "10", 2024-03-04T09:09:00.739 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.739 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.739 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-04T09:09:00.739 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.739 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.740 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.740 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.740 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.740 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.740 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.740 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.740 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.740 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.741 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.741 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.741 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.741 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.741 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.741 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.741 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.741 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.742 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.742 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.742 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.742 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.742 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.742 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.742 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.743 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.743 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.743 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.743 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.743 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.743 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.743 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.743 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.744 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.744 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.744 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.744 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.744 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.744 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.744 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.745 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.745 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.745 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.745 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.745 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.745 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.745 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.745 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.745 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.746 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.746 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.746 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.746 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.746 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.746 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.746 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.747 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.747 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.747 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.747 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.747 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.747 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.747 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.747 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.747 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.748 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.748 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.748 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.748 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.748 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.748 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.748 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.749 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "localpool", 2024-03-04T09:09:00.749 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.749 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.749 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.749 INFO:teuthology.orchestra.run.smithi071.stdout: "failure_domain": { 2024-03-04T09:09:00.749 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "failure_domain", 2024-03-04T09:09:00.749 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.749 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.749 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.750 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "host", 2024-03-04T09:09:00.750 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.750 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.750 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.750 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "failure domain for any created local pool", 2024-03-04T09:09:00.750 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-04T09:09:00.750 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.750 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.751 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.751 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.751 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.751 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.751 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.751 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.751 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.751 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.752 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.752 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.752 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.752 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.752 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.752 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.752 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.752 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.752 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.753 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.753 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.753 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.753 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.753 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.753 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.753 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.754 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.754 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.754 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.754 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.754 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.754 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.754 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.754 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.754 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.755 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.755 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.755 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.755 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.755 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.755 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.756 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.756 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.756 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.756 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.756 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.756 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.756 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.756 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.756 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.757 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.757 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.757 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.757 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.757 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.757 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.757 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.758 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.758 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.758 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.758 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.758 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.758 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.758 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.758 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.758 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.759 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.759 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.759 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": { 2024-03-04T09:09:00.759 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "min_size", 2024-03-04T09:09:00.759 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.759 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.759 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.759 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.760 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.760 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.760 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.760 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "default min_size for any created local pool", 2024-03-04T09:09:00.760 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-04T09:09:00.760 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.760 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.760 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.761 INFO:teuthology.orchestra.run.smithi071.stdout: "num_rep": { 2024-03-04T09:09:00.761 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "num_rep", 2024-03-04T09:09:00.761 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.761 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.761 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.761 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "3", 2024-03-04T09:09:00.761 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.761 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.761 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.762 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "default replica count for any created local pool", 2024-03-04T09:09:00.762 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.762 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.762 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.762 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": { 2024-03-04T09:09:00.762 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "pg_num", 2024-03-04T09:09:00.762 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.763 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.763 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.763 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "128", 2024-03-04T09:09:00.763 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.763 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.763 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.763 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "default pg_num for any created local pool", 2024-03-04T09:09:00.763 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.763 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.764 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.764 INFO:teuthology.orchestra.run.smithi071.stdout: "prefix": { 2024-03-04T09:09:00.764 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "prefix", 2024-03-04T09:09:00.764 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.764 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.764 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.764 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.765 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.765 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.765 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.765 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "name prefix for any created local pool", 2024-03-04T09:09:00.765 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.765 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.765 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.766 INFO:teuthology.orchestra.run.smithi071.stdout: "subtree": { 2024-03-04T09:09:00.766 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subtree", 2024-03-04T09:09:00.766 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.766 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.766 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.766 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "rack", 2024-03-04T09:09:00.766 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.766 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.767 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.767 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-04T09:09:00.767 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-04T09:09:00.767 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.767 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.767 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.767 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.768 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mds_autoscaler", 2024-03-04T09:09:00.768 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.768 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.768 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.768 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.768 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.768 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.769 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.769 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.769 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.769 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.769 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.769 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.769 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.769 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.769 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.770 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.770 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.770 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.770 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.770 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.770 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.770 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.770 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.771 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.771 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.771 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.771 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.771 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.771 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.771 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.772 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.772 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.772 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.772 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.772 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.772 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.772 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.772 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.773 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.773 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.773 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.773 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.773 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.773 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.773 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.773 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.774 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.774 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.774 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.774 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.774 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.774 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.774 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.774 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.774 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.775 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.775 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.775 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.775 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.775 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.775 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.775 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.776 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.776 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.776 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.776 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.776 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.776 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.776 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.776 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.776 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.777 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mirroring", 2024-03-04T09:09:00.777 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.777 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.777 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.777 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.777 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.778 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.778 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.778 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.778 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.778 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.778 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.778 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.778 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.778 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.779 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.779 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.779 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.779 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.779 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.779 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.779 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.779 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.780 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.780 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.780 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.780 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.780 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.780 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.780 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.781 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.781 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.781 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.781 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.781 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.781 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.781 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.781 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.782 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.782 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.782 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.782 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.782 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.782 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.782 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.782 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.783 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.783 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.783 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.783 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.783 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.783 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.783 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.783 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.783 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.784 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.784 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.784 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.784 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.784 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.784 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.784 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.785 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.785 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.785 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.785 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.785 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.785 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.785 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.785 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.786 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.786 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.786 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "nfs", 2024-03-04T09:09:00.786 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.786 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.786 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.787 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.787 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.787 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.787 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.787 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.787 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.787 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.787 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.787 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.788 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.788 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.788 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.788 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.788 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.788 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.788 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.788 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.789 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.789 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.789 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.789 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.789 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.789 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.789 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.790 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.790 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.790 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.790 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.790 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.790 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.790 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.790 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.791 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.791 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.791 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.791 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.791 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.791 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.791 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.791 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.792 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.792 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.792 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.792 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.792 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.792 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.792 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.792 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.793 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.793 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.793 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.793 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.793 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.793 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.793 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.793 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.794 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.794 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.794 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.794 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.794 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.794 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.794 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.794 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.795 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.795 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.795 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.795 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.795 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "orchestrator", 2024-03-04T09:09:00.795 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.795 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.796 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.796 INFO:teuthology.orchestra.run.smithi071.stdout: "fail_fs": { 2024-03-04T09:09:00.796 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "fail_fs", 2024-03-04T09:09:00.796 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.796 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.796 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.796 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.796 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.797 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.797 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.797 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-04T09:09:00.797 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.797 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.797 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.797 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.797 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.798 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.798 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.798 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.798 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.798 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.798 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.798 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.798 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.799 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.799 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.799 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.799 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.799 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.799 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.799 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.799 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.800 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.800 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.800 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.800 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.800 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.800 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.800 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.801 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.801 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.801 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.801 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.801 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.801 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.801 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.801 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.802 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.802 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.802 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.802 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.802 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.802 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.802 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.802 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.803 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.803 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.803 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.803 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.803 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.803 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.803 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.803 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.803 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.804 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.804 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.804 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.804 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.804 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.804 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.804 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.805 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.805 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.805 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.805 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.805 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.805 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.805 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.805 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.805 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.806 INFO:teuthology.orchestra.run.smithi071.stdout: "orchestrator": { 2024-03-04T09:09:00.806 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "orchestrator", 2024-03-04T09:09:00.806 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.806 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.806 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.806 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.806 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.806 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.807 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.807 INFO:teuthology.orchestra.run.smithi071.stdout: "cephadm", 2024-03-04T09:09:00.807 INFO:teuthology.orchestra.run.smithi071.stdout: "rook", 2024-03-04T09:09:00.807 INFO:teuthology.orchestra.run.smithi071.stdout: "test_orchestrator" 2024-03-04T09:09:00.807 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.807 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Orchestrator backend", 2024-03-04T09:09:00.807 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.807 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.808 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.808 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.808 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.808 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "osd_perf_query", 2024-03-04T09:09:00.808 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.808 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.808 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.809 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.809 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.809 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.809 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.809 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.809 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.809 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.809 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.810 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.810 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.810 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.810 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.810 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.810 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.810 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.810 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.810 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.811 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.811 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.811 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.811 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.811 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.811 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.811 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.811 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.812 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.812 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.812 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.812 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.812 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.812 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.812 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.813 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.813 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.813 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.813 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.813 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.813 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.813 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.813 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.814 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.814 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.814 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.814 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.814 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.814 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.814 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.814 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.815 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.815 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.815 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.815 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.815 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.815 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.815 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.815 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.816 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.816 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.816 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.816 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.816 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.816 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.816 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.816 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.817 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.817 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.817 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.817 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.817 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "osd_support", 2024-03-04T09:09:00.817 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.817 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.818 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.818 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.818 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.818 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.818 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.818 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.818 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.818 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.819 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.819 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.819 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.819 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.819 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.819 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.819 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.819 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.819 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.820 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.820 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.820 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.820 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.820 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.820 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.820 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.821 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.821 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.821 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.821 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.821 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.821 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.821 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.821 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.822 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.822 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.822 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.822 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.822 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.822 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.822 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.822 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.823 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.823 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.823 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.823 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.823 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.823 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.823 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.823 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.823 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.824 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.824 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.824 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.824 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.824 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.824 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.824 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.825 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.825 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.825 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.825 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.825 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.825 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.825 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.825 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.825 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.826 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.826 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.826 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.826 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.826 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "pg_autoscaler", 2024-03-04T09:09:00.826 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.827 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.827 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.827 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.827 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.827 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.827 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.827 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.827 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.827 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.828 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.828 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.828 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.828 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.828 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.828 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.828 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.829 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.829 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.829 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.829 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.829 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.829 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.829 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.830 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.830 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.830 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.830 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.830 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.830 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.830 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.830 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.830 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.831 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.831 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.831 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.831 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.831 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.831 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.831 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.831 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.832 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.832 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.832 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.832 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.832 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.832 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.832 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.832 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.833 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.833 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.833 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.833 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.833 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.833 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.833 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.834 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.834 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.834 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.834 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.834 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.834 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.834 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.834 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.834 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.835 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.835 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.835 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.835 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.835 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-03-04T09:09:00.835 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-03-04T09:09:00.836 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:00.836 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.836 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.836 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-03-04T09:09:00.836 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.836 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.836 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.836 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.836 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.837 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.837 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.837 INFO:teuthology.orchestra.run.smithi071.stdout: "threshold": { 2024-03-04T09:09:00.837 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "threshold", 2024-03-04T09:09:00.837 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-03-04T09:09:00.837 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.837 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.838 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "3.0", 2024-03-04T09:09:00.838 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1.0", 2024-03-04T09:09:00.838 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.838 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.838 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "scaling threshold", 2024-03-04T09:09:00.838 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-04T09:09:00.838 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.838 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.839 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.839 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.839 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "progress", 2024-03-04T09:09:00.839 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.839 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.839 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.840 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_pg_recovery_event": { 2024-03-04T09:09:00.840 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "allow_pg_recovery_event", 2024-03-04T09:09:00.840 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.840 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.840 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.840 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.840 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.840 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.841 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.841 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-04T09:09:00.841 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.841 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.841 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.841 INFO:teuthology.orchestra.run.smithi071.stdout: "enabled": { 2024-03-04T09:09:00.841 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "enabled", 2024-03-04T09:09:00.841 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.842 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.842 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.842 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.842 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.842 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.842 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.842 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.842 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.843 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.843 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.843 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.843 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.843 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.843 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.843 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.843 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.844 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.844 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.844 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.844 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.844 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.844 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.844 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.844 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.845 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.845 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.845 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.845 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.845 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.845 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.845 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.845 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.846 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.846 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.846 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.846 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.846 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.846 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.846 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.846 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.847 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.847 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.847 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.847 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.847 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.847 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.847 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.847 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.848 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.848 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.848 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.848 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.848 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.848 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.848 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.848 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.848 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.849 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.849 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.849 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.849 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.849 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.849 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.849 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.850 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.850 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.850 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.850 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.850 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.850 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.850 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.850 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.850 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.851 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.851 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.851 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.851 INFO:teuthology.orchestra.run.smithi071.stdout: "max_completed_events": { 2024-03-04T09:09:00.851 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_completed_events", 2024-03-04T09:09:00.851 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.851 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.852 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.852 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "50", 2024-03-04T09:09:00.852 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.852 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.852 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.852 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "number of past completed events to remember", 2024-03-04T09:09:00.852 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.852 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.853 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.853 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-03-04T09:09:00.853 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-03-04T09:09:00.853 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:00.853 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.853 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.853 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5", 2024-03-04T09:09:00.853 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.854 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.854 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.854 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how long the module is going to sleep", 2024-03-04T09:09:00.854 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.854 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.854 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.854 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.854 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.855 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "prometheus", 2024-03-04T09:09:00.855 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.855 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.855 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.855 INFO:teuthology.orchestra.run.smithi071.stdout: "cache": { 2024-03-04T09:09:00.855 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cache", 2024-03-04T09:09:00.855 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.856 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.856 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.856 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.856 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.856 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.856 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.856 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.856 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.857 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.857 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.857 INFO:teuthology.orchestra.run.smithi071.stdout: "exclude_perf_counters": { 2024-03-04T09:09:00.857 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "exclude_perf_counters", 2024-03-04T09:09:00.857 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.857 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.857 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.857 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.858 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.858 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.858 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.858 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-04T09:09:00.858 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-04T09:09:00.858 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.858 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.859 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.859 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.859 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.859 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.859 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.859 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.859 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.859 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.860 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.860 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.860 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.860 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.860 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.860 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.860 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.861 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.861 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.861 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.861 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.861 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.861 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.861 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.862 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.862 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.862 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.862 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.862 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.862 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.862 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.862 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.863 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.863 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.863 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.863 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.863 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.863 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.863 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.864 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.864 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.864 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.864 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.864 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.864 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.864 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.864 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.864 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.865 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.865 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.865 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.865 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.865 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.865 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.865 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.866 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.866 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.866 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.866 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.866 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.866 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.866 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.867 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.867 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.867 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.867 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.867 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.867 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.867 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_stats_pools": { 2024-03-04T09:09:00.868 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rbd_stats_pools", 2024-03-04T09:09:00.868 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.868 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.868 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.868 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.868 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.868 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.868 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.868 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.869 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.869 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.869 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.869 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-04T09:09:00.869 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-04T09:09:00.869 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.869 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.870 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.870 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "300", 2024-03-04T09:09:00.870 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.870 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.870 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.870 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.870 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.870 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.870 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.871 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.871 INFO:teuthology.orchestra.run.smithi071.stdout: "scrape_interval": { 2024-03-04T09:09:00.871 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "scrape_interval", 2024-03-04T09:09:00.871 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-03-04T09:09:00.871 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.871 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.871 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "15.0", 2024-03-04T09:09:00.871 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.872 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.872 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.872 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.872 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.872 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.872 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.872 INFO:teuthology.orchestra.run.smithi071.stdout: "server_addr": { 2024-03-04T09:09:00.872 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_addr", 2024-03-04T09:09:00.873 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.873 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.873 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.873 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "::", 2024-03-04T09:09:00.873 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.873 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.873 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.873 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-04T09:09:00.874 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.874 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.874 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.874 INFO:teuthology.orchestra.run.smithi071.stdout: "server_port": { 2024-03-04T09:09:00.874 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_port", 2024-03-04T09:09:00.874 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.874 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.874 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.875 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "9283", 2024-03-04T09:09:00.875 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.875 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.875 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.875 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-04T09:09:00.875 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.875 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.875 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.876 INFO:teuthology.orchestra.run.smithi071.stdout: "stale_cache_strategy": { 2024-03-04T09:09:00.876 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "stale_cache_strategy", 2024-03-04T09:09:00.876 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.876 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.876 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.876 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "log", 2024-03-04T09:09:00.876 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.876 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.877 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.877 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.877 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.877 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.877 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.877 INFO:teuthology.orchestra.run.smithi071.stdout: "standby_behaviour": { 2024-03-04T09:09:00.877 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "standby_behaviour", 2024-03-04T09:09:00.877 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.878 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.878 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.878 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "default", 2024-03-04T09:09:00.878 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.878 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.878 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.878 INFO:teuthology.orchestra.run.smithi071.stdout: "default", 2024-03-04T09:09:00.878 INFO:teuthology.orchestra.run.smithi071.stdout: "error" 2024-03-04T09:09:00.878 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.879 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.879 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.879 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.879 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.879 INFO:teuthology.orchestra.run.smithi071.stdout: "standby_error_status_code": { 2024-03-04T09:09:00.879 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "standby_error_status_code", 2024-03-04T09:09:00.879 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.880 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.880 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.880 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "500", 2024-03-04T09:09:00.880 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "400", 2024-03-04T09:09:00.880 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "599", 2024-03-04T09:09:00.880 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.880 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.880 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.881 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.881 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.881 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.881 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.881 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rbd_support", 2024-03-04T09:09:00.881 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.882 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.882 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.882 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.882 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.882 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.882 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.882 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.882 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.882 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.883 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.883 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.883 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.883 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.883 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.883 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.883 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.883 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.883 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.884 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.884 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.884 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.884 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.884 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.884 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.884 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.885 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.885 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.885 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.885 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.885 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.885 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.885 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.885 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.886 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.886 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.886 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.886 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.886 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.886 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.886 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.887 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.887 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.887 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.887 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.887 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.887 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.887 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.887 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.887 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.888 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.888 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.888 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.888 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.888 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.888 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.888 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.889 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.889 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.889 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.889 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.889 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.889 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.889 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.889 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.890 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.890 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.890 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.890 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.890 INFO:teuthology.orchestra.run.smithi071.stdout: "max_concurrent_snap_create": { 2024-03-04T09:09:00.890 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_concurrent_snap_create", 2024-03-04T09:09:00.890 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.890 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.891 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.891 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-03-04T09:09:00.891 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.891 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.891 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.891 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.891 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.891 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.892 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.892 INFO:teuthology.orchestra.run.smithi071.stdout: "mirror_snapshot_schedule": { 2024-03-04T09:09:00.892 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mirror_snapshot_schedule", 2024-03-04T09:09:00.892 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.892 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.892 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.892 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.893 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.893 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.893 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.893 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.893 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.893 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.893 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.893 INFO:teuthology.orchestra.run.smithi071.stdout: "trash_purge_schedule": { 2024-03-04T09:09:00.894 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "trash_purge_schedule", 2024-03-04T09:09:00.894 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.894 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.894 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.894 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.894 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.894 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.894 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.894 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.895 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.895 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.895 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.895 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.895 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.895 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.895 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "restful", 2024-03-04T09:09:00.895 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.896 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.896 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.896 INFO:teuthology.orchestra.run.smithi071.stdout: "enable_auth": { 2024-03-04T09:09:00.896 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "enable_auth", 2024-03-04T09:09:00.896 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.896 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.896 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.896 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.897 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.897 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.897 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.897 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.897 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.897 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.897 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.898 INFO:teuthology.orchestra.run.smithi071.stdout: "key_file": { 2024-03-04T09:09:00.898 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "key_file", 2024-03-04T09:09:00.898 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.898 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.898 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.898 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.898 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.898 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.898 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.899 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.899 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.899 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.899 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.899 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.899 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.899 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.899 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.900 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.900 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.900 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.900 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.900 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.900 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.900 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.900 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.901 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.901 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.901 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.901 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.901 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.901 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.901 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.901 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.902 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.902 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.902 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.902 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.902 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.902 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.902 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.902 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.902 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.903 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.903 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.903 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.903 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.903 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.904 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.904 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.904 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.904 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.904 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.904 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.904 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.904 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.905 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.905 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.905 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.905 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.905 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.905 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.905 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.905 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.905 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.906 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.906 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.906 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.906 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.906 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.906 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.906 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.907 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.907 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.907 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.907 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.907 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.907 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.907 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.907 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.908 INFO:teuthology.orchestra.run.smithi071.stdout: "server_addr": { 2024-03-04T09:09:00.908 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_addr", 2024-03-04T09:09:00.908 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.908 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.908 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.908 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.908 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.908 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.909 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.909 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.909 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.909 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.909 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.909 INFO:teuthology.orchestra.run.smithi071.stdout: "server_port": { 2024-03-04T09:09:00.909 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_port", 2024-03-04T09:09:00.910 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.910 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.910 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.910 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.910 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.910 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.910 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.910 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.911 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.911 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.911 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.911 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.911 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.911 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rgw", 2024-03-04T09:09:00.912 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.912 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.912 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.912 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.912 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.912 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.912 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.912 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.912 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.913 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.913 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.913 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.913 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.913 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.913 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.913 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.913 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.914 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.914 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.914 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.914 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.914 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.914 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.914 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.914 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.915 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.915 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.915 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.915 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.915 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.915 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.915 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.915 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.915 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.916 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.916 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.916 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.916 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.916 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.916 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.916 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.917 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.917 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.917 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.917 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.917 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.917 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.917 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.917 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.917 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.918 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.918 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.918 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.918 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.918 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.918 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.918 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.919 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.919 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.919 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.919 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.919 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.919 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.919 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.919 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.919 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.920 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.920 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.920 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.923 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.923 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.923 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rook", 2024-03-04T09:09:00.923 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.924 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.924 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.924 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.924 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.924 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.924 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.924 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.924 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.925 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.925 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.925 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.925 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.925 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.925 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.925 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.925 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.925 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.926 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.926 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.926 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.926 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.926 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.926 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.926 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.927 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.927 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.927 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.927 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.927 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.927 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.927 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.927 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.927 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.928 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.928 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.928 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.928 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.928 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.928 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.928 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.929 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.929 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.929 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.929 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.929 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.929 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.929 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.930 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.930 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.930 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.930 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.930 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.930 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.930 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.930 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.931 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.931 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.931 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.931 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.931 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.931 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.931 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.931 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.932 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.932 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.932 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.932 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.932 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.932 INFO:teuthology.orchestra.run.smithi071.stdout: "storage_class": { 2024-03-04T09:09:00.932 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "storage_class", 2024-03-04T09:09:00.933 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.933 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.933 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.933 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "local", 2024-03-04T09:09:00.933 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.933 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.933 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.933 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-04T09:09:00.933 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.934 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.934 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.934 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.934 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.934 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "selftest", 2024-03-04T09:09:00.934 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.935 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.935 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.935 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.935 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.935 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.935 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.935 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.935 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.935 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.936 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.936 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.936 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.936 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.936 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.936 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.936 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.936 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.937 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.937 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.937 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.937 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.937 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.937 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.937 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.937 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.938 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.938 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.938 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.938 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.938 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.938 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.938 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.938 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.938 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.939 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.939 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.939 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.939 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.939 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.939 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.939 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.940 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.940 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.940 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.940 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.940 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.940 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.940 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.940 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.940 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.941 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.941 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.941 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.941 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.941 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.941 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.941 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.942 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.942 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.942 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.942 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.942 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.942 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.942 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.942 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.942 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.943 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.943 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.943 INFO:teuthology.orchestra.run.smithi071.stdout: "roption1": { 2024-03-04T09:09:00.943 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "roption1", 2024-03-04T09:09:00.943 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.943 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.943 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.943 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.944 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.944 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.944 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.944 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.944 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.944 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.944 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.945 INFO:teuthology.orchestra.run.smithi071.stdout: "roption2": { 2024-03-04T09:09:00.945 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "roption2", 2024-03-04T09:09:00.945 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.945 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.945 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.945 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "xyz", 2024-03-04T09:09:00.945 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.945 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.946 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.946 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.946 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.946 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.946 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.946 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption1": { 2024-03-04T09:09:00.946 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption1", 2024-03-04T09:09:00.946 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.947 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.947 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.947 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.947 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.947 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.947 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.947 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.947 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.948 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.948 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.948 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption2": { 2024-03-04T09:09:00.948 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption2", 2024-03-04T09:09:00.948 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.948 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.949 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.949 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.949 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.949 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.949 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.949 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.949 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.949 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.949 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.950 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption3": { 2024-03-04T09:09:00.950 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption3", 2024-03-04T09:09:00.950 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-03-04T09:09:00.950 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.950 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.950 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.950 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.951 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.951 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.951 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.951 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.951 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.951 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.951 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption4": { 2024-03-04T09:09:00.951 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption4", 2024-03-04T09:09:00.952 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.952 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.952 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.952 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.952 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.952 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.952 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.952 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.953 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.953 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.953 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.953 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption5": { 2024-03-04T09:09:00.953 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption5", 2024-03-04T09:09:00.953 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.954 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.954 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.954 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.954 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.954 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.954 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.954 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.954 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.954 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.955 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.955 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption6": { 2024-03-04T09:09:00.955 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption6", 2024-03-04T09:09:00.955 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.955 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.955 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.955 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:00.956 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.956 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.956 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.956 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.956 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.956 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.956 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.956 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption7": { 2024-03-04T09:09:00.957 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption7", 2024-03-04T09:09:00.957 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:00.957 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.957 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.957 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.957 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1", 2024-03-04T09:09:00.957 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "42", 2024-03-04T09:09:00.957 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.957 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.958 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.958 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.958 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.958 INFO:teuthology.orchestra.run.smithi071.stdout: "testkey": { 2024-03-04T09:09:00.958 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "testkey", 2024-03-04T09:09:00.958 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.958 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.959 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.959 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.959 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.959 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.959 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.959 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.959 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.959 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.959 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.960 INFO:teuthology.orchestra.run.smithi071.stdout: "testlkey": { 2024-03-04T09:09:00.960 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "testlkey", 2024-03-04T09:09:00.960 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.960 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.960 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.960 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.960 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.960 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.961 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.961 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.961 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.961 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.961 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.961 INFO:teuthology.orchestra.run.smithi071.stdout: "testnewline": { 2024-03-04T09:09:00.961 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "testnewline", 2024-03-04T09:09:00.962 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.962 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.962 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.962 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.962 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.962 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.962 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.962 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.962 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.963 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.963 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.963 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.963 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.963 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "snap_schedule", 2024-03-04T09:09:00.963 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.963 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.964 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.964 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_m_granularity": { 2024-03-04T09:09:00.964 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "allow_m_granularity", 2024-03-04T09:09:00.964 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.964 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.964 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.964 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.964 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.965 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.965 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.965 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "allow minute scheduled snapshots", 2024-03-04T09:09:00.965 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.965 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.965 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.965 INFO:teuthology.orchestra.run.smithi071.stdout: "dump_on_update": { 2024-03-04T09:09:00.965 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "dump_on_update", 2024-03-04T09:09:00.966 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.966 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.966 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.966 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.966 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.966 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.966 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.966 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "dump database to debug log on update", 2024-03-04T09:09:00.967 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.967 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.967 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.967 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.967 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.967 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.967 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.967 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.968 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.968 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.968 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.968 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.968 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.968 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.968 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.968 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.969 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.969 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.969 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.969 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.969 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.969 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.969 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.969 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.970 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.970 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.970 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.970 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.970 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.970 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.970 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.970 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.971 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.971 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.971 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.971 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.971 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.971 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.971 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.971 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.972 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.972 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.972 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.972 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.972 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.972 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.972 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.972 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.973 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.973 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.973 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.973 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.973 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.973 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.973 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.973 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.974 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.974 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.974 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.974 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.974 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.974 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.974 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.974 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.974 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.975 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.975 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.975 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.975 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.975 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.975 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.976 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "stats", 2024-03-04T09:09:00.976 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.976 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.976 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.976 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.976 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.976 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.976 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.977 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.977 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.977 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.977 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.977 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.977 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.977 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.977 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.977 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.978 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.978 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.978 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.978 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.978 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.978 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.978 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.979 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.979 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.979 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.979 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.979 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.979 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.979 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.979 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.979 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.980 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.980 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.980 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.980 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.980 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.980 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.980 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.981 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.981 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.981 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.981 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.981 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.981 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.981 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.981 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.982 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.982 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.982 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.982 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.982 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.982 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.982 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.982 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.983 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.983 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.983 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.983 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.983 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.983 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.983 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.983 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.984 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.984 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.984 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.984 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.984 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.984 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.984 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.984 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.985 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "status", 2024-03-04T09:09:00.985 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.985 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.985 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.985 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.985 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.986 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.986 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.986 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.986 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.986 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.986 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.986 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.986 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.987 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.987 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.987 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.987 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.987 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.987 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.987 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.987 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.987 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.988 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.988 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:00.988 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:00.988 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.988 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.988 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.988 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.989 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.989 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.989 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.989 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.989 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.989 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.989 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.989 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:00.990 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:00.990 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.990 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.990 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.990 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:00.990 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.990 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.990 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.991 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.991 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.991 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.991 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.991 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:00.991 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:00.991 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:00.991 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.992 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.992 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.992 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.992 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:00.992 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:00.992 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:00.992 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.993 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.993 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:00.993 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.993 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.993 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.993 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.993 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.993 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.993 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.994 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.994 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:00.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:00.994 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "telegraf", 2024-03-04T09:09:00.994 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:00.994 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:00.994 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:00.994 INFO:teuthology.orchestra.run.smithi071.stdout: "address": { 2024-03-04T09:09:00.995 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "address", 2024-03-04T09:09:00.995 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.995 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.995 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.995 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-04T09:09:00.995 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.995 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.995 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.996 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.996 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.996 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.996 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.996 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-03-04T09:09:00.996 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-03-04T09:09:00.996 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:00.997 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.997 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:00.997 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "15", 2024-03-04T09:09:00.997 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.997 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.997 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:00.997 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:00.997 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:00.997 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:00.998 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:00.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:00.998 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:00.998 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:00.998 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:00.998 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:00.998 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:00.998 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:00.999 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:00.999 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:00.999 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:00.999 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:00.999 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:00.999 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:00.999 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:00.999 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.000 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.000 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.000 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.000 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.000 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.000 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.000 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.001 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.001 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.001 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.001 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.001 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.001 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.001 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.001 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.001 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.002 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.002 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.002 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.002 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.002 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.002 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.002 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.003 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.003 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.003 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.003 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.003 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.003 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.003 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.003 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.004 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.004 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.004 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.004 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.004 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.004 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.004 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.004 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.005 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.005 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.005 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.005 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.005 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.005 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.005 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.005 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.006 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.006 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.006 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.006 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.006 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.006 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.006 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.007 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "telemetry", 2024-03-04T09:09:01.007 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.007 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.007 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.007 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_basic": { 2024-03-04T09:09:01.007 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_basic", 2024-03-04T09:09:01.007 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.008 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.008 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.008 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.008 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.008 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.008 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.008 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-04T09:09:01.008 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.009 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.009 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.009 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_crash": { 2024-03-04T09:09:01.009 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_crash", 2024-03-04T09:09:01.009 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.009 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.009 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.009 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.010 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.010 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.010 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.010 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-04T09:09:01.010 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.010 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.010 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.011 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_device": { 2024-03-04T09:09:01.011 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_device", 2024-03-04T09:09:01.011 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.011 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.011 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.011 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.011 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.011 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.011 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.012 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-04T09:09:01.012 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.012 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.012 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.012 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_ident": { 2024-03-04T09:09:01.012 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_ident", 2024-03-04T09:09:01.012 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.013 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.013 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.013 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.013 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.013 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.013 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.013 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-04T09:09:01.013 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.014 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.014 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.014 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_perf": { 2024-03-04T09:09:01.014 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_perf", 2024-03-04T09:09:01.014 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.014 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.014 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.015 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.015 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.015 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.015 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.015 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-04T09:09:01.015 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.015 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.015 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.016 INFO:teuthology.orchestra.run.smithi071.stdout: "contact": { 2024-03-04T09:09:01.016 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "contact", 2024-03-04T09:09:01.016 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.016 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.016 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.016 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.016 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.016 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.017 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.017 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.017 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.017 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.017 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.017 INFO:teuthology.orchestra.run.smithi071.stdout: "description": { 2024-03-04T09:09:01.017 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "description", 2024-03-04T09:09:01.018 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.018 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.018 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.018 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.018 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.018 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.018 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.018 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.018 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.019 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.019 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.019 INFO:teuthology.orchestra.run.smithi071.stdout: "device_url": { 2024-03-04T09:09:01.019 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "device_url", 2024-03-04T09:09:01.019 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.019 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.019 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.020 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-04T09:09:01.020 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.020 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.020 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.020 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.020 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.020 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.020 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.021 INFO:teuthology.orchestra.run.smithi071.stdout: "enabled": { 2024-03-04T09:09:01.021 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "enabled", 2024-03-04T09:09:01.021 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.021 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.021 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.021 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.021 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.021 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.022 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.022 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.022 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.022 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.022 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.022 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-03-04T09:09:01.022 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-03-04T09:09:01.022 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.023 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.023 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.023 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "24", 2024-03-04T09:09:01.023 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "8", 2024-03-04T09:09:01.023 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.023 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.023 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.023 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.024 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.024 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.024 INFO:teuthology.orchestra.run.smithi071.stdout: "last_opt_revision": { 2024-03-04T09:09:01.024 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "last_opt_revision", 2024-03-04T09:09:01.024 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.024 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.024 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.024 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "1", 2024-03-04T09:09:01.025 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.025 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.025 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.025 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.025 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.025 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.025 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.025 INFO:teuthology.orchestra.run.smithi071.stdout: "leaderboard": { 2024-03-04T09:09:01.026 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "leaderboard", 2024-03-04T09:09:01.026 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.026 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.026 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.026 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.026 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.026 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.026 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.027 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.027 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.027 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.027 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.027 INFO:teuthology.orchestra.run.smithi071.stdout: "leaderboard_description": { 2024-03-04T09:09:01.027 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "leaderboard_description", 2024-03-04T09:09:01.027 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.028 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.028 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.028 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.028 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.028 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.028 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.028 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.028 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.029 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.029 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.029 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.029 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.029 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.029 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.029 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.029 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.029 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.030 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.030 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.030 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.030 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.030 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.030 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.030 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.030 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.031 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.031 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.031 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.031 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.031 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.031 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.031 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.032 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.032 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.032 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.032 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.032 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.032 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.032 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.032 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.032 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.033 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.033 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.033 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.033 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.033 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.033 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.033 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.034 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.034 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.034 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.034 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.034 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.034 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.034 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.034 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.035 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.035 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.035 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.035 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.035 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.035 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.035 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.035 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.036 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.036 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.036 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.036 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.036 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.036 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.036 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.036 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.037 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.037 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.037 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.037 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.037 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.037 INFO:teuthology.orchestra.run.smithi071.stdout: "organization": { 2024-03-04T09:09:01.037 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "organization", 2024-03-04T09:09:01.038 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.038 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.038 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.038 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.038 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.038 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.038 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.038 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.039 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.039 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.039 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.039 INFO:teuthology.orchestra.run.smithi071.stdout: "proxy": { 2024-03-04T09:09:01.039 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "proxy", 2024-03-04T09:09:01.039 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.039 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.039 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.040 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.040 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.040 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.040 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.040 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.040 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.040 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.040 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.041 INFO:teuthology.orchestra.run.smithi071.stdout: "url": { 2024-03-04T09:09:01.041 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "url", 2024-03-04T09:09:01.041 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.041 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.041 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.041 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-04T09:09:01.041 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.041 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.042 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.042 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.042 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.042 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.042 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.042 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.042 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.042 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.043 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "test_orchestrator", 2024-03-04T09:09:01.043 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.043 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.043 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.043 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.043 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.043 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.043 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.043 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.044 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.044 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.044 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.044 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.044 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.044 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.044 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.045 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.045 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.045 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.045 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.045 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.045 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.045 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.045 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.046 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.046 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.046 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.046 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.046 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.046 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.046 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.046 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.046 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.047 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.047 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.047 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.047 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.047 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.047 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.047 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.048 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.048 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.048 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.048 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.048 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.048 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.048 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.048 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.048 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.049 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.049 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.049 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.049 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.049 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.049 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.049 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.049 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.050 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.050 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.050 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.050 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.050 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.050 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.050 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.050 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.051 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.051 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.051 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.051 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.051 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.051 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.051 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.052 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "volumes", 2024-03-04T09:09:01.052 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.052 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.052 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.052 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.052 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.052 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.053 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.053 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.053 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.053 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.053 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.053 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.053 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.053 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.053 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.054 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.054 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.054 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.054 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.054 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.054 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.054 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.054 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.054 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.055 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.055 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.055 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.055 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.055 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.055 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.055 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.055 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.056 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.056 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.056 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.056 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.056 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.056 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.056 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.057 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.057 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.057 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.057 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.057 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.057 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.057 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.057 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.057 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.058 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.058 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.058 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.058 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.058 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.058 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.058 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.058 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.059 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.059 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.059 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.059 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.059 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.059 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.059 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.059 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.059 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.060 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.060 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.060 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.060 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.060 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.060 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.060 INFO:teuthology.orchestra.run.smithi071.stdout: "max_concurrent_clones": { 2024-03-04T09:09:01.060 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_concurrent_clones", 2024-03-04T09:09:01.061 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.061 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.061 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.061 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "4", 2024-03-04T09:09:01.061 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.061 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.061 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.061 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-04T09:09:01.062 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.062 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.062 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.062 INFO:teuthology.orchestra.run.smithi071.stdout: "periodic_async_work": { 2024-03-04T09:09:01.062 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "periodic_async_work", 2024-03-04T09:09:01.062 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.062 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.062 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.063 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.063 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.063 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.063 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.063 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Periodically check for async work", 2024-03-04T09:09:01.063 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.063 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.063 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.064 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot_clone_delay": { 2024-03-04T09:09:01.064 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "snapshot_clone_delay", 2024-03-04T09:09:01.064 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.064 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.064 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.064 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-03-04T09:09:01.064 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.064 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.065 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.065 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-04T09:09:01.065 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.065 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.065 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.065 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot_clone_no_wait": { 2024-03-04T09:09:01.065 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "snapshot_clone_no_wait", 2024-03-04T09:09:01.065 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.066 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.066 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.066 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.066 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.066 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.066 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.066 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-04T09:09:01.066 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.067 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.067 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.067 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.067 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.067 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.067 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.067 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "zabbix", 2024-03-04T09:09:01.067 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.068 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.068 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.068 INFO:teuthology.orchestra.run.smithi071.stdout: "discovery_interval": { 2024-03-04T09:09:01.068 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "discovery_interval", 2024-03-04T09:09:01.068 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-03-04T09:09:01.068 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.068 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.068 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "100", 2024-03-04T09:09:01.069 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.069 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.069 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.069 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.069 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.069 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.069 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.069 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.069 INFO:teuthology.orchestra.run.smithi071.stdout: "identifier": { 2024-03-04T09:09:01.070 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "identifier", 2024-03-04T09:09:01.070 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.070 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.070 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.070 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.070 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.070 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.070 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.071 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.071 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.071 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.071 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.071 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-03-04T09:09:01.071 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-03-04T09:09:01.071 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.071 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.072 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.072 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-03-04T09:09:01.072 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.072 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.072 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.072 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.072 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.072 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.073 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.073 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.073 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.073 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.073 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.073 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.073 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.073 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.073 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.074 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.074 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.074 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.074 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.074 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.074 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.074 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.074 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.074 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.075 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.075 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.075 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.075 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.075 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.075 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.075 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.075 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.076 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.076 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.076 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.076 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.076 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.076 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.076 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.076 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.077 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.077 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.077 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.077 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.077 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.077 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.077 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.077 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.077 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.078 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.078 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.078 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.078 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.078 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.078 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.078 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.078 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.079 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.079 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.079 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.079 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.079 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.079 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.079 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.079 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.079 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.080 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.080 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.080 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.080 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.080 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.080 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.080 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.080 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.081 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.081 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.081 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.081 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.081 INFO:teuthology.orchestra.run.smithi071.stdout: "zabbix_host": { 2024-03-04T09:09:01.081 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "zabbix_host", 2024-03-04T09:09:01.081 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.081 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.081 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.082 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.082 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.082 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.082 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.082 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.082 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.082 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.082 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.083 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.083 INFO:teuthology.orchestra.run.smithi071.stdout: "zabbix_port": { 2024-03-04T09:09:01.083 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "zabbix_port", 2024-03-04T09:09:01.083 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.083 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.083 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.083 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10051", 2024-03-04T09:09:01.083 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.084 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.084 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.087 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.087 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.087 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.087 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.087 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.087 INFO:teuthology.orchestra.run.smithi071.stdout: "zabbix_sender": { 2024-03-04T09:09:01.088 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "zabbix_sender", 2024-03-04T09:09:01.088 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.088 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.088 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.088 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-04T09:09:01.088 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.088 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.088 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.089 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.089 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.089 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.089 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.089 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.089 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.089 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.089 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-03-04T09:09:01.090 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.090 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.090 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 25220, 2024-03-04T09:09:01.090 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "y", 2024-03-04T09:09:01.090 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_features": 4540701547738038271, 2024-03-04T09:09:01.090 INFO:teuthology.orchestra.run.smithi071.stdout: "available_modules": [ 2024-03-04T09:09:01.090 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.090 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "alerts", 2024-03-04T09:09:01.090 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.091 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.091 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.091 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-03-04T09:09:01.091 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-03-04T09:09:01.091 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.091 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.091 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.091 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-03-04T09:09:01.092 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.092 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.092 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.092 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "How frequently to reexamine health status", 2024-03-04T09:09:01.092 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.092 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.092 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.092 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.093 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.093 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.093 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.093 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.093 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.093 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.093 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.093 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.094 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.094 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.094 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.094 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.094 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.094 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.094 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.094 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.095 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.095 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.095 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.095 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.095 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.095 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.095 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.095 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.096 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.096 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.096 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.096 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.096 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.096 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.096 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.096 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.096 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.097 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.097 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.097 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.097 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.097 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.097 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.097 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.097 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.098 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.098 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.098 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.098 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.098 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.098 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.098 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.098 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.098 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.099 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.099 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.099 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.099 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.099 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.099 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.099 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.099 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.100 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.100 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.100 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.100 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.100 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.100 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.100 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.100 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.100 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.101 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.101 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.101 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.101 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_destination": { 2024-03-04T09:09:01.101 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_destination", 2024-03-04T09:09:01.101 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.101 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.101 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.101 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.102 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.102 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.102 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.102 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Email address to send alerts to", 2024-03-04T09:09:01.102 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.102 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.102 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.102 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.103 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_from_name": { 2024-03-04T09:09:01.103 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_from_name", 2024-03-04T09:09:01.103 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.103 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.103 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.103 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "Ceph", 2024-03-04T09:09:01.103 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.103 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.104 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.104 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Email From: name", 2024-03-04T09:09:01.104 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.104 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.104 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.104 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.104 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_host": { 2024-03-04T09:09:01.104 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_host", 2024-03-04T09:09:01.104 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.105 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.105 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.105 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.105 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.105 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.105 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.105 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "SMTP server", 2024-03-04T09:09:01.105 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.106 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.106 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.106 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.106 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_password": { 2024-03-04T09:09:01.106 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_password", 2024-03-04T09:09:01.106 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.106 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.106 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.106 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.107 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.107 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.107 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.107 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Password to authenticate with", 2024-03-04T09:09:01.107 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.107 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.107 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.107 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.108 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_port": { 2024-03-04T09:09:01.108 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_port", 2024-03-04T09:09:01.108 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.108 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.108 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.108 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "465", 2024-03-04T09:09:01.108 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.108 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.108 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.109 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "SMTP port", 2024-03-04T09:09:01.109 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.109 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.109 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.109 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.109 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_sender": { 2024-03-04T09:09:01.109 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_sender", 2024-03-04T09:09:01.109 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.110 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.110 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.110 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.110 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.110 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.110 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.110 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "SMTP envelope sender", 2024-03-04T09:09:01.110 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.111 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.111 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.111 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.111 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_ssl": { 2024-03-04T09:09:01.111 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_ssl", 2024-03-04T09:09:01.111 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.111 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.111 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.111 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.112 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.112 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.112 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.112 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-04T09:09:01.112 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.112 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.112 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.112 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.113 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_user": { 2024-03-04T09:09:01.113 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_user", 2024-03-04T09:09:01.113 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.113 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.113 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.113 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.113 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.113 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.114 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.114 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "User to authenticate as", 2024-03-04T09:09:01.114 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.114 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.114 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.114 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.114 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.114 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.114 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.115 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "balancer", 2024-03-04T09:09:01.115 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.115 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.115 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.115 INFO:teuthology.orchestra.run.smithi071.stdout: "active": { 2024-03-04T09:09:01.115 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "active", 2024-03-04T09:09:01.115 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.115 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.116 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.116 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.116 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.116 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.116 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.116 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "automatically balance PGs across cluster", 2024-03-04T09:09:01.116 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.116 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.117 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.117 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.117 INFO:teuthology.orchestra.run.smithi071.stdout: "begin_time": { 2024-03-04T09:09:01.117 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "begin_time", 2024-03-04T09:09:01.117 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.117 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.117 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.117 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0000", 2024-03-04T09:09:01.118 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.118 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.118 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.118 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "beginning time of day to automatically balance", 2024-03-04T09:09:01.118 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-04T09:09:01.118 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.118 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.118 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.119 INFO:teuthology.orchestra.run.smithi071.stdout: "begin_weekday": { 2024-03-04T09:09:01.119 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "begin_weekday", 2024-03-04T09:09:01.119 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-03-04T09:09:01.119 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.119 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.119 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-03-04T09:09:01.119 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "0", 2024-03-04T09:09:01.119 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "6", 2024-03-04T09:09:01.120 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.120 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-04T09:09:01.120 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-04T09:09:01.120 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.120 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.120 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.120 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_compat_max_iterations": { 2024-03-04T09:09:01.120 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crush_compat_max_iterations", 2024-03-04T09:09:01.121 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-03-04T09:09:01.121 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.121 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.121 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "25", 2024-03-04T09:09:01.121 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1", 2024-03-04T09:09:01.121 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "250", 2024-03-04T09:09:01.121 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.121 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-04T09:09:01.121 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.122 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.122 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.122 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.122 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_compat_metrics": { 2024-03-04T09:09:01.122 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crush_compat_metrics", 2024-03-04T09:09:01.122 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.122 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.123 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.123 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "pgs,objects,bytes", 2024-03-04T09:09:01.123 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.123 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.123 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.123 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-04T09:09:01.123 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-04T09:09:01.123 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.124 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.124 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.124 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_compat_step": { 2024-03-04T09:09:01.124 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crush_compat_step", 2024-03-04T09:09:01.124 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-03-04T09:09:01.124 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.124 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.124 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0.5", 2024-03-04T09:09:01.125 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "0.001", 2024-03-04T09:09:01.125 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "0.999", 2024-03-04T09:09:01.125 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.125 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "aggressiveness of optimization", 2024-03-04T09:09:01.125 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-04T09:09:01.125 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.125 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.125 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.125 INFO:teuthology.orchestra.run.smithi071.stdout: "end_time": { 2024-03-04T09:09:01.126 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "end_time", 2024-03-04T09:09:01.126 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.126 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.126 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.126 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "2359", 2024-03-04T09:09:01.126 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.126 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.126 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.127 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "ending time of day to automatically balance", 2024-03-04T09:09:01.127 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-04T09:09:01.127 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.127 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.127 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.127 INFO:teuthology.orchestra.run.smithi071.stdout: "end_weekday": { 2024-03-04T09:09:01.127 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "end_weekday", 2024-03-04T09:09:01.127 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-03-04T09:09:01.128 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.128 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.128 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-03-04T09:09:01.128 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "0", 2024-03-04T09:09:01.128 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "6", 2024-03-04T09:09:01.128 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.128 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-04T09:09:01.128 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-04T09:09:01.128 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.129 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.129 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.129 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.129 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.129 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.129 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.129 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.129 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.129 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.130 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.130 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.130 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.130 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.130 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.130 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.130 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.130 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.131 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.131 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.131 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.131 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.131 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.131 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.131 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.131 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.131 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.132 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.132 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.132 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.132 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.132 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.132 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.132 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.132 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.133 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.133 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.133 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.133 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.133 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.133 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.133 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.133 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.133 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.134 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.134 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.134 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.134 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.134 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.134 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.134 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.134 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.135 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.135 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.135 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.135 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.135 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.135 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.135 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.135 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.135 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.136 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.136 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.136 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.136 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.136 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.136 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.136 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.136 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.136 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.137 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.137 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.137 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.137 INFO:teuthology.orchestra.run.smithi071.stdout: "min_score": { 2024-03-04T09:09:01.137 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "min_score", 2024-03-04T09:09:01.137 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-03-04T09:09:01.137 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.138 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.138 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-03-04T09:09:01.138 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.138 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.138 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.138 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-04T09:09:01.138 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.138 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.138 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.139 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.139 INFO:teuthology.orchestra.run.smithi071.stdout: "mode": { 2024-03-04T09:09:01.139 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mode", 2024-03-04T09:09:01.139 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.139 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.139 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.139 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "upmap", 2024-03-04T09:09:01.139 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.139 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.140 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.140 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-compat", 2024-03-04T09:09:01.140 INFO:teuthology.orchestra.run.smithi071.stdout: "none", 2024-03-04T09:09:01.140 INFO:teuthology.orchestra.run.smithi071.stdout: "read", 2024-03-04T09:09:01.140 INFO:teuthology.orchestra.run.smithi071.stdout: "upmap", 2024-03-04T09:09:01.140 INFO:teuthology.orchestra.run.smithi071.stdout: "upmap-read" 2024-03-04T09:09:01.140 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.140 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Balancer mode", 2024-03-04T09:09:01.141 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.141 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.141 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.141 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.141 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_ids": { 2024-03-04T09:09:01.141 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "pool_ids", 2024-03-04T09:09:01.141 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.141 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.141 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.142 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.142 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.142 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.142 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.142 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-04T09:09:01.142 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.142 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.142 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.143 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.143 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-03-04T09:09:01.143 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-03-04T09:09:01.143 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.143 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.143 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.143 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-03-04T09:09:01.143 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.144 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.144 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.144 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-04T09:09:01.144 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.144 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.144 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.144 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.144 INFO:teuthology.orchestra.run.smithi071.stdout: "upmap_max_deviation": { 2024-03-04T09:09:01.144 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "upmap_max_deviation", 2024-03-04T09:09:01.145 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.145 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.145 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.145 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5", 2024-03-04T09:09:01.145 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1", 2024-03-04T09:09:01.145 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.145 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.145 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-04T09:09:01.146 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-04T09:09:01.146 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.146 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.146 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.146 INFO:teuthology.orchestra.run.smithi071.stdout: "upmap_max_optimizations": { 2024-03-04T09:09:01.146 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "upmap_max_optimizations", 2024-03-04T09:09:01.146 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-03-04T09:09:01.146 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.146 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.147 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-03-04T09:09:01.147 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.147 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.147 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.147 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-04T09:09:01.147 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.147 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.147 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.148 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.148 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.148 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephadm", 2024-03-04T09:09:01.148 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.148 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.148 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.149 INFO:teuthology.orchestra.run.smithi071.stdout: "agent_down_multiplier": { 2024-03-04T09:09:01.149 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "agent_down_multiplier", 2024-03-04T09:09:01.149 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-03-04T09:09:01.149 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.149 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.149 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "3.0", 2024-03-04T09:09:01.149 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.149 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.149 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.150 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-04T09:09:01.150 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.150 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.150 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.150 INFO:teuthology.orchestra.run.smithi071.stdout: "agent_refresh_rate": { 2024-03-04T09:09:01.150 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "agent_refresh_rate", 2024-03-04T09:09:01.150 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.150 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.151 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.151 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "20", 2024-03-04T09:09:01.151 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.151 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.151 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.151 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-04T09:09:01.151 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.151 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.152 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.152 INFO:teuthology.orchestra.run.smithi071.stdout: "agent_starting_port": { 2024-03-04T09:09:01.152 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "agent_starting_port", 2024-03-04T09:09:01.152 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.152 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.152 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.152 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "4721", 2024-03-04T09:09:01.152 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.153 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.153 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.153 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-04T09:09:01.153 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.153 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.153 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.153 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_ptrace": { 2024-03-04T09:09:01.153 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "allow_ptrace", 2024-03-04T09:09:01.154 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.154 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.154 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.154 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.154 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.154 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.154 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.154 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-04T09:09:01.155 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-04T09:09:01.155 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.155 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.155 INFO:teuthology.orchestra.run.smithi071.stdout: "autotune_interval": { 2024-03-04T09:09:01.155 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "autotune_interval", 2024-03-04T09:09:01.155 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.155 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.155 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.156 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-03-04T09:09:01.156 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.156 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.156 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.156 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-04T09:09:01.156 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.156 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.156 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.157 INFO:teuthology.orchestra.run.smithi071.stdout: "autotune_memory_target_ratio": { 2024-03-04T09:09:01.157 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "autotune_memory_target_ratio", 2024-03-04T09:09:01.157 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-03-04T09:09:01.157 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.157 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.157 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0.7", 2024-03-04T09:09:01.157 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.158 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.158 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.158 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-04T09:09:01.158 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.158 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.158 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.158 INFO:teuthology.orchestra.run.smithi071.stdout: "cephadm_log_destination": { 2024-03-04T09:09:01.159 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephadm_log_destination", 2024-03-04T09:09:01.159 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.159 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.159 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.159 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.159 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.159 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.159 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.160 INFO:teuthology.orchestra.run.smithi071.stdout: "file", 2024-03-04T09:09:01.160 INFO:teuthology.orchestra.run.smithi071.stdout: "file,syslog", 2024-03-04T09:09:01.160 INFO:teuthology.orchestra.run.smithi071.stdout: "syslog" 2024-03-04T09:09:01.160 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.160 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-04T09:09:01.160 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.160 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.160 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.161 INFO:teuthology.orchestra.run.smithi071.stdout: "cgroups_split": { 2024-03-04T09:09:01.161 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cgroups_split", 2024-03-04T09:09:01.161 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.161 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.161 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.161 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.161 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.161 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.162 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.162 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-04T09:09:01.162 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.162 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.162 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.162 INFO:teuthology.orchestra.run.smithi071.stdout: "config_checks_enabled": { 2024-03-04T09:09:01.162 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "config_checks_enabled", 2024-03-04T09:09:01.162 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.163 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.163 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.163 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.163 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.163 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.163 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.163 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-04T09:09:01.163 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.164 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.164 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.164 INFO:teuthology.orchestra.run.smithi071.stdout: "config_dashboard": { 2024-03-04T09:09:01.164 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "config_dashboard", 2024-03-04T09:09:01.164 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.164 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.164 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.165 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.165 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.165 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.165 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.165 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-04T09:09:01.165 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.165 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.165 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.166 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_alertmanager": { 2024-03-04T09:09:01.166 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_alertmanager", 2024-03-04T09:09:01.166 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.166 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.166 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.166 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-04T09:09:01.166 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.166 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.166 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.167 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Prometheus container image", 2024-03-04T09:09:01.167 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.167 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.167 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.167 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.167 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_base": { 2024-03-04T09:09:01.167 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_base", 2024-03-04T09:09:01.167 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.168 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.168 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.168 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-04T09:09:01.168 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.168 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.168 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.168 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Container image name, without the tag", 2024-03-04T09:09:01.168 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.168 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.169 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.169 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_elasticsearch": { 2024-03-04T09:09:01.169 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_elasticsearch", 2024-03-04T09:09:01.169 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.169 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.169 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.169 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-04T09:09:01.170 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.170 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.170 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.170 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "elasticsearch container image", 2024-03-04T09:09:01.170 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.170 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.170 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.170 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_grafana": { 2024-03-04T09:09:01.171 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_grafana", 2024-03-04T09:09:01.171 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.171 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.171 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.171 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-04T09:09:01.171 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.171 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.171 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.172 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Prometheus container image", 2024-03-04T09:09:01.172 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.172 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.172 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.172 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_haproxy": { 2024-03-04T09:09:01.172 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_haproxy", 2024-03-04T09:09:01.172 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.172 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.173 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.173 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-04T09:09:01.173 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.173 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.173 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.173 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "HAproxy container image", 2024-03-04T09:09:01.173 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.173 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.173 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.174 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_jaeger_agent": { 2024-03-04T09:09:01.174 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_jaeger_agent", 2024-03-04T09:09:01.174 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.174 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.174 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.174 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-04T09:09:01.174 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.175 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.175 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.175 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Jaeger agent container image", 2024-03-04T09:09:01.175 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.175 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.175 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.175 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_jaeger_collector": { 2024-03-04T09:09:01.175 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_jaeger_collector", 2024-03-04T09:09:01.176 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.176 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.176 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.176 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-04T09:09:01.176 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.176 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.176 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.176 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Jaeger collector container image", 2024-03-04T09:09:01.177 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.177 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.177 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.177 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_jaeger_query": { 2024-03-04T09:09:01.177 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_jaeger_query", 2024-03-04T09:09:01.177 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.177 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.178 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.178 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-04T09:09:01.178 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.178 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.178 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.178 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Jaeger query container image", 2024-03-04T09:09:01.178 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.178 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.179 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.179 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_keepalived": { 2024-03-04T09:09:01.179 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_keepalived", 2024-03-04T09:09:01.179 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.179 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.179 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.179 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-04T09:09:01.180 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.180 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.180 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.180 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Keepalived container image", 2024-03-04T09:09:01.180 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.180 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.180 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.181 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_loki": { 2024-03-04T09:09:01.181 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_loki", 2024-03-04T09:09:01.181 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.181 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.181 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.181 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-04T09:09:01.181 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.181 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.181 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.182 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Loki container image", 2024-03-04T09:09:01.182 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.182 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.182 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.182 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_node_exporter": { 2024-03-04T09:09:01.182 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_node_exporter", 2024-03-04T09:09:01.182 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.183 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.183 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.183 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-04T09:09:01.183 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.183 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.183 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.183 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Prometheus container image", 2024-03-04T09:09:01.183 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.183 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.184 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.184 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_nvmeof": { 2024-03-04T09:09:01.184 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_nvmeof", 2024-03-04T09:09:01.184 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.184 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.184 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.184 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-04T09:09:01.185 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.185 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.185 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.185 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Nvme-of container image", 2024-03-04T09:09:01.185 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.185 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.185 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.185 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_prometheus": { 2024-03-04T09:09:01.186 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_prometheus", 2024-03-04T09:09:01.186 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.186 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.186 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.186 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-04T09:09:01.186 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.186 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.186 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.187 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Prometheus container image", 2024-03-04T09:09:01.187 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.187 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.187 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.187 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_promtail": { 2024-03-04T09:09:01.187 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_promtail", 2024-03-04T09:09:01.187 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.188 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.188 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.188 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-04T09:09:01.188 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.188 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.188 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.188 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Promtail container image", 2024-03-04T09:09:01.188 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.189 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.189 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.189 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_snmp_gateway": { 2024-03-04T09:09:01.189 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_snmp_gateway", 2024-03-04T09:09:01.189 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.189 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.189 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.189 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-04T09:09:01.190 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.190 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.190 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.190 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "SNMP Gateway container image", 2024-03-04T09:09:01.190 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.190 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.190 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.191 INFO:teuthology.orchestra.run.smithi071.stdout: "container_init": { 2024-03-04T09:09:01.191 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_init", 2024-03-04T09:09:01.191 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.191 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.191 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.191 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.191 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.191 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.191 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.192 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Run podman/docker with `--init`", 2024-03-04T09:09:01.192 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.192 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.192 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.192 INFO:teuthology.orchestra.run.smithi071.stdout: "daemon_cache_timeout": { 2024-03-04T09:09:01.192 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "daemon_cache_timeout", 2024-03-04T09:09:01.192 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.193 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.193 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.193 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-03-04T09:09:01.193 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.193 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.193 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.193 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-04T09:09:01.193 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.194 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.194 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.194 INFO:teuthology.orchestra.run.smithi071.stdout: "default_cephadm_command_timeout": { 2024-03-04T09:09:01.194 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "default_cephadm_command_timeout", 2024-03-04T09:09:01.194 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.194 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.194 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.194 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "900", 2024-03-04T09:09:01.195 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.195 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.195 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.195 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-04T09:09:01.195 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.195 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.195 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.195 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.196 INFO:teuthology.orchestra.run.smithi071.stdout: "default_registry": { 2024-03-04T09:09:01.196 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "default_registry", 2024-03-04T09:09:01.196 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.196 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.196 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.196 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "docker.io", 2024-03-04T09:09:01.196 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.196 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.196 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.197 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-04T09:09:01.197 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.197 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.197 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.197 INFO:teuthology.orchestra.run.smithi071.stdout: "device_cache_timeout": { 2024-03-04T09:09:01.197 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "device_cache_timeout", 2024-03-04T09:09:01.197 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.198 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.198 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.198 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "1800", 2024-03-04T09:09:01.198 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.198 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.198 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.198 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "seconds to cache device inventory", 2024-03-04T09:09:01.198 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.199 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.199 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.199 INFO:teuthology.orchestra.run.smithi071.stdout: "device_enhanced_scan": { 2024-03-04T09:09:01.199 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "device_enhanced_scan", 2024-03-04T09:09:01.199 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.199 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.199 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.199 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.200 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.200 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.200 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.200 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-04T09:09:01.200 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.200 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.200 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.201 INFO:teuthology.orchestra.run.smithi071.stdout: "facts_cache_timeout": { 2024-03-04T09:09:01.201 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "facts_cache_timeout", 2024-03-04T09:09:01.201 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.201 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.201 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.201 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-03-04T09:09:01.201 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.201 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.202 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.202 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "seconds to cache host facts data", 2024-03-04T09:09:01.202 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.202 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.202 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.202 INFO:teuthology.orchestra.run.smithi071.stdout: "host_check_interval": { 2024-03-04T09:09:01.202 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "host_check_interval", 2024-03-04T09:09:01.203 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.203 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.203 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.203 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-03-04T09:09:01.203 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.203 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.203 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.203 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to perform a host check", 2024-03-04T09:09:01.203 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.204 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.204 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.204 INFO:teuthology.orchestra.run.smithi071.stdout: "hw_monitoring": { 2024-03-04T09:09:01.204 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "hw_monitoring", 2024-03-04T09:09:01.204 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.204 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.204 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.205 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.205 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.205 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.205 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.205 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-04T09:09:01.205 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.205 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.205 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.206 INFO:teuthology.orchestra.run.smithi071.stdout: "inventory_list_all": { 2024-03-04T09:09:01.206 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "inventory_list_all", 2024-03-04T09:09:01.206 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.206 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.206 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.206 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.206 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.207 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.207 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.207 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-04T09:09:01.207 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.207 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.207 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.207 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.207 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.208 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.208 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.208 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.208 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.208 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.208 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.208 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.208 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.209 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.209 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.209 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.209 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.209 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.209 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.209 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.209 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.210 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.210 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.210 INFO:teuthology.orchestra.run.smithi071.stdout: "log_refresh_metadata": { 2024-03-04T09:09:01.210 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_refresh_metadata", 2024-03-04T09:09:01.210 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.210 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.210 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.211 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.211 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.211 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.211 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.211 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-04T09:09:01.211 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.211 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.211 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.212 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.212 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.212 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.212 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.212 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.212 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.212 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.212 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.213 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.213 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-04T09:09:01.213 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.213 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.213 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.213 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.213 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.214 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.214 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.214 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.214 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.214 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.214 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.214 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.214 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.215 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.215 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.215 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.215 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.215 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.215 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.215 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.215 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.216 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.216 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.216 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.216 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.216 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.216 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.216 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.216 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.217 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.217 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.217 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.217 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.217 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.217 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.217 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.217 INFO:teuthology.orchestra.run.smithi071.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-04T09:09:01.218 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-04T09:09:01.218 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.218 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.218 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.218 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.218 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.218 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.218 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.218 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-04T09:09:01.219 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.219 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.219 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.219 INFO:teuthology.orchestra.run.smithi071.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-04T09:09:01.219 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-04T09:09:01.219 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.220 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.220 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.220 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "*", 2024-03-04T09:09:01.220 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.220 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.220 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.220 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-04T09:09:01.220 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.221 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.221 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.221 INFO:teuthology.orchestra.run.smithi071.stdout: "max_count_per_host": { 2024-03-04T09:09:01.221 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_count_per_host", 2024-03-04T09:09:01.221 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.221 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.221 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.221 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-03-04T09:09:01.222 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.222 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.222 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.222 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "max number of daemons per service per host", 2024-03-04T09:09:01.222 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.222 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.222 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.223 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd_draining_count": { 2024-03-04T09:09:01.223 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_osd_draining_count", 2024-03-04T09:09:01.223 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.223 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.223 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.223 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-03-04T09:09:01.223 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.223 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.223 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.224 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-04T09:09:01.224 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.224 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.224 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.224 INFO:teuthology.orchestra.run.smithi071.stdout: "migration_current": { 2024-03-04T09:09:01.224 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "migration_current", 2024-03-04T09:09:01.224 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.225 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.225 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.225 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.225 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.225 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.225 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.225 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "internal - do not modify", 2024-03-04T09:09:01.225 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.225 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.226 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.226 INFO:teuthology.orchestra.run.smithi071.stdout: "mode": { 2024-03-04T09:09:01.226 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mode", 2024-03-04T09:09:01.226 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.226 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.226 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.226 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "root", 2024-03-04T09:09:01.226 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.227 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.227 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.227 INFO:teuthology.orchestra.run.smithi071.stdout: "cephadm-package", 2024-03-04T09:09:01.227 INFO:teuthology.orchestra.run.smithi071.stdout: "root" 2024-03-04T09:09:01.227 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.227 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "mode for remote execution of cephadm", 2024-03-04T09:09:01.227 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.227 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.227 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.228 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.228 INFO:teuthology.orchestra.run.smithi071.stdout: "oob_default_addr": { 2024-03-04T09:09:01.228 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "oob_default_addr", 2024-03-04T09:09:01.228 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.228 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.228 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.228 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "169.254.1.1", 2024-03-04T09:09:01.228 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-04T09:09:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: "prometheus_alerts_path": { 2024-03-04T09:09:01.230 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "prometheus_alerts_path", 2024-03-04T09:09:01.230 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.230 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.230 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.230 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-04T09:09:01.230 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.230 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.230 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.230 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-04T09:09:01.231 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.231 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.231 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.231 INFO:teuthology.orchestra.run.smithi071.stdout: "registry_insecure": { 2024-03-04T09:09:01.231 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "registry_insecure", 2024-03-04T09:09:01.231 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.231 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.232 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.232 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.232 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.232 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.232 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.232 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-04T09:09:01.232 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.232 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.232 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.233 INFO:teuthology.orchestra.run.smithi071.stdout: "registry_password": { 2024-03-04T09:09:01.233 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "registry_password", 2024-03-04T09:09:01.233 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.233 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.233 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.233 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.233 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.234 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.234 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.234 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-04T09:09:01.234 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.234 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.234 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.234 INFO:teuthology.orchestra.run.smithi071.stdout: "registry_url": { 2024-03-04T09:09:01.234 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "registry_url", 2024-03-04T09:09:01.235 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.235 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.235 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.235 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.235 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.235 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.235 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.235 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-04T09:09:01.236 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.236 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.236 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.236 INFO:teuthology.orchestra.run.smithi071.stdout: "registry_username": { 2024-03-04T09:09:01.236 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "registry_username", 2024-03-04T09:09:01.236 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.236 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.237 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.237 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.237 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.237 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.237 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.237 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-04T09:09:01.237 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.237 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.237 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.238 INFO:teuthology.orchestra.run.smithi071.stdout: "secure_monitoring_stack": { 2024-03-04T09:09:01.238 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "secure_monitoring_stack", 2024-03-04T09:09:01.238 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.238 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.238 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.238 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.238 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.239 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.239 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.239 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-04T09:09:01.239 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.239 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.239 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.239 INFO:teuthology.orchestra.run.smithi071.stdout: "service_discovery_port": { 2024-03-04T09:09:01.240 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "service_discovery_port", 2024-03-04T09:09:01.240 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.240 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.240 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.240 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8765", 2024-03-04T09:09:01.240 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.240 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.240 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.241 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "cephadm service discovery port", 2024-03-04T09:09:01.241 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.241 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.241 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.241 INFO:teuthology.orchestra.run.smithi071.stdout: "ssh_config_file": { 2024-03-04T09:09:01.241 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ssh_config_file", 2024-03-04T09:09:01.241 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.242 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.242 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.242 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.242 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.242 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.242 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.242 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-04T09:09:01.242 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.243 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.243 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.243 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.243 INFO:teuthology.orchestra.run.smithi071.stdout: "use_agent": { 2024-03-04T09:09:01.243 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "use_agent", 2024-03-04T09:09:01.243 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.243 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.243 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.244 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.244 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.244 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.244 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.244 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-04T09:09:01.244 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.244 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.244 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.245 INFO:teuthology.orchestra.run.smithi071.stdout: "use_repo_digest": { 2024-03-04T09:09:01.245 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "use_repo_digest", 2024-03-04T09:09:01.245 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.245 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.245 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.245 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.245 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.245 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.246 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.246 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-04T09:09:01.246 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.246 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.246 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.246 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_on_failed_host_check": { 2024-03-04T09:09:01.246 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_on_failed_host_check", 2024-03-04T09:09:01.247 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.247 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.247 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.247 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.247 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.247 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.247 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.247 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "raise a health warning if the host check fails", 2024-03-04T09:09:01.247 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_on_stray_daemons": { 2024-03-04T09:09:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_on_stray_daemons", 2024-03-04T09:09:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-04T09:09:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_on_stray_hosts": { 2024-03-04T09:09:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_on_stray_hosts", 2024-03-04T09:09:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-04T09:09:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crash", 2024-03-04T09:09:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "retain_interval": { 2024-03-04T09:09:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "retain_interval", 2024-03-04T09:09:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "31536000", 2024-03-04T09:09:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-04T09:09:01.262 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.262 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.262 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.262 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_recent_interval": { 2024-03-04T09:09:01.262 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_recent_interval", 2024-03-04T09:09:01.262 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.262 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.263 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.263 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "1209600", 2024-03-04T09:09:01.263 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.263 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.263 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.263 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-04T09:09:01.263 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.263 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.263 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.264 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.264 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.264 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "dashboard", 2024-03-04T09:09:01.264 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.264 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.264 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.265 INFO:teuthology.orchestra.run.smithi071.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-04T09:09:01.265 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-04T09:09:01.265 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.265 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.265 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.265 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-03-04T09:09:01.265 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.265 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.266 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.266 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.266 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.266 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.266 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.266 INFO:teuthology.orchestra.run.smithi071.stdout: "ALERTMANAGER_API_HOST": { 2024-03-04T09:09:01.266 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-04T09:09:01.266 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.267 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.267 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.267 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.267 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.267 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.267 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.267 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.267 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.268 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.268 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.268 INFO:teuthology.orchestra.run.smithi071.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-04T09:09:01.268 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-04T09:09:01.268 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.268 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.268 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.269 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.269 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.269 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.269 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.269 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.269 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.269 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.269 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.270 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.270 INFO:teuthology.orchestra.run.smithi071.stdout: "AUDIT_API_ENABLED": { 2024-03-04T09:09:01.270 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "AUDIT_API_ENABLED", 2024-03-04T09:09:01.270 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.270 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.270 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.270 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.270 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.270 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.271 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.271 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.271 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.271 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.271 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.271 INFO:teuthology.orchestra.run.smithi071.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-04T09:09:01.271 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-04T09:09:01.272 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.272 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.272 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.272 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.272 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.272 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.272 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.272 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.272 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.273 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.273 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.273 INFO:teuthology.orchestra.run.smithi071.stdout: "ENABLE_BROWSABLE_API": { 2024-03-04T09:09:01.273 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-04T09:09:01.273 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.273 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.273 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.274 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.274 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.274 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.274 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.274 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.274 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.274 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.274 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.274 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.275 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-04T09:09:01.275 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-04T09:09:01.275 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.275 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.275 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.275 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.275 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.275 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.275 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.276 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.276 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.276 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.276 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.276 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-04T09:09:01.276 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-04T09:09:01.276 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.277 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.277 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.277 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.277 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.277 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.277 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.277 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.277 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.278 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.278 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.278 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-04T09:09:01.278 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-04T09:09:01.278 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.278 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.278 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.278 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.279 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.279 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.279 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.279 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.279 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.279 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.279 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.280 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-04T09:09:01.280 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-04T09:09:01.280 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.280 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.280 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.280 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.280 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.280 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.280 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.281 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.281 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.281 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.281 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.281 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-04T09:09:01.281 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-04T09:09:01.281 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.282 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.282 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.282 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.282 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.282 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.282 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.282 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.282 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.282 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.283 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.283 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-04T09:09:01.283 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-04T09:09:01.283 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.283 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.283 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.283 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.284 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.284 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.284 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.284 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.284 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.284 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.284 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.285 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-04T09:09:01.285 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-04T09:09:01.285 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.285 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.285 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.285 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.285 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.285 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.286 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.286 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.286 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.286 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.286 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.286 INFO:teuthology.orchestra.run.smithi071.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-04T09:09:01.286 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-04T09:09:01.287 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.287 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.287 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.287 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.287 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.287 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.287 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.287 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.288 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.288 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.288 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.288 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_API_PASSWORD": { 2024-03-04T09:09:01.288 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-04T09:09:01.288 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.288 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.289 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.289 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "admin", 2024-03-04T09:09:01.289 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.289 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.289 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.289 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.289 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.289 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.290 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.290 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-04T09:09:01.290 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-04T09:09:01.290 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.290 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.290 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.290 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.290 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.291 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.291 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.291 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.291 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.291 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.291 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.291 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_API_URL": { 2024-03-04T09:09:01.291 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_API_URL", 2024-03-04T09:09:01.292 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.292 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.292 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.292 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.292 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.292 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.292 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.292 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.293 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.293 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.293 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.293 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_API_USERNAME": { 2024-03-04T09:09:01.293 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-04T09:09:01.293 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.293 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.294 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.294 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "admin", 2024-03-04T09:09:01.294 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.294 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.294 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.294 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.294 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.294 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.295 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.295 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-04T09:09:01.295 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-04T09:09:01.295 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.295 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.295 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.295 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.296 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.296 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.296 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.296 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.296 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.296 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.296 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.297 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-04T09:09:01.297 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-04T09:09:01.297 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.297 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.297 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.297 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.297 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.297 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.298 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.298 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.298 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.298 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.298 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.298 INFO:teuthology.orchestra.run.smithi071.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-04T09:09:01.298 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-04T09:09:01.298 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.299 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.299 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.299 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.299 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.299 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.299 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.299 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.299 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.300 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.300 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.300 INFO:teuthology.orchestra.run.smithi071.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-04T09:09:01.300 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-04T09:09:01.300 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.300 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.300 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.301 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.301 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.301 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.301 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.301 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.301 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.301 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.301 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.302 INFO:teuthology.orchestra.run.smithi071.stdout: "PROMETHEUS_API_HOST": { 2024-03-04T09:09:01.302 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-04T09:09:01.302 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.302 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.302 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.302 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.302 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.302 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.303 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.303 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.303 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.303 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.303 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.303 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.303 INFO:teuthology.orchestra.run.smithi071.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-04T09:09:01.303 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-04T09:09:01.303 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.304 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.304 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.304 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.304 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.304 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.304 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.304 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.304 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.305 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.305 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.305 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-04T09:09:01.305 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-04T09:09:01.305 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.305 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.305 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.306 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.306 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.306 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.306 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.306 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.306 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.306 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.306 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.307 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-04T09:09:01.307 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-04T09:09:01.307 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.307 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.307 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.307 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.307 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.308 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.308 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.308 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.308 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.308 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.308 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.308 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-04T09:09:01.308 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-04T09:09:01.309 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.309 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.309 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.309 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.309 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.309 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.309 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.309 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.310 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.310 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.310 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.310 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-04T09:09:01.310 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-04T09:09:01.310 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.310 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.310 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.311 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.311 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.311 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.311 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.311 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.311 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.311 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.311 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.312 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.312 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-04T09:09:01.312 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-04T09:09:01.312 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.312 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.312 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.312 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.312 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.313 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.313 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.313 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.313 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.313 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.313 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.313 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.313 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-04T09:09:01.313 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-04T09:09:01.314 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.314 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.314 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.314 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.314 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.314 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.314 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.314 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.315 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.315 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.315 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.315 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-04T09:09:01.315 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-04T09:09:01.315 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.315 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.315 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.316 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.316 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.316 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.316 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.316 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.316 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.316 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.316 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.317 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_ENABLED": { 2024-03-04T09:09:01.317 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-04T09:09:01.317 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.317 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.317 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.317 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.317 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.317 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.318 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.318 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.318 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.318 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.318 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.318 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-04T09:09:01.318 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-04T09:09:01.319 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.319 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.319 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.319 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-04T09:09:01.319 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.319 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.319 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.319 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.320 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.320 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.320 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.320 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-04T09:09:01.320 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-04T09:09:01.320 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.320 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.320 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.321 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-03-04T09:09:01.321 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.321 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.321 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.321 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.321 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.321 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.321 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.322 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-04T09:09:01.322 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-04T09:09:01.322 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.322 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.322 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.322 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8", 2024-03-04T09:09:01.322 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.322 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.323 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.323 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.323 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.323 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.323 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.323 INFO:teuthology.orchestra.run.smithi071.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-04T09:09:01.323 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-04T09:09:01.323 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.324 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.324 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.324 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "45", 2024-03-04T09:09:01.324 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.324 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.324 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.324 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.324 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.325 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.325 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.325 INFO:teuthology.orchestra.run.smithi071.stdout: "RGW_API_ACCESS_KEY": { 2024-03-04T09:09:01.325 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-04T09:09:01.325 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.325 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.325 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.325 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.326 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.326 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.326 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.326 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.326 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.326 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.326 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.327 INFO:teuthology.orchestra.run.smithi071.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-04T09:09:01.327 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-04T09:09:01.327 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.327 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.327 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.327 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "admin", 2024-03-04T09:09:01.327 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.327 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.328 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.328 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.328 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.328 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.328 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.328 INFO:teuthology.orchestra.run.smithi071.stdout: "RGW_API_SECRET_KEY": { 2024-03-04T09:09:01.328 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-04T09:09:01.329 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.329 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.329 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.329 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.329 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.329 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.329 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.329 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.329 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.330 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.330 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.330 INFO:teuthology.orchestra.run.smithi071.stdout: "RGW_API_SSL_VERIFY": { 2024-03-04T09:09:01.330 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-04T09:09:01.330 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.330 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.330 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.330 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.331 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.331 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.331 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.331 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.331 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.331 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.331 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.332 INFO:teuthology.orchestra.run.smithi071.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-04T09:09:01.332 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-04T09:09:01.332 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.332 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.332 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.332 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-03-04T09:09:01.332 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.332 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.332 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.333 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.333 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.333 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.333 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.333 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.333 INFO:teuthology.orchestra.run.smithi071.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-04T09:09:01.333 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-04T09:09:01.335 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.335 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.335 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.335 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-03-04T09:09:01.335 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.335 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.336 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.336 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.336 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.336 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.336 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.336 INFO:teuthology.orchestra.run.smithi071.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-04T09:09:01.336 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-04T09:09:01.337 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.337 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.337 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.337 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5", 2024-03-04T09:09:01.337 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.337 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.337 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.337 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.338 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.338 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.338 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.338 INFO:teuthology.orchestra.run.smithi071.stdout: "cross_origin_url": { 2024-03-04T09:09:01.338 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cross_origin_url", 2024-03-04T09:09:01.338 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.338 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.338 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.339 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.339 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.339 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.339 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.339 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.339 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.339 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.339 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.340 INFO:teuthology.orchestra.run.smithi071.stdout: "crt_file": { 2024-03-04T09:09:01.340 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crt_file", 2024-03-04T09:09:01.340 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.340 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.340 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.340 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.340 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.340 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.341 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.341 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.341 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.341 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.341 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.341 INFO:teuthology.orchestra.run.smithi071.stdout: "debug": { 2024-03-04T09:09:01.341 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "debug", 2024-03-04T09:09:01.341 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.342 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.342 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.342 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.342 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.342 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.342 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.342 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Enable/disable debug options", 2024-03-04T09:09:01.342 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.343 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.343 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.343 INFO:teuthology.orchestra.run.smithi071.stdout: "jwt_token_ttl": { 2024-03-04T09:09:01.343 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "jwt_token_ttl", 2024-03-04T09:09:01.343 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.343 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.343 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.343 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "28800", 2024-03-04T09:09:01.344 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.344 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.344 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.344 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.344 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.344 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.344 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.345 INFO:teuthology.orchestra.run.smithi071.stdout: "key_file": { 2024-03-04T09:09:01.345 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "key_file", 2024-03-04T09:09:01.345 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.345 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.345 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.345 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.345 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.345 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.345 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.346 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.346 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.346 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.346 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.346 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.346 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.346 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.346 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.347 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.347 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.347 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.347 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.347 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.347 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.347 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.347 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.348 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.348 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.348 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.348 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.348 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.348 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.348 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.348 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.349 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.349 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.349 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.349 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.349 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.349 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.349 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.349 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.350 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.350 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.350 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.350 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.350 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.350 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.350 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.350 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.351 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.351 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.351 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.351 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.351 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.351 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.351 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.351 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.352 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.352 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.352 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.352 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.352 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.352 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.352 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.352 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.353 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.353 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.353 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.353 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.353 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.353 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.353 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.354 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.354 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.354 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.354 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.354 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.354 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.354 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.355 INFO:teuthology.orchestra.run.smithi071.stdout: "motd": { 2024-03-04T09:09:01.355 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "motd", 2024-03-04T09:09:01.355 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.355 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.355 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.355 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.355 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.355 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.355 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.356 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "The message of the day", 2024-03-04T09:09:01.356 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.356 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.356 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.356 INFO:teuthology.orchestra.run.smithi071.stdout: "redirect_resolve_ip_addr": { 2024-03-04T09:09:01.356 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "redirect_resolve_ip_addr", 2024-03-04T09:09:01.356 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.357 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.357 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.357 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.357 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.357 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.357 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.357 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.357 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.357 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.358 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.358 INFO:teuthology.orchestra.run.smithi071.stdout: "server_addr": { 2024-03-04T09:09:01.358 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_addr", 2024-03-04T09:09:01.358 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.358 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.358 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.358 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "::", 2024-03-04T09:09:01.359 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.359 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.359 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.359 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.359 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.359 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.359 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.359 INFO:teuthology.orchestra.run.smithi071.stdout: "server_port": { 2024-03-04T09:09:01.360 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_port", 2024-03-04T09:09:01.360 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.360 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.360 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.360 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8080", 2024-03-04T09:09:01.360 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.360 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.360 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.361 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.361 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.361 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.361 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.361 INFO:teuthology.orchestra.run.smithi071.stdout: "ssl": { 2024-03-04T09:09:01.361 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ssl", 2024-03-04T09:09:01.361 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.362 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.362 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.362 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.362 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.362 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.362 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.362 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.362 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.363 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.363 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.363 INFO:teuthology.orchestra.run.smithi071.stdout: "ssl_server_port": { 2024-03-04T09:09:01.363 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ssl_server_port", 2024-03-04T09:09:01.363 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.363 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.363 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.363 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8443", 2024-03-04T09:09:01.364 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.364 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.364 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.364 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.364 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.364 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.364 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.365 INFO:teuthology.orchestra.run.smithi071.stdout: "standby_behaviour": { 2024-03-04T09:09:01.365 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "standby_behaviour", 2024-03-04T09:09:01.365 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.365 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.365 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.365 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "redirect", 2024-03-04T09:09:01.365 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.365 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.365 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.366 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.366 INFO:teuthology.orchestra.run.smithi071.stdout: "redirect" 2024-03-04T09:09:01.366 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.366 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.366 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.366 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.366 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.367 INFO:teuthology.orchestra.run.smithi071.stdout: "standby_error_status_code": { 2024-03-04T09:09:01.367 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "standby_error_status_code", 2024-03-04T09:09:01.367 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.367 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.367 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.367 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "500", 2024-03-04T09:09:01.367 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "400", 2024-03-04T09:09:01.367 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "599", 2024-03-04T09:09:01.367 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.368 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.368 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.368 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.368 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.368 INFO:teuthology.orchestra.run.smithi071.stdout: "url_prefix": { 2024-03-04T09:09:01.368 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "url_prefix", 2024-03-04T09:09:01.368 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.369 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.369 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.369 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.369 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.369 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.369 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.369 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.369 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.369 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.370 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.370 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.370 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.370 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "devicehealth", 2024-03-04T09:09:01.370 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.370 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.371 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.371 INFO:teuthology.orchestra.run.smithi071.stdout: "enable_monitoring": { 2024-03-04T09:09:01.371 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "enable_monitoring", 2024-03-04T09:09:01.371 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.371 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.371 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.371 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.371 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.371 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.372 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.372 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "monitor device health metrics", 2024-03-04T09:09:01.372 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.372 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.372 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.372 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.372 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.373 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.373 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.373 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.373 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.373 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.373 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.373 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.373 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.374 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.374 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.374 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.374 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.374 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.374 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.374 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.374 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.374 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.375 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.375 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.375 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.375 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.375 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.375 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.375 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.376 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.376 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.376 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.376 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.376 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.376 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.376 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.377 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.377 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.377 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.377 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.377 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.377 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.377 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.377 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.377 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.378 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.378 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.378 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.378 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.378 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.378 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.378 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.378 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.378 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.379 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.379 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.379 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.379 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.379 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.379 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.379 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.380 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.380 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.380 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.380 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.380 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.380 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.380 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.380 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.381 INFO:teuthology.orchestra.run.smithi071.stdout: "mark_out_threshold": { 2024-03-04T09:09:01.381 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mark_out_threshold", 2024-03-04T09:09:01.381 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.381 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.381 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.381 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "2419200", 2024-03-04T09:09:01.381 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.382 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.382 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.382 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-04T09:09:01.382 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.382 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.382 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.382 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": { 2024-03-04T09:09:01.382 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "pool_name", 2024-03-04T09:09:01.383 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.383 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.383 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.383 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "device_health_metrics", 2024-03-04T09:09:01.383 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.383 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.383 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.383 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-04T09:09:01.383 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.384 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.384 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.384 INFO:teuthology.orchestra.run.smithi071.stdout: "retention_period": { 2024-03-04T09:09:01.384 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "retention_period", 2024-03-04T09:09:01.384 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.384 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.384 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.385 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "15552000", 2024-03-04T09:09:01.385 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.385 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.385 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.385 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how long to retain device health metrics", 2024-03-04T09:09:01.385 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.385 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.385 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.386 INFO:teuthology.orchestra.run.smithi071.stdout: "scrape_frequency": { 2024-03-04T09:09:01.386 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "scrape_frequency", 2024-03-04T09:09:01.386 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.386 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.386 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.386 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "86400", 2024-03-04T09:09:01.386 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.386 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.387 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.387 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-04T09:09:01.387 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.387 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.387 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.387 INFO:teuthology.orchestra.run.smithi071.stdout: "self_heal": { 2024-03-04T09:09:01.387 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "self_heal", 2024-03-04T09:09:01.387 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.388 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.388 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.388 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.388 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.388 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.388 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.388 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-04T09:09:01.388 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.388 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.389 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.389 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-03-04T09:09:01.389 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-03-04T09:09:01.389 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.389 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.389 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.389 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-03-04T09:09:01.390 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.390 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.390 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.390 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to wake up and check device health", 2024-03-04T09:09:01.390 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.390 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.390 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.390 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_threshold": { 2024-03-04T09:09:01.391 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_threshold", 2024-03-04T09:09:01.391 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.391 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.391 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.391 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "7257600", 2024-03-04T09:09:01.391 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.391 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.391 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.392 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-04T09:09:01.392 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.392 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.392 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.392 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.392 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.393 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "diskprediction_local", 2024-03-04T09:09:01.393 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.393 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.393 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.393 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.393 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.393 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.393 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.393 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.394 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.394 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.394 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.394 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.394 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.394 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.394 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.394 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.395 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.395 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.395 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.395 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.395 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.395 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.395 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.396 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.396 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.396 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.396 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.396 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.396 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.396 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.396 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.397 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.397 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.397 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.397 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.397 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.397 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.397 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.398 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.398 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.398 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.398 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.398 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.398 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.398 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.398 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.398 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.399 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.399 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.399 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.399 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.399 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.399 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.399 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.399 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.400 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.400 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.400 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.400 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.400 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.400 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.400 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.401 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.401 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.401 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.401 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.401 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.401 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.401 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.401 INFO:teuthology.orchestra.run.smithi071.stdout: "predict_interval": { 2024-03-04T09:09:01.402 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "predict_interval", 2024-03-04T09:09:01.402 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.402 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.402 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.402 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "86400", 2024-03-04T09:09:01.402 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.402 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.402 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.402 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.403 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.403 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.403 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.403 INFO:teuthology.orchestra.run.smithi071.stdout: "predictor_model": { 2024-03-04T09:09:01.403 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "predictor_model", 2024-03-04T09:09:01.403 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.403 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.404 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.404 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "prophetstor", 2024-03-04T09:09:01.404 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.404 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.404 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.404 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.404 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.404 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.404 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.405 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-03-04T09:09:01.405 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-03-04T09:09:01.405 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.405 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.405 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.405 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-03-04T09:09:01.405 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.406 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.406 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.406 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.406 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.406 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.406 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.406 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.406 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.407 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "influx", 2024-03-04T09:09:01.407 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": false, 2024-03-04T09:09:01.407 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "influxdb python module not found", 2024-03-04T09:09:01.407 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.407 INFO:teuthology.orchestra.run.smithi071.stdout: "batch_size": { 2024-03-04T09:09:01.407 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "batch_size", 2024-03-04T09:09:01.407 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.408 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.408 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.408 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5000", 2024-03-04T09:09:01.408 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.408 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.408 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.408 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-04T09:09:01.408 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.408 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.409 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.409 INFO:teuthology.orchestra.run.smithi071.stdout: "database": { 2024-03-04T09:09:01.409 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "database", 2024-03-04T09:09:01.409 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.409 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.409 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.409 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "ceph", 2024-03-04T09:09:01.409 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.410 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.410 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.410 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-04T09:09:01.410 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.410 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.410 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.410 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": { 2024-03-04T09:09:01.411 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "hostname", 2024-03-04T09:09:01.411 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.411 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.411 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.411 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.411 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.411 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.411 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.411 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "InfluxDB server hostname", 2024-03-04T09:09:01.412 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.412 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.412 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.412 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-03-04T09:09:01.412 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-03-04T09:09:01.412 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.412 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.413 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.413 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "30", 2024-03-04T09:09:01.413 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "5", 2024-03-04T09:09:01.413 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.413 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.413 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-04T09:09:01.413 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.413 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.413 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.414 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.414 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.414 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.414 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.414 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.414 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.414 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.415 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.415 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.415 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.415 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.415 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.415 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.415 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.415 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.415 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.416 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.416 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.416 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.416 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.416 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.416 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.416 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.417 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.417 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.417 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.417 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.417 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.417 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.417 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.417 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.418 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.418 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.418 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.418 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.418 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.418 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.418 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.418 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.419 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.419 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.419 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.419 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.419 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.419 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.419 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.419 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.419 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.420 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.420 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.420 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.420 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.420 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.420 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.420 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.421 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.421 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.421 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.421 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.421 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.421 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.421 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.421 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.421 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.422 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.422 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.422 INFO:teuthology.orchestra.run.smithi071.stdout: "password": { 2024-03-04T09:09:01.422 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "password", 2024-03-04T09:09:01.422 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.422 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.422 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.423 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.423 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.423 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.423 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.423 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "password of InfluxDB server user", 2024-03-04T09:09:01.423 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.423 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.423 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.424 INFO:teuthology.orchestra.run.smithi071.stdout: "port": { 2024-03-04T09:09:01.424 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "port", 2024-03-04T09:09:01.424 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.424 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.424 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.424 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8086", 2024-03-04T09:09:01.424 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.424 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.425 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.425 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "InfluxDB server port", 2024-03-04T09:09:01.425 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.425 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.425 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.425 INFO:teuthology.orchestra.run.smithi071.stdout: "ssl": { 2024-03-04T09:09:01.425 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ssl", 2024-03-04T09:09:01.425 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.426 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.426 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.426 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "false", 2024-03-04T09:09:01.426 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.426 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.426 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.426 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-04T09:09:01.426 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.426 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.427 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.427 INFO:teuthology.orchestra.run.smithi071.stdout: "threads": { 2024-03-04T09:09:01.427 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "threads", 2024-03-04T09:09:01.427 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.427 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.427 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.427 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5", 2024-03-04T09:09:01.428 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1", 2024-03-04T09:09:01.428 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "32", 2024-03-04T09:09:01.428 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.428 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-04T09:09:01.428 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.428 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.428 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.428 INFO:teuthology.orchestra.run.smithi071.stdout: "username": { 2024-03-04T09:09:01.429 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "username", 2024-03-04T09:09:01.429 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.429 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.429 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.429 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.429 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.429 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.429 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.430 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "username of InfluxDB server user", 2024-03-04T09:09:01.430 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.430 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.430 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.430 INFO:teuthology.orchestra.run.smithi071.stdout: "verify_ssl": { 2024-03-04T09:09:01.430 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "verify_ssl", 2024-03-04T09:09:01.430 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.430 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.431 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.431 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "true", 2024-03-04T09:09:01.431 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.431 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.431 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.431 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-04T09:09:01.431 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.431 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.432 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.432 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.432 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.432 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "insights", 2024-03-04T09:09:01.432 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.432 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.432 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.433 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.433 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.433 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.433 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.433 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.433 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.433 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.433 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.434 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.434 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.434 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.434 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.434 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.434 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.434 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.434 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.434 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.435 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.435 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.435 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.435 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.435 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.435 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.435 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.435 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.436 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.436 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.436 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.436 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.436 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.436 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.436 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.436 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.437 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.437 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.437 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.437 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.437 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.437 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.437 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.437 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.438 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.438 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.438 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.438 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.438 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.438 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.438 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.438 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.438 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.439 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.439 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.439 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.439 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.439 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.439 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.439 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.440 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.440 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.440 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.440 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.440 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.440 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.440 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.440 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.441 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.441 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.441 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.441 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "iostat", 2024-03-04T09:09:01.441 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.441 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.441 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.442 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.442 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.442 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.442 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.442 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.442 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.442 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.442 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.443 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.443 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.443 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.443 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.443 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.443 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.443 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.443 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.443 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.444 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.444 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.444 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.444 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.444 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.444 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.444 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.444 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.445 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.445 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.445 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.445 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.445 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.445 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.445 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.445 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.446 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.446 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.446 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.446 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.446 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.446 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.446 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.446 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.447 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.447 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.447 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.447 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.447 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.447 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.447 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.447 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.448 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.448 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.448 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.448 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.448 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.448 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.448 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.449 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.449 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.449 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.449 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.449 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.449 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.449 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.449 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.449 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.450 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.450 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.450 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.450 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "k8sevents", 2024-03-04T09:09:01.450 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.450 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.451 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.451 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_event_retention_days": { 2024-03-04T09:09:01.451 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ceph_event_retention_days", 2024-03-04T09:09:01.451 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.451 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.451 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.451 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "7", 2024-03-04T09:09:01.451 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.451 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.452 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.452 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-04T09:09:01.452 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.452 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.452 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.452 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.452 INFO:teuthology.orchestra.run.smithi071.stdout: "config_check_secs": { 2024-03-04T09:09:01.452 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "config_check_secs", 2024-03-04T09:09:01.453 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.453 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.453 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.453 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-03-04T09:09:01.453 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "10", 2024-03-04T09:09:01.453 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.453 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.453 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-04T09:09:01.453 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.454 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.454 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.454 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.454 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.454 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.454 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.454 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.455 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.455 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.455 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.455 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.455 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.455 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.455 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.455 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.455 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.456 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.456 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.456 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.456 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.456 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.456 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.456 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.457 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.457 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.457 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.457 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.457 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.457 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.457 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.457 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.457 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.458 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.458 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.458 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.458 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.458 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.458 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.458 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.459 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.459 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.459 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.459 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.459 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.459 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.459 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.459 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.459 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.460 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.460 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.460 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.460 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.460 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.460 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.460 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.461 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.461 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.461 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.461 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.461 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.461 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.461 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.461 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.461 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.462 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.462 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.462 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.462 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.462 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.462 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.463 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "localpool", 2024-03-04T09:09:01.463 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.463 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.463 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.463 INFO:teuthology.orchestra.run.smithi071.stdout: "failure_domain": { 2024-03-04T09:09:01.463 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "failure_domain", 2024-03-04T09:09:01.463 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.463 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.463 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.464 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "host", 2024-03-04T09:09:01.464 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.464 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.464 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.464 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "failure domain for any created local pool", 2024-03-04T09:09:01.464 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-04T09:09:01.464 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.464 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.465 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.465 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.465 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.465 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.465 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.465 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.465 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.465 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.466 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.466 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.466 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.466 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.466 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.466 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.466 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.466 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.467 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.467 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.467 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.467 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.467 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.467 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.467 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.467 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.468 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.468 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.468 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.468 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.468 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.468 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.468 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.468 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.469 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.469 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.469 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.469 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.469 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.469 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.469 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.469 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.470 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.470 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.470 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.470 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.470 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.470 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.470 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.470 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.471 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.471 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.471 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.471 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.471 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.471 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.471 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.471 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.472 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.472 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.472 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.472 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.472 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.472 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.472 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.472 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.473 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.473 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.473 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": { 2024-03-04T09:09:01.473 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "min_size", 2024-03-04T09:09:01.473 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.473 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.473 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.474 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.474 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.474 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.474 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.474 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "default min_size for any created local pool", 2024-03-04T09:09:01.474 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-04T09:09:01.474 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.474 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.475 INFO:teuthology.orchestra.run.smithi071.stdout: "num_rep": { 2024-03-04T09:09:01.475 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "num_rep", 2024-03-04T09:09:01.475 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.475 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.475 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.475 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "3", 2024-03-04T09:09:01.475 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.475 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.476 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.476 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "default replica count for any created local pool", 2024-03-04T09:09:01.476 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.476 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.476 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.476 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": { 2024-03-04T09:09:01.476 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "pg_num", 2024-03-04T09:09:01.477 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.477 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.477 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.477 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "128", 2024-03-04T09:09:01.477 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.477 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.477 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.477 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "default pg_num for any created local pool", 2024-03-04T09:09:01.477 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.478 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.478 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.478 INFO:teuthology.orchestra.run.smithi071.stdout: "prefix": { 2024-03-04T09:09:01.478 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "prefix", 2024-03-04T09:09:01.478 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.478 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.478 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.478 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.479 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.479 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.479 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.479 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "name prefix for any created local pool", 2024-03-04T09:09:01.479 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.479 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.479 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.480 INFO:teuthology.orchestra.run.smithi071.stdout: "subtree": { 2024-03-04T09:09:01.480 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subtree", 2024-03-04T09:09:01.480 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.480 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.480 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.480 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "rack", 2024-03-04T09:09:01.480 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.480 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.481 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.481 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-04T09:09:01.481 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-04T09:09:01.481 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.481 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.481 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.481 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.482 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mds_autoscaler", 2024-03-04T09:09:01.482 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.482 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.482 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.482 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.482 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.482 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.483 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.483 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.483 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.483 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.483 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.483 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.483 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.483 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.483 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.484 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.484 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.484 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.484 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.484 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.484 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.484 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.484 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.485 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.485 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.485 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.485 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.485 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.486 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.486 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.486 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.486 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.486 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.486 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.486 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.486 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.487 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.487 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.487 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.487 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.487 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.487 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.487 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.488 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.488 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.488 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.488 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.488 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.488 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.488 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.488 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.489 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.489 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.489 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.489 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.489 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.489 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.489 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.490 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.490 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.490 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.490 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.490 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.490 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.490 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.490 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.491 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.491 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.491 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.491 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.491 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.491 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mirroring", 2024-03-04T09:09:01.492 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.492 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.492 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.492 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.492 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.492 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.492 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.492 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.493 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.493 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.493 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.493 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.493 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.493 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.493 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.493 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.494 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.494 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.494 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.494 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.494 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.494 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.494 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.495 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.495 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.495 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.495 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.495 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.495 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.495 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.495 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.496 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.496 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.496 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.496 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.496 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.496 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.496 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.496 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.497 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.497 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.497 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.497 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.497 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.497 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.497 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.497 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.498 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.498 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.498 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.498 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.498 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.498 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.498 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.498 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.499 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.499 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.499 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.499 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.499 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.499 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.499 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.500 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.500 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.500 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.500 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.500 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.500 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.500 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.500 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.501 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.501 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "nfs", 2024-03-04T09:09:01.501 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.501 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.501 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.501 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.501 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.502 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.502 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.502 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.502 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.502 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.502 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.502 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.502 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.503 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.503 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.503 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.503 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.503 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.503 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.503 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.503 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.503 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.504 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.504 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.504 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.504 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.504 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.504 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.504 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.505 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.505 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.505 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.505 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.505 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.505 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.505 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.506 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.506 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.506 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.506 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.506 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.506 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.506 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.506 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.506 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.507 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.507 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.507 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.507 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.507 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.507 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.507 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.507 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.508 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.508 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.508 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.508 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.508 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.508 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.508 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.508 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.509 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.509 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.509 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.509 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.509 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.509 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.509 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.509 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.510 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.510 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.510 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "orchestrator", 2024-03-04T09:09:01.510 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.510 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.510 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.511 INFO:teuthology.orchestra.run.smithi071.stdout: "fail_fs": { 2024-03-04T09:09:01.511 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "fail_fs", 2024-03-04T09:09:01.511 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.511 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.511 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.511 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.511 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.511 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.511 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.512 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-04T09:09:01.512 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.512 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.512 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.512 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.512 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.512 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.513 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.513 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.513 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.513 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.513 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.513 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.513 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.513 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.513 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.514 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.514 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.514 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.514 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.514 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.514 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.514 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.514 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.515 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.515 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.515 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.515 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.515 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.515 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.515 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.516 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.516 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.516 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.516 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.516 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.516 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.516 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.517 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.517 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.517 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.517 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.517 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.517 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.517 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.517 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.518 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.518 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.518 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.518 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.518 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.518 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.518 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.518 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.519 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.519 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.519 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.519 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.519 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.519 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.519 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.519 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.520 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.520 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.520 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.520 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.520 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.520 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.520 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.520 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.521 INFO:teuthology.orchestra.run.smithi071.stdout: "orchestrator": { 2024-03-04T09:09:01.521 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "orchestrator", 2024-03-04T09:09:01.521 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.521 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.521 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.521 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.521 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.522 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.522 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.522 INFO:teuthology.orchestra.run.smithi071.stdout: "cephadm", 2024-03-04T09:09:01.522 INFO:teuthology.orchestra.run.smithi071.stdout: "rook", 2024-03-04T09:09:01.522 INFO:teuthology.orchestra.run.smithi071.stdout: "test_orchestrator" 2024-03-04T09:09:01.522 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.522 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Orchestrator backend", 2024-03-04T09:09:01.522 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.523 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.523 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.523 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.523 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.523 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.523 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "osd_perf_query", 2024-03-04T09:09:01.523 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.524 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.524 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.524 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.524 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.524 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.524 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.524 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.524 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.525 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.525 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.525 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.525 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.525 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.525 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.525 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.525 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.526 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.526 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.526 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.526 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.526 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.526 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.526 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.527 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.527 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.527 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.527 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.527 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.527 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.527 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.527 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.528 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.528 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.528 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.528 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.528 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.528 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.528 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.529 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.529 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.529 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.529 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.529 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.529 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.529 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.529 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.529 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.530 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.530 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.530 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.530 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.530 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.530 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.530 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.530 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.531 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.531 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.531 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.531 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.531 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.531 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.531 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.531 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.532 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.532 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.532 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.532 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.532 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.532 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.532 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.533 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "osd_support", 2024-03-04T09:09:01.533 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.533 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.533 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.533 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.533 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.533 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.534 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.534 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.534 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.534 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.534 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.534 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.534 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.534 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.534 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.535 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.535 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.535 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.535 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.535 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.535 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.535 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.535 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.536 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.536 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.536 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.536 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.536 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.536 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.536 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.536 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.537 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.537 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.537 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.537 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.539 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.539 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.539 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.539 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.540 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.540 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.540 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.540 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.540 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.540 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.540 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.540 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.540 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.541 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.541 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.541 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.541 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.541 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.541 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.541 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.541 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.542 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.542 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.542 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.542 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.542 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.542 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.542 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.543 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.543 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.543 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.543 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.543 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.543 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.543 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.543 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.544 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "pg_autoscaler", 2024-03-04T09:09:01.544 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.544 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.544 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.544 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.544 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.544 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.545 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.545 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.545 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.545 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.545 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.545 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.545 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.545 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.546 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.546 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.546 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.546 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.546 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.546 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.546 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.546 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.546 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.547 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.547 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.547 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.547 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.547 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.547 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.547 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.548 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.548 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.548 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.548 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.548 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.548 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.548 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.548 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.549 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.549 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.549 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.549 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.549 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.549 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.549 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.549 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.550 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.550 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.550 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.550 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.550 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.550 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.550 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.550 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.551 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.551 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.551 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.551 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.551 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.551 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.551 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.551 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.552 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.552 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.552 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.552 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.552 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.552 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.552 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.553 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-03-04T09:09:01.553 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-03-04T09:09:01.553 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.553 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.553 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.553 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-03-04T09:09:01.553 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.553 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.553 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.554 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.554 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.554 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.554 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.554 INFO:teuthology.orchestra.run.smithi071.stdout: "threshold": { 2024-03-04T09:09:01.554 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "threshold", 2024-03-04T09:09:01.554 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-03-04T09:09:01.555 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.555 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.555 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "3.0", 2024-03-04T09:09:01.555 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1.0", 2024-03-04T09:09:01.555 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.555 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.555 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "scaling threshold", 2024-03-04T09:09:01.555 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-04T09:09:01.556 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.556 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.556 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.556 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.556 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "progress", 2024-03-04T09:09:01.556 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.557 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.557 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.557 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_pg_recovery_event": { 2024-03-04T09:09:01.557 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "allow_pg_recovery_event", 2024-03-04T09:09:01.557 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.557 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.557 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.557 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.558 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.558 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.558 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.558 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-04T09:09:01.558 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.558 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.558 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.559 INFO:teuthology.orchestra.run.smithi071.stdout: "enabled": { 2024-03-04T09:09:01.559 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "enabled", 2024-03-04T09:09:01.559 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.559 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.559 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.559 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.559 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.559 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.559 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.560 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.560 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.560 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.560 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.560 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.560 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.560 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.561 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.561 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.561 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.561 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.561 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.561 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.561 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.561 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.561 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.562 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.562 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.562 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.562 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.562 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.562 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.562 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.562 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.563 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.563 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.563 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.563 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.563 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.563 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.563 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.564 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.564 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.564 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.564 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.564 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.564 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.564 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.564 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.565 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.565 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.565 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.565 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.565 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.565 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.565 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.565 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.566 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.566 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.566 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.566 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.566 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.566 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.566 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.566 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.566 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.567 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.567 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.567 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.567 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.567 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.567 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.567 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.567 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.568 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.568 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.568 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.568 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.568 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.568 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.568 INFO:teuthology.orchestra.run.smithi071.stdout: "max_completed_events": { 2024-03-04T09:09:01.569 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_completed_events", 2024-03-04T09:09:01.569 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.569 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.569 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.569 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "50", 2024-03-04T09:09:01.569 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.569 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.569 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.569 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "number of past completed events to remember", 2024-03-04T09:09:01.570 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.570 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.570 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.570 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-03-04T09:09:01.570 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-03-04T09:09:01.570 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.570 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.571 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.571 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5", 2024-03-04T09:09:01.571 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.571 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.571 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.571 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how long the module is going to sleep", 2024-03-04T09:09:01.571 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.571 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.571 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.572 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.572 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.572 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "prometheus", 2024-03-04T09:09:01.572 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.572 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.572 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.573 INFO:teuthology.orchestra.run.smithi071.stdout: "cache": { 2024-03-04T09:09:01.573 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cache", 2024-03-04T09:09:01.573 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.573 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.573 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.573 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.573 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.573 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.573 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.574 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.574 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.574 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.574 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.574 INFO:teuthology.orchestra.run.smithi071.stdout: "exclude_perf_counters": { 2024-03-04T09:09:01.574 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "exclude_perf_counters", 2024-03-04T09:09:01.574 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.575 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.575 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.575 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.575 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.575 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.575 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.575 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-04T09:09:01.575 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-04T09:09:01.575 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.576 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.576 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.576 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.576 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.576 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.576 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.576 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.577 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.577 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.577 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.577 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.577 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.577 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.577 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.577 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.578 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.578 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.578 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.578 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.578 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.578 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.578 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.578 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.578 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.579 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.579 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.579 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.579 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.579 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.579 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.579 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.579 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.580 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.580 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.580 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.580 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.580 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.580 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.580 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.581 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.581 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.581 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.581 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.581 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.581 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.581 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.581 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.581 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.582 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.582 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.582 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.582 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.582 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.582 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.582 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.582 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.583 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.583 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.583 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.583 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.583 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.583 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.583 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.583 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.584 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.584 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.584 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.584 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.584 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_stats_pools": { 2024-03-04T09:09:01.584 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rbd_stats_pools", 2024-03-04T09:09:01.584 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.584 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.585 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.585 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.585 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.585 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.585 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.585 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.585 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.585 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.586 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.586 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.586 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-04T09:09:01.586 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-04T09:09:01.586 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.586 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.586 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.586 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "300", 2024-03-04T09:09:01.586 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.587 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.587 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.587 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.587 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.587 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.587 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.587 INFO:teuthology.orchestra.run.smithi071.stdout: "scrape_interval": { 2024-03-04T09:09:01.587 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "scrape_interval", 2024-03-04T09:09:01.588 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-03-04T09:09:01.588 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.588 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.588 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "15.0", 2024-03-04T09:09:01.588 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.588 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.588 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.588 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.589 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.589 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.589 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.589 INFO:teuthology.orchestra.run.smithi071.stdout: "server_addr": { 2024-03-04T09:09:01.589 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_addr", 2024-03-04T09:09:01.589 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.589 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.590 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.590 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "::", 2024-03-04T09:09:01.590 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.590 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.590 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.590 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-04T09:09:01.590 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.590 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.591 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.591 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.591 INFO:teuthology.orchestra.run.smithi071.stdout: "server_port": { 2024-03-04T09:09:01.591 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_port", 2024-03-04T09:09:01.591 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.591 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.591 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.591 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "9283", 2024-03-04T09:09:01.592 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.592 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.592 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.592 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-04T09:09:01.592 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.592 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.592 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.592 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.593 INFO:teuthology.orchestra.run.smithi071.stdout: "stale_cache_strategy": { 2024-03-04T09:09:01.593 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "stale_cache_strategy", 2024-03-04T09:09:01.593 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.593 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.593 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.593 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "log", 2024-03-04T09:09:01.593 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.593 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.593 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.594 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.594 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.594 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.594 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.594 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.594 INFO:teuthology.orchestra.run.smithi071.stdout: "standby_behaviour": { 2024-03-04T09:09:01.594 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "standby_behaviour", 2024-03-04T09:09:01.594 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.595 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.595 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.595 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "default", 2024-03-04T09:09:01.595 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.595 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.595 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.595 INFO:teuthology.orchestra.run.smithi071.stdout: "default", 2024-03-04T09:09:01.595 INFO:teuthology.orchestra.run.smithi071.stdout: "error" 2024-03-04T09:09:01.596 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.596 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.596 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.596 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.596 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.596 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.596 INFO:teuthology.orchestra.run.smithi071.stdout: "standby_error_status_code": { 2024-03-04T09:09:01.596 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "standby_error_status_code", 2024-03-04T09:09:01.597 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.597 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.597 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.597 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "500", 2024-03-04T09:09:01.597 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "400", 2024-03-04T09:09:01.597 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "599", 2024-03-04T09:09:01.597 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.597 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.597 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.598 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.598 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.598 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.598 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.598 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.598 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.598 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rbd_support", 2024-03-04T09:09:01.598 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.598 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.599 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.599 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.599 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.599 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.599 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.599 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.599 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.599 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.599 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.600 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.600 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.600 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.600 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.600 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.600 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.600 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.600 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.601 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.601 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.601 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.601 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.601 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.601 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.601 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.601 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.602 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.602 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.602 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.602 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.602 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.602 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.602 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.602 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.603 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.603 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.603 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.603 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.603 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.603 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.603 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.603 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.603 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.604 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.604 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.604 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.604 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.604 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.604 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.604 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.604 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.605 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.605 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.605 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.605 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.605 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.605 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.605 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.605 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.606 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.606 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.606 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.606 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.606 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.606 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.606 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.606 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.606 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.607 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.607 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.607 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.607 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.607 INFO:teuthology.orchestra.run.smithi071.stdout: "max_concurrent_snap_create": { 2024-03-04T09:09:01.607 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_concurrent_snap_create", 2024-03-04T09:09:01.607 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.607 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.608 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.608 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-03-04T09:09:01.608 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.608 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.608 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.608 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.608 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.608 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.609 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.609 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.609 INFO:teuthology.orchestra.run.smithi071.stdout: "mirror_snapshot_schedule": { 2024-03-04T09:09:01.609 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mirror_snapshot_schedule", 2024-03-04T09:09:01.609 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.609 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.609 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.609 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.609 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.610 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.610 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.610 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.610 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.610 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.610 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.610 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.610 INFO:teuthology.orchestra.run.smithi071.stdout: "trash_purge_schedule": { 2024-03-04T09:09:01.610 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "trash_purge_schedule", 2024-03-04T09:09:01.611 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.611 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.611 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.611 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.611 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.611 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.611 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.611 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.612 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.612 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.612 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.612 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.612 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.612 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.612 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.612 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "restful", 2024-03-04T09:09:01.613 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.613 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.613 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.613 INFO:teuthology.orchestra.run.smithi071.stdout: "enable_auth": { 2024-03-04T09:09:01.613 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "enable_auth", 2024-03-04T09:09:01.613 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.613 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.613 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.614 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.614 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.614 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.614 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.614 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.614 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.614 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.614 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.615 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.615 INFO:teuthology.orchestra.run.smithi071.stdout: "key_file": { 2024-03-04T09:09:01.615 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "key_file", 2024-03-04T09:09:01.615 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.615 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.615 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.615 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.615 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.615 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.616 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.616 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.616 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.616 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.616 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.616 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.616 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.616 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.617 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.617 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.617 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.617 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.617 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.617 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.617 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.617 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.617 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.618 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.618 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.618 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.618 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.618 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.618 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.618 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.618 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.619 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.619 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.619 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.619 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.619 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.619 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.619 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.619 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.620 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.620 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.620 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.620 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.620 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.620 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.620 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.621 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.621 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.621 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.621 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.621 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.621 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.621 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.621 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.622 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.622 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.622 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.622 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.622 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.622 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.622 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.622 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.622 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.623 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.623 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.623 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.623 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.623 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.623 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.623 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.623 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.624 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.624 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.624 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.624 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.624 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.624 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.624 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.624 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.624 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.625 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.625 INFO:teuthology.orchestra.run.smithi071.stdout: "server_addr": { 2024-03-04T09:09:01.625 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_addr", 2024-03-04T09:09:01.625 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.625 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.625 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.625 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.625 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.626 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.626 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.626 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.626 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.626 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.626 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.626 INFO:teuthology.orchestra.run.smithi071.stdout: "server_port": { 2024-03-04T09:09:01.627 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_port", 2024-03-04T09:09:01.627 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.627 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.627 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.627 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.627 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.627 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.627 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.627 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.628 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.628 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.628 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.628 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.628 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.628 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.628 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rgw", 2024-03-04T09:09:01.629 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.629 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.629 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.629 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.629 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.629 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.629 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.629 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.630 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.630 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.630 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.630 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.630 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.630 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.630 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.630 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.631 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.631 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.631 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.631 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.631 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.631 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.631 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.631 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.631 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.632 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.632 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.632 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.632 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.632 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.632 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.632 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.632 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.633 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.633 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.633 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.633 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.633 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.633 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.633 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.633 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.634 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.634 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.634 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.634 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.634 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.634 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.634 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.634 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.635 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.635 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.635 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.635 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.635 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.635 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.635 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.635 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.636 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.636 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.636 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.636 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.636 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.636 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.636 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.636 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.636 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.637 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.637 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.637 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.637 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.637 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.637 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.637 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.638 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rook", 2024-03-04T09:09:01.638 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.638 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.638 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.638 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.638 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.638 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.638 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.639 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.639 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.639 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.639 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.639 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.639 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.639 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.639 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.639 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.639 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.640 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.640 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.640 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.640 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.640 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.640 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.640 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.641 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.641 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.641 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.641 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.641 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.641 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.641 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.641 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.641 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.642 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.642 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.642 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.642 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.642 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.642 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.642 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.642 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.643 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.643 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.643 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.643 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.643 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.643 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.643 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.643 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.643 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.644 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.644 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.644 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.644 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.644 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.644 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.644 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.644 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.645 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.645 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.645 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.645 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.645 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.645 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.645 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.645 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.646 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.646 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.646 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.646 INFO:teuthology.orchestra.run.smithi071.stdout: "storage_class": { 2024-03-04T09:09:01.646 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "storage_class", 2024-03-04T09:09:01.646 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.646 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.646 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.647 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "local", 2024-03-04T09:09:01.647 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.647 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.647 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.647 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-04T09:09:01.647 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.647 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.647 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.647 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.647 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.648 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "selftest", 2024-03-04T09:09:01.648 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.648 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.648 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.648 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.648 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.649 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.649 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.649 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.649 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.649 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.649 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.649 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.649 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.650 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.650 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.650 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.650 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.650 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.650 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.650 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.650 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.650 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.651 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.651 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.651 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.651 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.651 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.651 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.651 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.651 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.652 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.652 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.652 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.652 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.652 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.652 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.652 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.653 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.653 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.653 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.653 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.653 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.653 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.653 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.653 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.653 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.654 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.654 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.654 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.654 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.654 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.654 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.654 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.654 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.655 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.655 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.655 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.655 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.655 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.655 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.655 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.656 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.656 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.656 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.656 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.656 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.656 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.656 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.656 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.657 INFO:teuthology.orchestra.run.smithi071.stdout: "roption1": { 2024-03-04T09:09:01.657 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "roption1", 2024-03-04T09:09:01.657 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.657 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.657 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.657 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.657 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.657 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.658 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.658 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.658 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.658 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.658 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.658 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.658 INFO:teuthology.orchestra.run.smithi071.stdout: "roption2": { 2024-03-04T09:09:01.658 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "roption2", 2024-03-04T09:09:01.658 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.659 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.659 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.659 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "xyz", 2024-03-04T09:09:01.659 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.659 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.659 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.659 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.659 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.659 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.660 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.660 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.660 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption1": { 2024-03-04T09:09:01.660 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption1", 2024-03-04T09:09:01.660 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.660 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.660 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.660 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.661 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.661 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.661 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.661 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.661 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.661 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.661 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.661 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.662 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption2": { 2024-03-04T09:09:01.662 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption2", 2024-03-04T09:09:01.662 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.662 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.662 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.662 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.662 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.662 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.662 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.663 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.663 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.663 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.663 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.663 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption3": { 2024-03-04T09:09:01.663 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption3", 2024-03-04T09:09:01.663 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-03-04T09:09:01.664 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.664 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.664 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.664 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.664 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.664 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.664 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.664 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.664 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.665 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.665 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption4": { 2024-03-04T09:09:01.665 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption4", 2024-03-04T09:09:01.665 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.665 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.665 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.665 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.666 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.666 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.666 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.666 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.666 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.666 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.666 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.666 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption5": { 2024-03-04T09:09:01.667 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption5", 2024-03-04T09:09:01.667 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.667 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.667 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.667 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.667 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.667 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.667 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.668 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.668 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.668 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.668 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.668 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.668 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption6": { 2024-03-04T09:09:01.668 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption6", 2024-03-04T09:09:01.668 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.668 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.669 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.669 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.669 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.669 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.669 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.669 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.669 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.669 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.670 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.670 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption7": { 2024-03-04T09:09:01.670 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption7", 2024-03-04T09:09:01.670 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.670 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.670 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.670 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.670 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1", 2024-03-04T09:09:01.671 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "42", 2024-03-04T09:09:01.671 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.671 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.671 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.671 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.671 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.671 INFO:teuthology.orchestra.run.smithi071.stdout: "testkey": { 2024-03-04T09:09:01.672 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "testkey", 2024-03-04T09:09:01.672 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.672 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.672 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.672 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.672 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.672 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.672 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.673 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.673 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.673 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.673 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.673 INFO:teuthology.orchestra.run.smithi071.stdout: "testlkey": { 2024-03-04T09:09:01.673 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "testlkey", 2024-03-04T09:09:01.673 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.674 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.674 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.674 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.674 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.674 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.674 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.674 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.674 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.675 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.675 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.675 INFO:teuthology.orchestra.run.smithi071.stdout: "testnewline": { 2024-03-04T09:09:01.675 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "testnewline", 2024-03-04T09:09:01.675 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.675 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.675 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.675 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.676 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.676 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.676 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.676 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.676 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.676 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.676 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.676 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.677 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.677 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.677 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.677 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "snap_schedule", 2024-03-04T09:09:01.677 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.677 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.677 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.677 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_m_granularity": { 2024-03-04T09:09:01.678 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "allow_m_granularity", 2024-03-04T09:09:01.678 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.678 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.678 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.678 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.678 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.678 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.678 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.678 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "allow minute scheduled snapshots", 2024-03-04T09:09:01.679 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.679 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.679 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.679 INFO:teuthology.orchestra.run.smithi071.stdout: "dump_on_update": { 2024-03-04T09:09:01.679 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "dump_on_update", 2024-03-04T09:09:01.679 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.679 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.680 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.680 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.680 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.680 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.680 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.680 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "dump database to debug log on update", 2024-03-04T09:09:01.680 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.680 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.681 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.681 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.681 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.681 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.681 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.681 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.681 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.682 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.682 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.682 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.682 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.682 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.682 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.682 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.682 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.682 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.683 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.683 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.683 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.683 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.683 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.683 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.683 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.683 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.683 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.684 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.684 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.684 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.684 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.684 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.684 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.684 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.684 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.685 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.685 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.685 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.685 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.685 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.685 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.685 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.685 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.685 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.686 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.686 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.686 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.686 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.686 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.686 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.686 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.686 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.687 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.687 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.687 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.687 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.687 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.687 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.687 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.687 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.687 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.688 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.688 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.688 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.688 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.688 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.688 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.688 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.688 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.689 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.689 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.689 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.689 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.689 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.689 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.689 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.689 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "stats", 2024-03-04T09:09:01.689 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.690 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.690 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.690 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.690 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.690 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.690 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.690 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.690 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.690 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.691 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.691 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.691 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.691 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.691 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.691 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.691 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.691 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.692 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.692 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.692 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.692 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.692 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.692 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.692 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.692 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.693 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.693 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.693 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.693 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.693 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.693 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.693 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.693 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.693 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.694 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.694 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.694 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.694 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.694 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.694 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.694 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.694 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.695 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.695 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.695 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.695 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.695 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.695 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.695 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.695 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.695 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.696 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.696 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.696 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.696 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.696 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.696 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.696 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.696 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.697 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.697 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.697 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.697 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.697 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.697 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.697 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.697 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.697 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.698 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.698 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.698 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.698 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.698 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.698 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.698 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.698 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "status", 2024-03-04T09:09:01.698 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.699 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.699 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.699 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.699 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.699 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.699 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.699 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.699 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.700 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.700 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.700 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.700 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.700 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.700 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.700 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.700 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.700 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.701 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.701 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.701 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.701 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.701 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.701 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.701 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.701 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.702 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.702 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.702 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.702 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.702 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.702 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.702 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.702 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.702 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.703 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.703 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.703 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.703 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.703 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.703 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.703 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.703 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.703 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.704 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.704 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.704 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.704 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.704 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.704 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.704 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.704 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.705 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.705 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.705 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.705 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.705 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.705 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.705 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.705 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.705 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.706 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.706 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.706 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.706 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.706 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.706 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.706 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.706 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.707 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.707 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.707 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.707 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.707 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.707 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.707 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "telegraf", 2024-03-04T09:09:01.708 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.708 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.708 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.708 INFO:teuthology.orchestra.run.smithi071.stdout: "address": { 2024-03-04T09:09:01.708 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "address", 2024-03-04T09:09:01.708 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.708 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.708 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.709 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-04T09:09:01.709 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.709 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.709 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.709 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.709 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.709 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.709 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.710 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-03-04T09:09:01.710 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-03-04T09:09:01.710 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.710 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.710 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.710 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "15", 2024-03-04T09:09:01.710 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.710 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.711 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.711 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.711 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.711 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.711 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.711 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.711 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.711 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.712 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.712 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.712 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.712 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.712 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.712 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.712 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.712 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.713 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.713 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.713 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.713 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.713 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.713 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.713 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.713 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.713 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.714 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.714 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.714 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.714 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.714 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.714 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.715 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.715 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.715 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.715 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.715 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.715 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.715 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.715 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.715 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.716 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.716 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.716 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.716 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.716 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.716 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.716 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.716 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.717 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.717 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.717 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.717 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.717 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.717 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.717 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.717 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.718 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.718 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.718 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.718 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.718 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.718 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.718 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.718 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.719 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.719 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.719 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.719 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.719 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.719 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.719 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.719 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.720 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.720 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.720 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.720 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.720 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.720 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "telemetry", 2024-03-04T09:09:01.720 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.720 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.720 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.721 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_basic": { 2024-03-04T09:09:01.721 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_basic", 2024-03-04T09:09:01.721 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.721 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.721 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.721 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.721 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.721 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.722 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.722 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-04T09:09:01.722 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.722 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.722 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.722 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_crash": { 2024-03-04T09:09:01.722 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_crash", 2024-03-04T09:09:01.722 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.723 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.723 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.723 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.723 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.723 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.723 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.723 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-04T09:09:01.723 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.724 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.724 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.724 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.724 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_device": { 2024-03-04T09:09:01.724 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_device", 2024-03-04T09:09:01.724 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.724 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.724 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.724 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.725 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.725 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.725 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.725 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-04T09:09:01.725 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.725 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.725 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.725 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.725 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_ident": { 2024-03-04T09:09:01.726 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_ident", 2024-03-04T09:09:01.726 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.726 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.726 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.726 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.726 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.726 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.726 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.727 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-04T09:09:01.727 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.727 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.727 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.727 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_perf": { 2024-03-04T09:09:01.727 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_perf", 2024-03-04T09:09:01.727 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.728 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.728 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.728 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.728 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.728 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.728 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.728 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-04T09:09:01.728 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.728 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.729 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.729 INFO:teuthology.orchestra.run.smithi071.stdout: "contact": { 2024-03-04T09:09:01.729 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "contact", 2024-03-04T09:09:01.729 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.729 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.729 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.729 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.730 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.730 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.730 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.730 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.730 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.730 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.730 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.730 INFO:teuthology.orchestra.run.smithi071.stdout: "description": { 2024-03-04T09:09:01.731 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "description", 2024-03-04T09:09:01.731 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.731 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.731 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.731 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.731 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.731 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.731 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.731 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.732 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.732 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.732 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.732 INFO:teuthology.orchestra.run.smithi071.stdout: "device_url": { 2024-03-04T09:09:01.732 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "device_url", 2024-03-04T09:09:01.732 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.733 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.733 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.733 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-04T09:09:01.733 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.733 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.733 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.733 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.733 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.733 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.734 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.734 INFO:teuthology.orchestra.run.smithi071.stdout: "enabled": { 2024-03-04T09:09:01.734 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "enabled", 2024-03-04T09:09:01.734 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.734 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.734 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.734 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.734 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.735 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.735 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.735 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.735 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.735 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.735 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.735 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-03-04T09:09:01.736 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-03-04T09:09:01.736 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.736 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.736 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.736 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "24", 2024-03-04T09:09:01.736 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "8", 2024-03-04T09:09:01.736 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.736 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.737 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.737 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.737 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.737 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.737 INFO:teuthology.orchestra.run.smithi071.stdout: "last_opt_revision": { 2024-03-04T09:09:01.737 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "last_opt_revision", 2024-03-04T09:09:01.737 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.737 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.737 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.738 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "1", 2024-03-04T09:09:01.738 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.738 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.738 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.738 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.738 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.738 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.738 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.739 INFO:teuthology.orchestra.run.smithi071.stdout: "leaderboard": { 2024-03-04T09:09:01.739 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "leaderboard", 2024-03-04T09:09:01.739 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.739 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.739 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.739 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.739 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.739 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.739 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.739 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.740 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.740 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.740 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.740 INFO:teuthology.orchestra.run.smithi071.stdout: "leaderboard_description": { 2024-03-04T09:09:01.740 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "leaderboard_description", 2024-03-04T09:09:01.740 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.741 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.741 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.741 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.741 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.741 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.743 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.743 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.743 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.744 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.744 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.744 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.744 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.744 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.744 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.744 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.745 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.745 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.745 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.745 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.745 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.745 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.745 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.745 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.745 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.746 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.746 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.746 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.746 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.746 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.746 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.746 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.747 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.747 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.747 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.747 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.747 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.747 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.747 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.747 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.747 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.748 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.748 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.748 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.748 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.748 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.748 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.749 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.749 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.749 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.749 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.749 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.749 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.749 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.749 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.750 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.750 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.750 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.750 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.750 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.750 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.750 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.750 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.750 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.751 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.751 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.751 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.751 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.751 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.751 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.751 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.751 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.752 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.752 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.752 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.752 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.752 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.752 INFO:teuthology.orchestra.run.smithi071.stdout: "organization": { 2024-03-04T09:09:01.752 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "organization", 2024-03-04T09:09:01.753 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.753 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.753 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.753 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.753 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.753 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.753 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.753 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.753 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.754 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.754 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.754 INFO:teuthology.orchestra.run.smithi071.stdout: "proxy": { 2024-03-04T09:09:01.754 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "proxy", 2024-03-04T09:09:01.754 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.754 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.754 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.755 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.755 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.755 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.755 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.755 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.755 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.755 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.755 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.756 INFO:teuthology.orchestra.run.smithi071.stdout: "url": { 2024-03-04T09:09:01.756 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "url", 2024-03-04T09:09:01.756 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.756 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.756 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.756 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-04T09:09:01.756 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.756 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.757 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.757 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.757 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.757 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.757 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.757 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.757 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.758 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "test_orchestrator", 2024-03-04T09:09:01.758 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.758 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.758 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.758 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.758 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.758 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.758 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.759 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.759 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.759 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.759 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.759 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.759 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.759 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.759 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.760 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.760 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.760 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.760 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.760 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.760 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.760 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.760 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.761 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.761 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.761 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.761 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.761 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.761 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.761 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.762 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.762 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.762 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.762 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.762 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.762 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.762 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.762 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.763 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.763 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.763 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.763 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.763 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.763 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.763 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.763 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.764 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.764 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.764 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.764 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.764 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.764 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.764 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.764 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.765 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.765 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.765 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.765 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.765 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.765 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.765 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.766 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.766 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.766 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.766 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.766 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.766 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.766 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.766 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.767 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.767 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.767 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "volumes", 2024-03-04T09:09:01.767 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.767 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.767 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.767 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.768 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.768 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.768 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.768 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.768 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.768 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.768 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.768 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.769 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.769 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.769 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.769 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.769 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.769 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.769 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.770 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.770 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.770 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.770 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.770 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.770 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.770 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.770 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.770 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.771 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.771 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.771 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.771 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.771 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.771 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.771 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.771 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.772 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.772 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.772 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.772 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.772 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.772 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.772 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.772 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.773 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.773 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.773 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.773 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.773 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.773 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.773 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.773 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.773 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.774 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.774 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.774 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.774 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.774 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.774 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.774 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.775 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.775 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.775 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.775 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.775 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.775 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.775 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.775 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.776 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.776 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.776 INFO:teuthology.orchestra.run.smithi071.stdout: "max_concurrent_clones": { 2024-03-04T09:09:01.776 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_concurrent_clones", 2024-03-04T09:09:01.776 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.776 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.776 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.777 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "4", 2024-03-04T09:09:01.777 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.777 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.777 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.777 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-04T09:09:01.777 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.777 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.777 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.778 INFO:teuthology.orchestra.run.smithi071.stdout: "periodic_async_work": { 2024-03-04T09:09:01.778 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "periodic_async_work", 2024-03-04T09:09:01.778 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.778 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.778 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.778 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.778 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.778 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.778 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.778 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Periodically check for async work", 2024-03-04T09:09:01.779 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.779 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.779 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.779 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot_clone_delay": { 2024-03-04T09:09:01.779 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "snapshot_clone_delay", 2024-03-04T09:09:01.779 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.779 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.779 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.780 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-03-04T09:09:01.780 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.780 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.780 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.780 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-04T09:09:01.780 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.780 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.780 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.781 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot_clone_no_wait": { 2024-03-04T09:09:01.781 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "snapshot_clone_no_wait", 2024-03-04T09:09:01.781 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.781 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.781 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.781 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.781 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.781 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.781 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.782 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-04T09:09:01.782 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.782 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.782 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.782 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.782 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.783 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "zabbix", 2024-03-04T09:09:01.783 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.783 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.783 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.783 INFO:teuthology.orchestra.run.smithi071.stdout: "discovery_interval": { 2024-03-04T09:09:01.783 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "discovery_interval", 2024-03-04T09:09:01.783 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-03-04T09:09:01.783 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.783 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.784 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "100", 2024-03-04T09:09:01.784 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.784 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.784 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.784 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.784 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.784 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.785 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.785 INFO:teuthology.orchestra.run.smithi071.stdout: "identifier": { 2024-03-04T09:09:01.785 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "identifier", 2024-03-04T09:09:01.785 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.785 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.785 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.785 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.785 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.786 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.786 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.786 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.786 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.786 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.786 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.786 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-03-04T09:09:01.787 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-03-04T09:09:01.787 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.787 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.787 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.787 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-03-04T09:09:01.787 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.787 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.787 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.788 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.788 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.788 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.788 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.788 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.788 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.788 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.789 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.789 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.789 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.789 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.789 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.789 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.789 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.789 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.790 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.790 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.790 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.790 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.790 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.790 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.790 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.790 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.790 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.791 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.791 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.791 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.791 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.791 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.791 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.791 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.792 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.792 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.792 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.792 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.792 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.792 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.792 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.793 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.793 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.793 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.793 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.793 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.793 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.793 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.793 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.793 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.794 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.794 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.794 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.794 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.794 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.794 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.794 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.794 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.794 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.795 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.795 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.795 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.795 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.795 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.795 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.795 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.796 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.796 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.796 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.796 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.796 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.796 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.796 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.797 INFO:teuthology.orchestra.run.smithi071.stdout: "zabbix_host": { 2024-03-04T09:09:01.797 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "zabbix_host", 2024-03-04T09:09:01.797 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.797 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.797 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.797 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.797 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.797 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.798 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.798 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.798 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.798 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.798 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.798 INFO:teuthology.orchestra.run.smithi071.stdout: "zabbix_port": { 2024-03-04T09:09:01.798 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "zabbix_port", 2024-03-04T09:09:01.799 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.799 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.799 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.799 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10051", 2024-03-04T09:09:01.799 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.799 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.799 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.799 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.800 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.800 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.800 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.800 INFO:teuthology.orchestra.run.smithi071.stdout: "zabbix_sender": { 2024-03-04T09:09:01.800 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "zabbix_sender", 2024-03-04T09:09:01.800 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.800 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.801 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.801 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-04T09:09:01.801 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.801 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.801 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.801 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.801 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.801 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.802 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.802 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.802 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.802 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.802 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-03-04T09:09:01.802 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.802 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.802 INFO:teuthology.orchestra.run.smithi071.stdout: "modules": [ 2024-03-04T09:09:01.802 INFO:teuthology.orchestra.run.smithi071.stdout: "cephadm", 2024-03-04T09:09:01.803 INFO:teuthology.orchestra.run.smithi071.stdout: "dashboard", 2024-03-04T09:09:01.803 INFO:teuthology.orchestra.run.smithi071.stdout: "iostat", 2024-03-04T09:09:01.803 INFO:teuthology.orchestra.run.smithi071.stdout: "nfs", 2024-03-04T09:09:01.803 INFO:teuthology.orchestra.run.smithi071.stdout: "restful", 2024-03-04T09:09:01.803 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_schedule" 2024-03-04T09:09:01.803 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.803 INFO:teuthology.orchestra.run.smithi071.stdout: "available_modules": [ 2024-03-04T09:09:01.803 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.804 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "alerts", 2024-03-04T09:09:01.804 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.804 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.804 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.804 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-03-04T09:09:01.804 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-03-04T09:09:01.804 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.804 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.805 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.805 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-03-04T09:09:01.805 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.805 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.805 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.805 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "How frequently to reexamine health status", 2024-03-04T09:09:01.805 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.806 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.806 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.806 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.806 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.806 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.806 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.806 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.806 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.807 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.807 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.807 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.807 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.807 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.807 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.807 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.807 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.808 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.808 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.808 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.808 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.808 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.808 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.808 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.809 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.809 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.809 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.809 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.809 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.809 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.809 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.809 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.810 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.810 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.810 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.810 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.810 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.810 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.810 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.811 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.811 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.811 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.811 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.811 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.811 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.811 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.811 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.812 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.812 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.812 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.812 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.812 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.812 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.812 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.812 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.813 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.813 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.813 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.813 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.813 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.813 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.813 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.814 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.814 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.814 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.814 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.814 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.814 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.814 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.815 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_destination": { 2024-03-04T09:09:01.815 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_destination", 2024-03-04T09:09:01.815 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.815 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.815 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.815 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.815 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.815 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.816 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.816 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Email address to send alerts to", 2024-03-04T09:09:01.816 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.816 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.816 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.816 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_from_name": { 2024-03-04T09:09:01.816 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_from_name", 2024-03-04T09:09:01.817 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.817 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.817 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.817 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "Ceph", 2024-03-04T09:09:01.817 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.817 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.817 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.817 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Email From: name", 2024-03-04T09:09:01.817 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.818 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.818 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.818 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_host": { 2024-03-04T09:09:01.818 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_host", 2024-03-04T09:09:01.818 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.818 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.818 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.818 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.819 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.819 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.819 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.819 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "SMTP server", 2024-03-04T09:09:01.819 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.819 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.819 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.819 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_password": { 2024-03-04T09:09:01.820 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_password", 2024-03-04T09:09:01.820 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.820 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.820 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.820 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.820 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.820 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.820 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.821 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Password to authenticate with", 2024-03-04T09:09:01.821 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.821 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.821 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.821 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_port": { 2024-03-04T09:09:01.821 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_port", 2024-03-04T09:09:01.821 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.822 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.822 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.822 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "465", 2024-03-04T09:09:01.822 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.822 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.822 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.822 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "SMTP port", 2024-03-04T09:09:01.822 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.822 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.823 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.823 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_sender": { 2024-03-04T09:09:01.823 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_sender", 2024-03-04T09:09:01.823 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.823 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.823 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.823 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.823 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.824 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.824 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.824 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "SMTP envelope sender", 2024-03-04T09:09:01.824 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.824 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.824 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.824 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_ssl": { 2024-03-04T09:09:01.825 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_ssl", 2024-03-04T09:09:01.825 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.825 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.825 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.825 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.825 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.825 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.825 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.826 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-04T09:09:01.826 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.826 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.826 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.826 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_user": { 2024-03-04T09:09:01.826 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_user", 2024-03-04T09:09:01.826 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.827 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.827 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.827 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.827 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.827 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.827 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.827 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "User to authenticate as", 2024-03-04T09:09:01.827 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.827 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.828 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.828 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.828 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.828 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "balancer", 2024-03-04T09:09:01.828 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.828 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.829 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.829 INFO:teuthology.orchestra.run.smithi071.stdout: "active": { 2024-03-04T09:09:01.829 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "active", 2024-03-04T09:09:01.829 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.829 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.829 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.829 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.829 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.830 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.830 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.830 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "automatically balance PGs across cluster", 2024-03-04T09:09:01.830 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.830 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.830 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.830 INFO:teuthology.orchestra.run.smithi071.stdout: "begin_time": { 2024-03-04T09:09:01.830 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "begin_time", 2024-03-04T09:09:01.831 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.831 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.831 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.831 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0000", 2024-03-04T09:09:01.831 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.831 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.831 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.831 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "beginning time of day to automatically balance", 2024-03-04T09:09:01.832 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-04T09:09:01.832 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.832 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.832 INFO:teuthology.orchestra.run.smithi071.stdout: "begin_weekday": { 2024-03-04T09:09:01.832 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "begin_weekday", 2024-03-04T09:09:01.832 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-03-04T09:09:01.832 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.833 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.833 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-03-04T09:09:01.833 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "0", 2024-03-04T09:09:01.833 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "6", 2024-03-04T09:09:01.833 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.833 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-04T09:09:01.833 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-04T09:09:01.833 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.833 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.834 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_compat_max_iterations": { 2024-03-04T09:09:01.834 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crush_compat_max_iterations", 2024-03-04T09:09:01.834 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-03-04T09:09:01.834 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.834 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.834 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "25", 2024-03-04T09:09:01.834 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1", 2024-03-04T09:09:01.835 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "250", 2024-03-04T09:09:01.835 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.835 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-04T09:09:01.835 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.835 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.835 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.835 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_compat_metrics": { 2024-03-04T09:09:01.836 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crush_compat_metrics", 2024-03-04T09:09:01.836 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.836 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.836 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.836 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "pgs,objects,bytes", 2024-03-04T09:09:01.836 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.836 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.836 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.837 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-04T09:09:01.837 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-04T09:09:01.837 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.837 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.837 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_compat_step": { 2024-03-04T09:09:01.837 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crush_compat_step", 2024-03-04T09:09:01.837 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-03-04T09:09:01.838 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.838 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.838 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0.5", 2024-03-04T09:09:01.838 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "0.001", 2024-03-04T09:09:01.838 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "0.999", 2024-03-04T09:09:01.838 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.838 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "aggressiveness of optimization", 2024-03-04T09:09:01.838 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-04T09:09:01.838 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.839 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.839 INFO:teuthology.orchestra.run.smithi071.stdout: "end_time": { 2024-03-04T09:09:01.839 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "end_time", 2024-03-04T09:09:01.839 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.839 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.839 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.839 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "2359", 2024-03-04T09:09:01.839 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.840 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.840 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.840 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "ending time of day to automatically balance", 2024-03-04T09:09:01.840 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-04T09:09:01.840 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.840 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.841 INFO:teuthology.orchestra.run.smithi071.stdout: "end_weekday": { 2024-03-04T09:09:01.841 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "end_weekday", 2024-03-04T09:09:01.841 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-03-04T09:09:01.841 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.841 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.841 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-03-04T09:09:01.841 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "0", 2024-03-04T09:09:01.841 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "6", 2024-03-04T09:09:01.841 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.842 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-04T09:09:01.842 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-04T09:09:01.842 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.842 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.842 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.842 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.842 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.843 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.843 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.843 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.843 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.843 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.843 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.843 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.843 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.844 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.844 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.844 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.844 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.844 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.844 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.844 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.844 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.845 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.845 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.845 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.845 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.845 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.845 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.845 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.846 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.846 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.846 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.846 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.846 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.846 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.846 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.846 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.847 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.847 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.847 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.847 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.847 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.847 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.847 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.847 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.848 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.848 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.848 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.848 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.848 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.848 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.848 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.848 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.849 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.849 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.849 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.849 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.849 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.849 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.849 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.850 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.850 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.850 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.850 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.850 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.850 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.850 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.850 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.850 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.851 INFO:teuthology.orchestra.run.smithi071.stdout: "min_score": { 2024-03-04T09:09:01.851 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "min_score", 2024-03-04T09:09:01.851 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-03-04T09:09:01.851 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.851 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.851 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-03-04T09:09:01.851 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.851 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.851 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.852 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-04T09:09:01.852 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.852 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.852 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.852 INFO:teuthology.orchestra.run.smithi071.stdout: "mode": { 2024-03-04T09:09:01.852 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mode", 2024-03-04T09:09:01.852 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.852 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.853 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.853 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "upmap", 2024-03-04T09:09:01.853 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.853 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.853 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.853 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-compat", 2024-03-04T09:09:01.853 INFO:teuthology.orchestra.run.smithi071.stdout: "none", 2024-03-04T09:09:01.853 INFO:teuthology.orchestra.run.smithi071.stdout: "read", 2024-03-04T09:09:01.853 INFO:teuthology.orchestra.run.smithi071.stdout: "upmap", 2024-03-04T09:09:01.854 INFO:teuthology.orchestra.run.smithi071.stdout: "upmap-read" 2024-03-04T09:09:01.854 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.854 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Balancer mode", 2024-03-04T09:09:01.854 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.854 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.854 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.854 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_ids": { 2024-03-04T09:09:01.855 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "pool_ids", 2024-03-04T09:09:01.855 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.855 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.855 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.855 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.855 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.855 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.855 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.856 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-04T09:09:01.856 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.856 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.856 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.856 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-03-04T09:09:01.856 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-03-04T09:09:01.856 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.856 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.856 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.857 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-03-04T09:09:01.857 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.857 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.857 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.857 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-04T09:09:01.857 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.857 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.857 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.858 INFO:teuthology.orchestra.run.smithi071.stdout: "upmap_max_deviation": { 2024-03-04T09:09:01.858 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "upmap_max_deviation", 2024-03-04T09:09:01.858 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.858 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.858 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.858 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5", 2024-03-04T09:09:01.858 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1", 2024-03-04T09:09:01.858 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.858 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.859 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-04T09:09:01.859 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-04T09:09:01.859 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.859 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.859 INFO:teuthology.orchestra.run.smithi071.stdout: "upmap_max_optimizations": { 2024-03-04T09:09:01.859 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "upmap_max_optimizations", 2024-03-04T09:09:01.859 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-03-04T09:09:01.859 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.860 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.860 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-03-04T09:09:01.860 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.860 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.860 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.860 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-04T09:09:01.860 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.860 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.860 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.861 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.861 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.861 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephadm", 2024-03-04T09:09:01.861 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.861 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.861 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.861 INFO:teuthology.orchestra.run.smithi071.stdout: "agent_down_multiplier": { 2024-03-04T09:09:01.862 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "agent_down_multiplier", 2024-03-04T09:09:01.862 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-03-04T09:09:01.862 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.862 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.862 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "3.0", 2024-03-04T09:09:01.862 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.862 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.862 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.863 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-04T09:09:01.863 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.863 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.863 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.863 INFO:teuthology.orchestra.run.smithi071.stdout: "agent_refresh_rate": { 2024-03-04T09:09:01.863 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "agent_refresh_rate", 2024-03-04T09:09:01.863 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.864 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.864 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.864 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "20", 2024-03-04T09:09:01.864 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.864 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.864 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.864 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-04T09:09:01.864 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.865 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.865 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.865 INFO:teuthology.orchestra.run.smithi071.stdout: "agent_starting_port": { 2024-03-04T09:09:01.865 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "agent_starting_port", 2024-03-04T09:09:01.865 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.865 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.865 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.866 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "4721", 2024-03-04T09:09:01.866 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.866 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.866 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.866 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-04T09:09:01.866 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.866 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.866 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.867 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_ptrace": { 2024-03-04T09:09:01.867 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "allow_ptrace", 2024-03-04T09:09:01.867 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.867 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.867 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.867 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.867 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.867 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.868 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.868 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-04T09:09:01.868 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-04T09:09:01.868 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.868 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.868 INFO:teuthology.orchestra.run.smithi071.stdout: "autotune_interval": { 2024-03-04T09:09:01.868 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "autotune_interval", 2024-03-04T09:09:01.869 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.869 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.869 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.869 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-03-04T09:09:01.869 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.869 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.869 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.869 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-04T09:09:01.870 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.870 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.870 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.870 INFO:teuthology.orchestra.run.smithi071.stdout: "autotune_memory_target_ratio": { 2024-03-04T09:09:01.870 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "autotune_memory_target_ratio", 2024-03-04T09:09:01.870 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-03-04T09:09:01.870 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.870 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.871 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0.7", 2024-03-04T09:09:01.871 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.871 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.871 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.871 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-04T09:09:01.871 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.871 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.871 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.872 INFO:teuthology.orchestra.run.smithi071.stdout: "cephadm_log_destination": { 2024-03-04T09:09:01.872 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephadm_log_destination", 2024-03-04T09:09:01.872 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.872 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.872 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.872 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.872 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.873 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.873 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.873 INFO:teuthology.orchestra.run.smithi071.stdout: "file", 2024-03-04T09:09:01.873 INFO:teuthology.orchestra.run.smithi071.stdout: "file,syslog", 2024-03-04T09:09:01.873 INFO:teuthology.orchestra.run.smithi071.stdout: "syslog" 2024-03-04T09:09:01.873 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.873 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-04T09:09:01.873 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.873 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.874 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.874 INFO:teuthology.orchestra.run.smithi071.stdout: "cgroups_split": { 2024-03-04T09:09:01.874 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cgroups_split", 2024-03-04T09:09:01.874 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.874 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.874 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.874 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.875 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.875 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.875 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.875 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-04T09:09:01.875 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.875 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.875 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.876 INFO:teuthology.orchestra.run.smithi071.stdout: "config_checks_enabled": { 2024-03-04T09:09:01.876 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "config_checks_enabled", 2024-03-04T09:09:01.876 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.876 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.876 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.876 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.876 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.876 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.876 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.877 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-04T09:09:01.877 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.877 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.877 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.877 INFO:teuthology.orchestra.run.smithi071.stdout: "config_dashboard": { 2024-03-04T09:09:01.877 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "config_dashboard", 2024-03-04T09:09:01.877 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.878 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.878 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.878 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.878 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.878 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.878 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.878 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-04T09:09:01.878 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.878 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.879 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.879 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_alertmanager": { 2024-03-04T09:09:01.879 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_alertmanager", 2024-03-04T09:09:01.879 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.879 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.879 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.879 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-04T09:09:01.880 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.880 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.880 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.880 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Prometheus container image", 2024-03-04T09:09:01.880 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.880 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.880 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.881 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_base": { 2024-03-04T09:09:01.881 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_base", 2024-03-04T09:09:01.881 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.881 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.881 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.881 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-04T09:09:01.881 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.881 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.881 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.882 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Container image name, without the tag", 2024-03-04T09:09:01.882 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.882 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.882 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.882 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_elasticsearch": { 2024-03-04T09:09:01.882 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_elasticsearch", 2024-03-04T09:09:01.882 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.882 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.883 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.883 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-04T09:09:01.883 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.883 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.883 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.883 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "elasticsearch container image", 2024-03-04T09:09:01.883 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.883 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.884 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.884 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_grafana": { 2024-03-04T09:09:01.884 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_grafana", 2024-03-04T09:09:01.884 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.884 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.884 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.884 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-04T09:09:01.884 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.885 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.885 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.885 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Prometheus container image", 2024-03-04T09:09:01.885 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.885 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.885 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.885 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_haproxy": { 2024-03-04T09:09:01.886 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_haproxy", 2024-03-04T09:09:01.886 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.886 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.886 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.886 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-04T09:09:01.886 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.886 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.886 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.886 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "HAproxy container image", 2024-03-04T09:09:01.887 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.887 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.887 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.887 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_jaeger_agent": { 2024-03-04T09:09:01.887 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_jaeger_agent", 2024-03-04T09:09:01.887 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.887 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.888 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.888 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-04T09:09:01.888 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.888 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.888 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.888 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Jaeger agent container image", 2024-03-04T09:09:01.888 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.888 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.888 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.889 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_jaeger_collector": { 2024-03-04T09:09:01.889 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_jaeger_collector", 2024-03-04T09:09:01.889 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.889 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.889 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.889 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-04T09:09:01.889 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.890 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.890 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.890 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Jaeger collector container image", 2024-03-04T09:09:01.890 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.890 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.890 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.890 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_jaeger_query": { 2024-03-04T09:09:01.891 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_jaeger_query", 2024-03-04T09:09:01.891 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.891 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.891 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.891 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-04T09:09:01.891 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.891 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.891 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.892 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Jaeger query container image", 2024-03-04T09:09:01.892 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.892 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.892 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.892 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_keepalived": { 2024-03-04T09:09:01.892 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_keepalived", 2024-03-04T09:09:01.892 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.893 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.893 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.893 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-04T09:09:01.893 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.893 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.893 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.893 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Keepalived container image", 2024-03-04T09:09:01.893 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.893 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.894 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.894 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_loki": { 2024-03-04T09:09:01.894 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_loki", 2024-03-04T09:09:01.894 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.894 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.894 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.894 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-04T09:09:01.895 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.895 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.895 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.895 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Loki container image", 2024-03-04T09:09:01.895 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.895 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.895 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.895 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_node_exporter": { 2024-03-04T09:09:01.896 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_node_exporter", 2024-03-04T09:09:01.896 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.896 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.896 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.896 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-04T09:09:01.896 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.896 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.896 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.897 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Prometheus container image", 2024-03-04T09:09:01.897 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.897 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.897 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.897 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_nvmeof": { 2024-03-04T09:09:01.897 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_nvmeof", 2024-03-04T09:09:01.897 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.897 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.898 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.898 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-04T09:09:01.898 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.898 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.898 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.898 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Nvme-of container image", 2024-03-04T09:09:01.898 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.898 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.899 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.899 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_prometheus": { 2024-03-04T09:09:01.899 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_prometheus", 2024-03-04T09:09:01.899 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.899 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.899 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.899 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-04T09:09:01.899 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.900 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.900 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.900 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Prometheus container image", 2024-03-04T09:09:01.900 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.900 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.900 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.900 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_promtail": { 2024-03-04T09:09:01.901 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_promtail", 2024-03-04T09:09:01.901 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.901 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.901 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.901 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-04T09:09:01.901 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.901 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.901 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.901 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Promtail container image", 2024-03-04T09:09:01.902 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.902 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.902 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.902 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_snmp_gateway": { 2024-03-04T09:09:01.902 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_snmp_gateway", 2024-03-04T09:09:01.902 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.902 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.903 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.903 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-04T09:09:01.903 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.903 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.903 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.903 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "SNMP Gateway container image", 2024-03-04T09:09:01.903 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.903 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.903 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.904 INFO:teuthology.orchestra.run.smithi071.stdout: "container_init": { 2024-03-04T09:09:01.904 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_init", 2024-03-04T09:09:01.904 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.904 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.904 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.904 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.905 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.905 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.905 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.905 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Run podman/docker with `--init`", 2024-03-04T09:09:01.905 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.905 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.905 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.905 INFO:teuthology.orchestra.run.smithi071.stdout: "daemon_cache_timeout": { 2024-03-04T09:09:01.906 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "daemon_cache_timeout", 2024-03-04T09:09:01.906 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.906 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.906 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.906 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-03-04T09:09:01.906 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.906 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.906 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.906 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-04T09:09:01.907 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.907 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.907 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.907 INFO:teuthology.orchestra.run.smithi071.stdout: "default_cephadm_command_timeout": { 2024-03-04T09:09:01.907 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "default_cephadm_command_timeout", 2024-03-04T09:09:01.907 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.907 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.908 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.908 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "900", 2024-03-04T09:09:01.908 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.908 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.908 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.908 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-04T09:09:01.908 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.908 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.909 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.909 INFO:teuthology.orchestra.run.smithi071.stdout: "default_registry": { 2024-03-04T09:09:01.909 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "default_registry", 2024-03-04T09:09:01.909 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.909 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.909 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.909 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "docker.io", 2024-03-04T09:09:01.910 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.910 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.910 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.910 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-04T09:09:01.910 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.910 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.910 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.910 INFO:teuthology.orchestra.run.smithi071.stdout: "device_cache_timeout": { 2024-03-04T09:09:01.911 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "device_cache_timeout", 2024-03-04T09:09:01.911 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.911 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.911 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.911 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "1800", 2024-03-04T09:09:01.911 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.911 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.911 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.912 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "seconds to cache device inventory", 2024-03-04T09:09:01.912 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.912 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.912 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.912 INFO:teuthology.orchestra.run.smithi071.stdout: "device_enhanced_scan": { 2024-03-04T09:09:01.912 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "device_enhanced_scan", 2024-03-04T09:09:01.912 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.913 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.913 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.913 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.913 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.913 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.913 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.913 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-04T09:09:01.913 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.913 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.914 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.914 INFO:teuthology.orchestra.run.smithi071.stdout: "facts_cache_timeout": { 2024-03-04T09:09:01.914 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "facts_cache_timeout", 2024-03-04T09:09:01.914 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.914 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.914 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.914 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-03-04T09:09:01.915 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.915 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.915 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.915 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "seconds to cache host facts data", 2024-03-04T09:09:01.915 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.915 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.915 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.916 INFO:teuthology.orchestra.run.smithi071.stdout: "host_check_interval": { 2024-03-04T09:09:01.916 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "host_check_interval", 2024-03-04T09:09:01.916 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.916 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.916 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.916 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-03-04T09:09:01.916 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.916 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.916 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.917 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to perform a host check", 2024-03-04T09:09:01.917 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.917 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.917 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.917 INFO:teuthology.orchestra.run.smithi071.stdout: "hw_monitoring": { 2024-03-04T09:09:01.917 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "hw_monitoring", 2024-03-04T09:09:01.917 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.918 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.918 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.918 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.918 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.918 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.918 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.918 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-04T09:09:01.918 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.918 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.919 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.919 INFO:teuthology.orchestra.run.smithi071.stdout: "inventory_list_all": { 2024-03-04T09:09:01.919 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "inventory_list_all", 2024-03-04T09:09:01.919 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.919 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.919 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.919 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.920 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.920 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.920 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.920 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-04T09:09:01.920 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.920 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.920 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.920 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.921 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.921 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.921 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.921 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.921 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.921 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.921 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.921 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.921 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.922 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.922 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.922 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.922 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.922 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.922 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.922 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.922 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.923 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.923 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.923 INFO:teuthology.orchestra.run.smithi071.stdout: "log_refresh_metadata": { 2024-03-04T09:09:01.923 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_refresh_metadata", 2024-03-04T09:09:01.923 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.923 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.923 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.923 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.924 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.924 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.924 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.924 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-04T09:09:01.924 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.924 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.924 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.925 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.925 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.925 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.925 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.925 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.925 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.925 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.925 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.925 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.926 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-04T09:09:01.926 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.926 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.926 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.926 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.926 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.926 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.926 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.927 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.927 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.927 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.927 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.927 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.927 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.927 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.927 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.928 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.928 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.928 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.928 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.928 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.928 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.928 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.928 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.929 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.929 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.929 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.929 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.929 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.929 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.929 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.929 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.929 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.930 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.930 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.930 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.930 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.930 INFO:teuthology.orchestra.run.smithi071.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-04T09:09:01.930 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-04T09:09:01.930 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.931 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.931 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.931 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.931 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.931 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.931 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.931 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-04T09:09:01.931 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.932 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.932 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.932 INFO:teuthology.orchestra.run.smithi071.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-04T09:09:01.932 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-04T09:09:01.932 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.932 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.932 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.933 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "*", 2024-03-04T09:09:01.933 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.933 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.933 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.933 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-04T09:09:01.933 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.933 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.933 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.934 INFO:teuthology.orchestra.run.smithi071.stdout: "max_count_per_host": { 2024-03-04T09:09:01.934 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_count_per_host", 2024-03-04T09:09:01.934 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.934 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.934 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.934 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-03-04T09:09:01.934 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.935 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.935 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.935 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "max number of daemons per service per host", 2024-03-04T09:09:01.935 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.935 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.935 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.935 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd_draining_count": { 2024-03-04T09:09:01.936 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_osd_draining_count", 2024-03-04T09:09:01.936 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.936 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.936 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.936 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-03-04T09:09:01.936 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.936 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.936 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.937 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-04T09:09:01.937 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.937 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.937 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.937 INFO:teuthology.orchestra.run.smithi071.stdout: "migration_current": { 2024-03-04T09:09:01.937 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "migration_current", 2024-03-04T09:09:01.937 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.937 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.938 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.938 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.938 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.938 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.938 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.938 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "internal - do not modify", 2024-03-04T09:09:01.938 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.938 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.939 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.939 INFO:teuthology.orchestra.run.smithi071.stdout: "mode": { 2024-03-04T09:09:01.939 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mode", 2024-03-04T09:09:01.939 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.939 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.939 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.939 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "root", 2024-03-04T09:09:01.940 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.940 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.940 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.940 INFO:teuthology.orchestra.run.smithi071.stdout: "cephadm-package", 2024-03-04T09:09:01.940 INFO:teuthology.orchestra.run.smithi071.stdout: "root" 2024-03-04T09:09:01.940 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.940 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "mode for remote execution of cephadm", 2024-03-04T09:09:01.940 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.941 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.941 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.941 INFO:teuthology.orchestra.run.smithi071.stdout: "oob_default_addr": { 2024-03-04T09:09:01.941 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "oob_default_addr", 2024-03-04T09:09:01.941 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.941 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.941 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.941 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "169.254.1.1", 2024-03-04T09:09:01.942 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.942 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.942 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.942 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-04T09:09:01.942 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.942 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.942 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.942 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.942 INFO:teuthology.orchestra.run.smithi071.stdout: "prometheus_alerts_path": { 2024-03-04T09:09:01.943 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "prometheus_alerts_path", 2024-03-04T09:09:01.943 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.943 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.943 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.943 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-04T09:09:01.943 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.943 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.943 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.944 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-04T09:09:01.944 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.944 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.944 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.944 INFO:teuthology.orchestra.run.smithi071.stdout: "registry_insecure": { 2024-03-04T09:09:01.947 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "registry_insecure", 2024-03-04T09:09:01.947 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.947 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.947 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.947 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.948 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.948 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.948 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.948 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-04T09:09:01.948 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.948 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.948 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.949 INFO:teuthology.orchestra.run.smithi071.stdout: "registry_password": { 2024-03-04T09:09:01.949 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "registry_password", 2024-03-04T09:09:01.949 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.949 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.949 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.949 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.949 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.949 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.950 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.950 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-04T09:09:01.950 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.950 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.950 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.950 INFO:teuthology.orchestra.run.smithi071.stdout: "registry_url": { 2024-03-04T09:09:01.950 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "registry_url", 2024-03-04T09:09:01.951 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.951 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.951 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.951 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.951 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.951 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.951 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.951 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-04T09:09:01.951 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.952 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.952 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.952 INFO:teuthology.orchestra.run.smithi071.stdout: "registry_username": { 2024-03-04T09:09:01.952 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "registry_username", 2024-03-04T09:09:01.952 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.952 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.952 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.953 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.953 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.953 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.953 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.953 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-04T09:09:01.953 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.953 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.953 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.954 INFO:teuthology.orchestra.run.smithi071.stdout: "secure_monitoring_stack": { 2024-03-04T09:09:01.954 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "secure_monitoring_stack", 2024-03-04T09:09:01.954 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.954 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.954 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.954 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.954 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.955 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.955 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.955 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-04T09:09:01.955 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.955 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.955 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.955 INFO:teuthology.orchestra.run.smithi071.stdout: "service_discovery_port": { 2024-03-04T09:09:01.955 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "service_discovery_port", 2024-03-04T09:09:01.956 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.956 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.956 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.956 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8765", 2024-03-04T09:09:01.956 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.956 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.956 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.956 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "cephadm service discovery port", 2024-03-04T09:09:01.957 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.957 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.957 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.957 INFO:teuthology.orchestra.run.smithi071.stdout: "ssh_config_file": { 2024-03-04T09:09:01.957 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ssh_config_file", 2024-03-04T09:09:01.957 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.957 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.957 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.958 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.958 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.958 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.958 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.958 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-04T09:09:01.958 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.958 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.958 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.959 INFO:teuthology.orchestra.run.smithi071.stdout: "use_agent": { 2024-03-04T09:09:01.959 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "use_agent", 2024-03-04T09:09:01.959 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.959 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.959 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.959 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.959 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.959 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.959 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.960 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-04T09:09:01.960 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.960 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.960 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.960 INFO:teuthology.orchestra.run.smithi071.stdout: "use_repo_digest": { 2024-03-04T09:09:01.960 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "use_repo_digest", 2024-03-04T09:09:01.960 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.961 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.961 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.961 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.961 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.961 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.961 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.961 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-04T09:09:01.961 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.961 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.962 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.962 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_on_failed_host_check": { 2024-03-04T09:09:01.962 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_on_failed_host_check", 2024-03-04T09:09:01.962 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.962 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.962 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.962 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.963 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.963 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.963 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.963 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "raise a health warning if the host check fails", 2024-03-04T09:09:01.963 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.963 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.963 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.963 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_on_stray_daemons": { 2024-03-04T09:09:01.964 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_on_stray_daemons", 2024-03-04T09:09:01.964 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.964 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.964 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.964 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.964 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.964 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.964 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.964 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-04T09:09:01.965 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.965 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.965 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.965 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_on_stray_hosts": { 2024-03-04T09:09:01.965 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_on_stray_hosts", 2024-03-04T09:09:01.965 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.965 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.965 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.966 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.966 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.966 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.966 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.966 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-04T09:09:01.966 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.966 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.967 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.967 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.967 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.967 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crash", 2024-03-04T09:09:01.967 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.967 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.967 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.968 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:01.968 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:01.968 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.968 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.968 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.968 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.968 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.968 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.969 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.969 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.969 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.969 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.969 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.969 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.969 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.969 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.969 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.970 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.970 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.970 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.970 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:01.970 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:01.970 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.970 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.971 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.971 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.971 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.971 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.971 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.971 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.971 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.971 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.971 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.972 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:01.972 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:01.972 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.972 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.972 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.972 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:01.972 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.973 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.973 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:01.973 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:01.973 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:01.973 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:01.973 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:01.973 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:01.973 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:01.974 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:01.974 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.974 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.974 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.974 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.974 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:01.974 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:01.974 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.975 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.975 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.975 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.975 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.975 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.975 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.975 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.975 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.975 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.976 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.976 INFO:teuthology.orchestra.run.smithi071.stdout: "retain_interval": { 2024-03-04T09:09:01.976 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "retain_interval", 2024-03-04T09:09:01.976 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.976 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.976 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.976 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "31536000", 2024-03-04T09:09:01.977 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.977 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.977 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.977 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-04T09:09:01.977 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.977 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.977 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.977 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_recent_interval": { 2024-03-04T09:09:01.978 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_recent_interval", 2024-03-04T09:09:01.978 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:01.978 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.978 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:01.978 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "1209600", 2024-03-04T09:09:01.978 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.978 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.978 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.979 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-04T09:09:01.979 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.979 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.979 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.979 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.979 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:01.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.979 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:01.979 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "dashboard", 2024-03-04T09:09:01.980 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:01.980 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:01.980 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:01.980 INFO:teuthology.orchestra.run.smithi071.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-04T09:09:01.980 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-04T09:09:01.980 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:01.980 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.980 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.981 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-03-04T09:09:01.981 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.981 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.981 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.981 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.981 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.981 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.981 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.982 INFO:teuthology.orchestra.run.smithi071.stdout: "ALERTMANAGER_API_HOST": { 2024-03-04T09:09:01.982 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-04T09:09:01.982 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:01.982 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.982 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.982 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:01.982 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.982 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.983 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.983 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.983 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.983 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.983 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.983 INFO:teuthology.orchestra.run.smithi071.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-04T09:09:01.983 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-04T09:09:01.984 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.984 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.984 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.984 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.984 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.984 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.984 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.984 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.985 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.985 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.985 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.985 INFO:teuthology.orchestra.run.smithi071.stdout: "AUDIT_API_ENABLED": { 2024-03-04T09:09:01.985 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "AUDIT_API_ENABLED", 2024-03-04T09:09:01.985 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.985 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.985 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.986 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:01.986 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.986 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.986 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.986 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.986 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.986 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.986 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.987 INFO:teuthology.orchestra.run.smithi071.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-04T09:09:01.987 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-04T09:09:01.987 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.987 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.987 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.987 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.987 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.987 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.988 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.988 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.988 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.988 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.988 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.988 INFO:teuthology.orchestra.run.smithi071.stdout: "ENABLE_BROWSABLE_API": { 2024-03-04T09:09:01.988 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-04T09:09:01.989 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.989 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.989 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.989 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.989 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.989 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.989 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.989 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.989 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.990 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.990 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.990 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-04T09:09:01.990 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-04T09:09:01.990 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.990 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.990 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.991 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.991 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.991 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.991 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.991 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.991 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.991 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.991 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.992 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-04T09:09:01.992 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-04T09:09:01.992 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.992 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.992 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.992 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.992 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.992 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.993 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.993 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.993 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.993 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.993 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.993 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-04T09:09:01.993 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-04T09:09:01.994 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.994 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.994 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.994 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.994 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.994 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.994 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.994 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.994 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.995 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.995 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.995 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-04T09:09:01.995 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-04T09:09:01.995 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.995 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.995 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.996 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.996 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.996 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.996 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.996 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.996 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.996 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.996 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.997 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-04T09:09:01.997 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-04T09:09:01.997 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.997 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.997 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.997 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.997 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.997 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.998 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.998 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.998 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.998 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.998 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:01.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:01.998 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-04T09:09:01.998 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-04T09:09:01.998 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:01.999 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:01.999 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:01.999 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:01.999 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:01.999 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:01.999 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:01.999 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:01.999 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:01.999 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:01.999 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.000 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-04T09:09:02.000 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-04T09:09:02.000 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.000 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.000 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.000 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:02.001 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.001 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.001 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.001 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.001 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.001 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.001 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.001 INFO:teuthology.orchestra.run.smithi071.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-04T09:09:02.002 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-04T09:09:02.002 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.002 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.002 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.002 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.002 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.002 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.002 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.003 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.003 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.003 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.003 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.003 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_API_PASSWORD": { 2024-03-04T09:09:02.003 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-04T09:09:02.003 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.003 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.004 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.004 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "admin", 2024-03-04T09:09:02.004 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.004 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.004 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.004 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.004 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.004 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.005 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.005 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-04T09:09:02.005 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-04T09:09:02.005 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.005 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.005 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.005 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:02.005 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.006 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.006 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.006 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.006 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.006 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.006 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.007 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_API_URL": { 2024-03-04T09:09:02.007 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_API_URL", 2024-03-04T09:09:02.007 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.007 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.007 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.007 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.007 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.007 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.007 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.008 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.008 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.008 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.008 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.008 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_API_USERNAME": { 2024-03-04T09:09:02.008 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-04T09:09:02.008 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.009 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.009 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.009 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "admin", 2024-03-04T09:09:02.009 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.009 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.009 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.009 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.009 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.009 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.010 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.010 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-04T09:09:02.010 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-04T09:09:02.010 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.010 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.010 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.010 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.011 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.011 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.011 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.011 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.011 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.011 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.011 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.012 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-04T09:09:02.012 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-04T09:09:02.012 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.012 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.012 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.012 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.012 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.012 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.013 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.013 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.013 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.013 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.013 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.013 INFO:teuthology.orchestra.run.smithi071.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-04T09:09:02.013 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-04T09:09:02.014 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.014 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.014 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.014 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:02.014 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.014 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.014 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.014 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.014 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.015 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.015 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.015 INFO:teuthology.orchestra.run.smithi071.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-04T09:09:02.015 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-04T09:09:02.015 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.015 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.015 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.016 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.016 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.016 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.016 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.016 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.016 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.016 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.016 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.017 INFO:teuthology.orchestra.run.smithi071.stdout: "PROMETHEUS_API_HOST": { 2024-03-04T09:09:02.017 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-04T09:09:02.017 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.017 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.017 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.017 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.017 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.017 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.018 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.018 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.018 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.018 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.018 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.018 INFO:teuthology.orchestra.run.smithi071.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-04T09:09:02.018 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-04T09:09:02.019 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.019 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.019 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.019 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:02.019 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.019 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.019 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.019 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.020 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.020 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.020 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.020 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-04T09:09:02.020 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-04T09:09:02.020 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.020 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.021 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.021 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.021 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.021 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.021 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.021 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.021 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.021 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.021 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.022 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-04T09:09:02.022 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-04T09:09:02.022 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.022 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.022 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.022 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.022 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.023 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.023 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.023 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.023 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.023 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.023 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.023 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-04T09:09:02.024 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-04T09:09:02.024 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.024 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.024 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.024 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:02.024 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.024 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.024 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.025 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.025 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.025 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.025 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.025 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.025 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-04T09:09:02.025 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-04T09:09:02.025 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.026 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.026 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.026 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:02.026 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.026 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.026 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.026 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.026 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.026 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.027 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.027 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-04T09:09:02.027 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-04T09:09:02.027 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.027 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.027 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.027 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.028 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.028 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.028 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.028 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.028 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.028 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.028 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.029 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-04T09:09:02.029 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-04T09:09:02.029 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.029 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.029 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.029 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.029 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.029 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.030 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.030 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.030 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.030 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.030 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.030 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-04T09:09:02.030 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-04T09:09:02.030 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.031 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.031 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.031 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.031 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.031 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.031 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.031 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.031 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.032 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.032 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.032 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_ENABLED": { 2024-03-04T09:09:02.032 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-04T09:09:02.032 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.032 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.032 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.033 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:02.033 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.033 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.033 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.033 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.033 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.033 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.033 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.034 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-04T09:09:02.034 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-04T09:09:02.034 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.034 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.034 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.034 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-04T09:09:02.034 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.034 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.035 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.035 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.035 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.035 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.035 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.035 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-04T09:09:02.035 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-04T09:09:02.036 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.036 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.036 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.036 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-03-04T09:09:02.036 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.036 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.036 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.036 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.037 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.037 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.037 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.037 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-04T09:09:02.037 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-04T09:09:02.037 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.037 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.038 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.038 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8", 2024-03-04T09:09:02.038 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.038 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.038 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.038 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.038 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.038 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.038 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.039 INFO:teuthology.orchestra.run.smithi071.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-04T09:09:02.039 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-04T09:09:02.039 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.039 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.039 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.039 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "45", 2024-03-04T09:09:02.039 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.040 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.040 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.040 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.040 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.040 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.040 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.040 INFO:teuthology.orchestra.run.smithi071.stdout: "RGW_API_ACCESS_KEY": { 2024-03-04T09:09:02.041 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-04T09:09:02.041 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.041 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.041 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.041 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.041 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.041 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.041 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.041 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.042 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.042 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.042 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.042 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.042 INFO:teuthology.orchestra.run.smithi071.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-04T09:09:02.042 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-04T09:09:02.042 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.042 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.043 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.043 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "admin", 2024-03-04T09:09:02.043 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.043 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.043 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.043 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.043 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.043 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.043 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.044 INFO:teuthology.orchestra.run.smithi071.stdout: "RGW_API_SECRET_KEY": { 2024-03-04T09:09:02.044 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-04T09:09:02.044 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.044 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.044 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.044 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.044 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.045 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.045 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.045 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.045 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.045 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.045 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.045 INFO:teuthology.orchestra.run.smithi071.stdout: "RGW_API_SSL_VERIFY": { 2024-03-04T09:09:02.045 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-04T09:09:02.046 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.046 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.046 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.046 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:02.046 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.046 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.046 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.046 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.046 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.047 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.047 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.047 INFO:teuthology.orchestra.run.smithi071.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-04T09:09:02.047 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-04T09:09:02.047 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.047 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.047 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.048 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-03-04T09:09:02.048 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.048 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.048 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.048 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.048 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.048 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.048 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.049 INFO:teuthology.orchestra.run.smithi071.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-04T09:09:02.049 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-04T09:09:02.049 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.049 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.049 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.049 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-03-04T09:09:02.049 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.050 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.050 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.050 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.050 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.050 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.050 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.050 INFO:teuthology.orchestra.run.smithi071.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-04T09:09:02.050 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-04T09:09:02.051 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.051 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.051 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.051 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5", 2024-03-04T09:09:02.051 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.051 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.051 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.051 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.052 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.052 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.052 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.052 INFO:teuthology.orchestra.run.smithi071.stdout: "cross_origin_url": { 2024-03-04T09:09:02.052 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cross_origin_url", 2024-03-04T09:09:02.052 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.052 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.053 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.053 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.053 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.053 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.053 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.053 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.053 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.053 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.053 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.054 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.054 INFO:teuthology.orchestra.run.smithi071.stdout: "crt_file": { 2024-03-04T09:09:02.054 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crt_file", 2024-03-04T09:09:02.054 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.054 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.054 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.054 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.054 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.055 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.055 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.055 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.055 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.055 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.055 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.055 INFO:teuthology.orchestra.run.smithi071.stdout: "debug": { 2024-03-04T09:09:02.055 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "debug", 2024-03-04T09:09:02.056 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.056 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.056 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.056 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.056 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.056 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.056 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.056 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Enable/disable debug options", 2024-03-04T09:09:02.057 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.057 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.057 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.057 INFO:teuthology.orchestra.run.smithi071.stdout: "jwt_token_ttl": { 2024-03-04T09:09:02.057 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "jwt_token_ttl", 2024-03-04T09:09:02.057 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.057 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.058 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.058 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "28800", 2024-03-04T09:09:02.058 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.058 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.058 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.058 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.058 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.058 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.059 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.059 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.059 INFO:teuthology.orchestra.run.smithi071.stdout: "key_file": { 2024-03-04T09:09:02.059 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "key_file", 2024-03-04T09:09:02.059 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.059 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.059 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.059 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.059 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.060 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.060 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.060 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.060 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.060 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.060 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.060 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.061 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.061 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.061 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.061 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.061 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.061 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.061 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.061 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.062 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.062 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.062 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.062 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.062 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.062 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.062 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.062 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.062 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.063 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.063 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.063 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.063 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.063 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.063 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.063 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.063 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.064 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.064 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.064 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.064 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.064 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.064 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.064 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.064 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.065 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.065 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.065 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.065 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.065 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.065 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.065 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.065 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.066 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.066 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.066 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.066 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.066 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.066 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.066 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.066 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.066 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.067 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.067 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.067 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.067 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.067 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.067 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.067 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.067 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.067 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.068 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.068 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.068 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.068 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.068 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.068 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.068 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.069 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.069 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.069 INFO:teuthology.orchestra.run.smithi071.stdout: "motd": { 2024-03-04T09:09:02.069 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "motd", 2024-03-04T09:09:02.069 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.069 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.069 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.069 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.069 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.070 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.070 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.070 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "The message of the day", 2024-03-04T09:09:02.070 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.070 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.070 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.070 INFO:teuthology.orchestra.run.smithi071.stdout: "redirect_resolve_ip_addr": { 2024-03-04T09:09:02.071 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "redirect_resolve_ip_addr", 2024-03-04T09:09:02.071 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.071 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.071 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.071 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.071 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.071 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.071 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.072 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.072 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.072 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.072 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.072 INFO:teuthology.orchestra.run.smithi071.stdout: "server_addr": { 2024-03-04T09:09:02.072 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_addr", 2024-03-04T09:09:02.072 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.073 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.073 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.073 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "::", 2024-03-04T09:09:02.073 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.073 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.073 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.073 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.073 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.073 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.074 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.074 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.074 INFO:teuthology.orchestra.run.smithi071.stdout: "server_port": { 2024-03-04T09:09:02.074 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_port", 2024-03-04T09:09:02.074 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.074 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.074 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.074 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8080", 2024-03-04T09:09:02.075 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.075 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.075 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.075 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.075 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.075 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.075 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.075 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.076 INFO:teuthology.orchestra.run.smithi071.stdout: "ssl": { 2024-03-04T09:09:02.076 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ssl", 2024-03-04T09:09:02.076 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.076 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.076 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.076 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:02.076 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.076 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.077 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.077 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.077 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.077 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.077 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.077 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.077 INFO:teuthology.orchestra.run.smithi071.stdout: "ssl_server_port": { 2024-03-04T09:09:02.077 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ssl_server_port", 2024-03-04T09:09:02.077 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.077 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.078 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.078 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8443", 2024-03-04T09:09:02.078 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.078 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.078 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.078 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.078 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.078 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.078 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.079 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.079 INFO:teuthology.orchestra.run.smithi071.stdout: "standby_behaviour": { 2024-03-04T09:09:02.079 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "standby_behaviour", 2024-03-04T09:09:02.079 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.079 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.079 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.079 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "redirect", 2024-03-04T09:09:02.079 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.080 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.080 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.080 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.080 INFO:teuthology.orchestra.run.smithi071.stdout: "redirect" 2024-03-04T09:09:02.080 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.080 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.080 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.080 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.081 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.081 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.081 INFO:teuthology.orchestra.run.smithi071.stdout: "standby_error_status_code": { 2024-03-04T09:09:02.081 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "standby_error_status_code", 2024-03-04T09:09:02.081 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.081 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.081 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.081 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "500", 2024-03-04T09:09:02.082 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "400", 2024-03-04T09:09:02.082 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "599", 2024-03-04T09:09:02.082 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.082 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.082 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.082 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.082 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.082 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.082 INFO:teuthology.orchestra.run.smithi071.stdout: "url_prefix": { 2024-03-04T09:09:02.083 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "url_prefix", 2024-03-04T09:09:02.083 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.083 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.083 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.083 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.083 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.083 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.083 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.084 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.084 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.084 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.084 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.084 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.084 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.084 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.084 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.084 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "devicehealth", 2024-03-04T09:09:02.085 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.085 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.085 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.085 INFO:teuthology.orchestra.run.smithi071.stdout: "enable_monitoring": { 2024-03-04T09:09:02.085 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "enable_monitoring", 2024-03-04T09:09:02.085 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.085 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.085 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.085 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:02.086 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.086 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.086 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.086 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "monitor device health metrics", 2024-03-04T09:09:02.086 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.086 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.086 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.086 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.087 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.087 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.087 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.087 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.087 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.087 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.087 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.087 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.087 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.088 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.088 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.088 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.088 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.088 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.088 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.088 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.088 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.089 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.089 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.089 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.089 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.089 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.089 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.089 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.089 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.089 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.090 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.090 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.090 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.090 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.090 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.090 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.090 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.090 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.090 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.091 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.091 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.091 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.091 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.091 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.091 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.091 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.091 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.092 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.092 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.092 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.092 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.092 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.092 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.092 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.092 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.092 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.093 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.093 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.093 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.093 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.093 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.093 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.093 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.093 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.093 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.094 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.094 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.094 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.094 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.094 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.094 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.094 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.094 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.095 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.095 INFO:teuthology.orchestra.run.smithi071.stdout: "mark_out_threshold": { 2024-03-04T09:09:02.095 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mark_out_threshold", 2024-03-04T09:09:02.095 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:02.095 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.095 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.095 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "2419200", 2024-03-04T09:09:02.095 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.096 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.096 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.096 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-04T09:09:02.096 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.096 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.096 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.096 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.096 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": { 2024-03-04T09:09:02.096 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "pool_name", 2024-03-04T09:09:02.097 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.097 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.097 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.097 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "device_health_metrics", 2024-03-04T09:09:02.097 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.097 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.097 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.097 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-04T09:09:02.098 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.098 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.098 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.098 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.098 INFO:teuthology.orchestra.run.smithi071.stdout: "retention_period": { 2024-03-04T09:09:02.098 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "retention_period", 2024-03-04T09:09:02.098 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:02.098 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.098 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.099 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "15552000", 2024-03-04T09:09:02.099 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.099 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.099 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.099 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how long to retain device health metrics", 2024-03-04T09:09:02.099 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.099 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.099 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.099 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.100 INFO:teuthology.orchestra.run.smithi071.stdout: "scrape_frequency": { 2024-03-04T09:09:02.100 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "scrape_frequency", 2024-03-04T09:09:02.100 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:02.100 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.100 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.100 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "86400", 2024-03-04T09:09:02.100 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.100 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.101 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.101 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-04T09:09:02.101 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.101 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.101 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.101 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.101 INFO:teuthology.orchestra.run.smithi071.stdout: "self_heal": { 2024-03-04T09:09:02.101 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "self_heal", 2024-03-04T09:09:02.102 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.102 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.102 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.102 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:02.102 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.102 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.102 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.102 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-04T09:09:02.103 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.103 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.103 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.103 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.103 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-03-04T09:09:02.103 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-03-04T09:09:02.103 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:02.103 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.104 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.104 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-03-04T09:09:02.104 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.104 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.104 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.104 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to wake up and check device health", 2024-03-04T09:09:02.104 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.104 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.105 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.105 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.105 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_threshold": { 2024-03-04T09:09:02.105 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_threshold", 2024-03-04T09:09:02.105 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:02.105 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.105 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.105 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "7257600", 2024-03-04T09:09:02.106 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.106 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.106 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.106 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-04T09:09:02.106 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.106 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.106 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.106 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.106 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.107 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.107 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.107 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "diskprediction_local", 2024-03-04T09:09:02.107 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.107 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.107 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.107 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.107 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.108 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.108 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.108 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.108 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.108 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.108 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.108 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.108 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.109 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.109 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.109 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.109 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.109 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.109 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.109 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.109 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.109 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.110 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.110 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.110 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.110 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.110 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.110 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.110 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.110 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.111 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.111 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.111 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.111 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.111 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.111 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.111 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.111 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.111 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.112 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.112 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.112 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.112 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.112 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.112 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.112 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.113 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.113 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.113 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.113 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.113 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.113 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.113 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.113 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.113 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.114 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.114 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.114 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.114 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.114 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.114 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.114 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.114 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.115 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.115 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.115 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.115 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.115 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.115 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.115 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.115 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.116 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.116 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.116 INFO:teuthology.orchestra.run.smithi071.stdout: "predict_interval": { 2024-03-04T09:09:02.116 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "predict_interval", 2024-03-04T09:09:02.116 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.116 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.116 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.116 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "86400", 2024-03-04T09:09:02.117 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.117 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.117 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.117 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.117 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.117 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.117 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.117 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.117 INFO:teuthology.orchestra.run.smithi071.stdout: "predictor_model": { 2024-03-04T09:09:02.118 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "predictor_model", 2024-03-04T09:09:02.118 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.118 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.118 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.118 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "prophetstor", 2024-03-04T09:09:02.118 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.118 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.118 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.119 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.119 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.119 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.119 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.119 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.119 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-03-04T09:09:02.119 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-03-04T09:09:02.119 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.119 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.120 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.120 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-03-04T09:09:02.120 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.120 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.120 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.120 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.120 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.120 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.121 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.121 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.121 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.121 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.121 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.121 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "influx", 2024-03-04T09:09:02.121 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": false, 2024-03-04T09:09:02.121 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "influxdb python module not found", 2024-03-04T09:09:02.122 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.122 INFO:teuthology.orchestra.run.smithi071.stdout: "batch_size": { 2024-03-04T09:09:02.122 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "batch_size", 2024-03-04T09:09:02.122 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.122 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.122 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.122 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5000", 2024-03-04T09:09:02.122 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.123 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.123 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.123 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-04T09:09:02.123 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.123 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.123 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.123 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.123 INFO:teuthology.orchestra.run.smithi071.stdout: "database": { 2024-03-04T09:09:02.123 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "database", 2024-03-04T09:09:02.124 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.124 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.124 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.124 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "ceph", 2024-03-04T09:09:02.124 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.124 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.124 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.124 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-04T09:09:02.125 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.125 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.125 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.125 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.125 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": { 2024-03-04T09:09:02.125 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "hostname", 2024-03-04T09:09:02.125 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.125 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.126 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.126 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.126 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.126 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.126 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.126 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "InfluxDB server hostname", 2024-03-04T09:09:02.126 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.126 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.126 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.127 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.127 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-03-04T09:09:02.127 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-03-04T09:09:02.127 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:02.127 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.127 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.127 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "30", 2024-03-04T09:09:02.127 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "5", 2024-03-04T09:09:02.127 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.128 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.128 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-04T09:09:02.128 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.128 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.128 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.128 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.128 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.128 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.129 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.129 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.129 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.129 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.129 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.129 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.129 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.130 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.130 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.130 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.130 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.130 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.130 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.130 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.130 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.131 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.131 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.131 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.131 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.131 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.131 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.131 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.131 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.132 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.132 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.132 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.132 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.132 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.132 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.132 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.132 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.133 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.133 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.133 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.133 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.133 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.133 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.133 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.133 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.133 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.134 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.134 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.134 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.134 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.134 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.134 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.134 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.134 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.135 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.135 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.135 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.135 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.135 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.135 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.135 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.135 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.135 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.136 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.136 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.136 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.136 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.136 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.136 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.136 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.136 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.137 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.137 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.137 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.137 INFO:teuthology.orchestra.run.smithi071.stdout: "password": { 2024-03-04T09:09:02.137 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "password", 2024-03-04T09:09:02.137 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.137 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.137 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.137 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.138 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.138 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.138 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.138 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "password of InfluxDB server user", 2024-03-04T09:09:02.138 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.138 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.138 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.138 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.139 INFO:teuthology.orchestra.run.smithi071.stdout: "port": { 2024-03-04T09:09:02.139 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "port", 2024-03-04T09:09:02.139 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.139 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.139 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.139 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8086", 2024-03-04T09:09:02.139 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.139 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.139 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.140 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "InfluxDB server port", 2024-03-04T09:09:02.140 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.140 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.140 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.140 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.140 INFO:teuthology.orchestra.run.smithi071.stdout: "ssl": { 2024-03-04T09:09:02.140 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ssl", 2024-03-04T09:09:02.140 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.141 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.141 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.141 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "false", 2024-03-04T09:09:02.141 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.141 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.141 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.141 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-04T09:09:02.141 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.141 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.142 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.142 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.142 INFO:teuthology.orchestra.run.smithi071.stdout: "threads": { 2024-03-04T09:09:02.142 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "threads", 2024-03-04T09:09:02.142 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.142 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.142 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.142 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5", 2024-03-04T09:09:02.143 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1", 2024-03-04T09:09:02.143 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "32", 2024-03-04T09:09:02.143 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.143 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-04T09:09:02.143 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.143 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.143 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.143 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.143 INFO:teuthology.orchestra.run.smithi071.stdout: "username": { 2024-03-04T09:09:02.144 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "username", 2024-03-04T09:09:02.144 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.144 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.144 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.144 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.144 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.144 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.144 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.145 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "username of InfluxDB server user", 2024-03-04T09:09:02.145 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.145 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.145 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.145 INFO:teuthology.orchestra.run.smithi071.stdout: "verify_ssl": { 2024-03-04T09:09:02.145 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "verify_ssl", 2024-03-04T09:09:02.145 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.145 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.146 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.146 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "true", 2024-03-04T09:09:02.146 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.146 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.146 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.146 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-04T09:09:02.146 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.146 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.146 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.147 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.147 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.147 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.147 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.147 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "insights", 2024-03-04T09:09:02.147 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.147 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.147 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.148 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.148 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.148 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.148 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.148 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.148 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.148 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.148 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.148 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.149 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.149 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.149 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.149 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.149 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.149 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.149 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.149 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.150 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.150 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.150 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.150 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.150 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.150 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.150 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.151 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.151 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.151 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.151 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.151 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.151 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.151 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.151 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.151 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.152 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.152 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.152 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.152 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.152 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.152 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.152 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.153 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.153 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.153 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.153 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.153 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.153 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.153 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.153 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.153 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.154 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.154 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.154 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.154 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.154 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.154 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.154 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.155 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.155 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.155 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.155 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.155 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.155 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.155 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.155 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.155 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.156 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.156 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.156 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.156 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "iostat", 2024-03-04T09:09:02.156 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.157 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.157 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.157 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.157 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.157 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.157 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.157 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.157 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.158 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.158 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.158 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.158 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.158 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.158 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.158 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.158 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.159 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.159 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.159 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.159 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.159 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.159 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.159 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.159 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.160 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.160 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.160 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.160 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.160 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.160 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.160 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.161 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.161 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.161 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.161 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.161 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.161 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.161 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.161 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.162 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.162 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.162 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.162 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.162 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.162 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.162 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.162 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.163 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.163 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.163 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.163 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.163 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.163 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.163 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.164 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.164 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.164 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.164 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.164 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.164 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.164 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.165 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.165 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.165 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.165 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.165 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.165 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.165 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.165 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.166 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.166 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.166 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "k8sevents", 2024-03-04T09:09:02.166 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.166 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.166 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.166 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_event_retention_days": { 2024-03-04T09:09:02.166 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ceph_event_retention_days", 2024-03-04T09:09:02.167 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.167 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.167 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.167 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "7", 2024-03-04T09:09:02.167 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.167 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.167 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.167 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-04T09:09:02.168 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.168 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.168 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.168 INFO:teuthology.orchestra.run.smithi071.stdout: "config_check_secs": { 2024-03-04T09:09:02.168 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "config_check_secs", 2024-03-04T09:09:02.168 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.168 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.169 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.169 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-03-04T09:09:02.169 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "10", 2024-03-04T09:09:02.169 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.169 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.169 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-04T09:09:02.169 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.169 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.170 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.170 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.170 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.170 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.170 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.170 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.170 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.171 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.171 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.171 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.171 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.171 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.171 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.171 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.171 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.171 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.172 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.172 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.172 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.172 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.172 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.172 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.172 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.173 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.173 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.173 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.173 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.173 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.173 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.173 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.173 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.174 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.174 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.174 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.174 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.174 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.174 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.174 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.174 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.175 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.175 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.175 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.175 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.175 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.175 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.175 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.175 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.176 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.176 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.176 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.176 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.176 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.176 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.176 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.177 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.177 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.177 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.177 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.177 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.177 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.177 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.177 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.178 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.178 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.178 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.178 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.178 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.178 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.178 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.179 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "localpool", 2024-03-04T09:09:02.179 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.179 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.179 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.179 INFO:teuthology.orchestra.run.smithi071.stdout: "failure_domain": { 2024-03-04T09:09:02.179 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "failure_domain", 2024-03-04T09:09:02.179 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.180 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.180 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.180 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "host", 2024-03-04T09:09:02.180 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.180 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.180 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.180 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "failure domain for any created local pool", 2024-03-04T09:09:02.180 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-04T09:09:02.181 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.181 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.181 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.181 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.181 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.181 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.181 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.181 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.182 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.182 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.182 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.182 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.182 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.182 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.182 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.182 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.183 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.183 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.183 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.183 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.183 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.183 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.183 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.183 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.184 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.184 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.184 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.184 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.184 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.184 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.184 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.184 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.185 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.185 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.185 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.185 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.185 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.185 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.185 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.185 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.186 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.186 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.186 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.186 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.186 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.186 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.186 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.186 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.187 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.187 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.187 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.187 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.187 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.187 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.187 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.188 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.188 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.188 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.188 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.188 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.188 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.188 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.188 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.189 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.189 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.189 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.189 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.189 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.189 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": { 2024-03-04T09:09:02.189 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "min_size", 2024-03-04T09:09:02.189 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.190 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.190 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.190 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.190 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.190 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.190 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.190 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "default min_size for any created local pool", 2024-03-04T09:09:02.190 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-04T09:09:02.191 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.191 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.191 INFO:teuthology.orchestra.run.smithi071.stdout: "num_rep": { 2024-03-04T09:09:02.191 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "num_rep", 2024-03-04T09:09:02.191 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.191 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.191 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.191 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "3", 2024-03-04T09:09:02.192 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.192 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.192 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.192 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "default replica count for any created local pool", 2024-03-04T09:09:02.192 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.192 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.192 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.193 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": { 2024-03-04T09:09:02.193 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "pg_num", 2024-03-04T09:09:02.193 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.193 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.193 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.193 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "128", 2024-03-04T09:09:02.193 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.193 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.194 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.194 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "default pg_num for any created local pool", 2024-03-04T09:09:02.194 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.194 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.194 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.194 INFO:teuthology.orchestra.run.smithi071.stdout: "prefix": { 2024-03-04T09:09:02.194 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "prefix", 2024-03-04T09:09:02.194 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.195 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.195 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.195 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.195 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.195 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.195 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.195 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "name prefix for any created local pool", 2024-03-04T09:09:02.195 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.196 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.196 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.196 INFO:teuthology.orchestra.run.smithi071.stdout: "subtree": { 2024-03-04T09:09:02.196 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subtree", 2024-03-04T09:09:02.196 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.196 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.196 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.197 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "rack", 2024-03-04T09:09:02.197 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.197 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.197 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.197 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-04T09:09:02.197 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-04T09:09:02.197 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.197 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.198 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.198 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.198 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mds_autoscaler", 2024-03-04T09:09:02.198 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.198 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.198 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.199 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.199 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.199 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.199 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.199 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.199 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.199 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.199 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.200 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.200 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.200 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.200 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.200 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.200 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.200 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.200 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.201 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.201 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.201 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.201 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.201 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.201 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.201 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.201 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.202 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.202 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.202 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.202 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.202 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.202 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.202 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.202 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.202 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.203 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.203 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.203 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.203 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.203 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.203 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.203 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.204 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.204 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.204 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.204 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.204 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.204 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.204 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.204 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.204 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.205 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.205 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.205 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.205 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.205 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.205 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.205 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.206 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.206 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.206 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.206 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.206 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.206 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.206 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.206 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.206 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.207 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.207 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.207 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.207 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mirroring", 2024-03-04T09:09:02.207 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.207 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.208 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.208 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.208 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.208 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.208 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.208 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.208 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.208 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.208 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.209 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.209 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.209 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.209 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.209 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.209 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.209 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.209 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.210 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.210 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.210 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.210 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.210 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.210 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.210 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.210 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.211 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.211 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.211 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.211 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.211 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.211 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.211 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.211 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.212 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.212 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.212 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.212 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.212 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.212 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.212 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.212 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.213 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.213 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.213 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.213 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.213 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.213 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.213 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.213 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.214 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.214 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.214 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.214 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.214 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.214 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.214 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.215 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.215 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.215 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.215 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.215 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.215 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.215 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.215 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.215 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.216 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.216 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.216 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.216 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.216 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "nfs", 2024-03-04T09:09:02.216 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.217 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.217 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.217 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.217 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.217 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.217 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.217 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.217 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.218 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.218 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.218 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.218 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.218 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.218 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.218 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.218 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.219 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.219 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.219 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.219 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.219 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.219 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.219 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.219 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.220 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.220 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.220 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.220 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.220 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.220 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.220 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.220 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.221 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.221 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.221 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.221 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.221 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.221 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.221 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.221 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.222 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.222 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.222 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.222 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.222 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.222 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.222 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.222 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.222 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.223 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.223 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.223 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.223 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.223 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.223 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.223 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.224 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.224 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.224 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.224 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.224 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.224 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.224 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.224 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.225 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.225 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.225 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.225 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.225 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.225 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.225 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "orchestrator", 2024-03-04T09:09:02.226 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.226 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.226 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.226 INFO:teuthology.orchestra.run.smithi071.stdout: "fail_fs": { 2024-03-04T09:09:02.226 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "fail_fs", 2024-03-04T09:09:02.226 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.226 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.226 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.226 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.227 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.227 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.227 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.227 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-04T09:09:02.227 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.227 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.227 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.228 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.228 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.228 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.228 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.228 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.228 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.228 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.228 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.229 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.229 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.229 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.229 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.229 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.229 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.229 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.229 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.229 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.230 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.230 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.230 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.230 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.230 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.230 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.230 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.231 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.231 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.231 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.231 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.231 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.231 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.231 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.231 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.231 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.232 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.232 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.232 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.232 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.232 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.232 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.232 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.233 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.233 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.233 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.233 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.233 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.233 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.233 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.233 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.233 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.234 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.234 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.234 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.234 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.234 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.234 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.234 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.235 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.235 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.235 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.235 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.235 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.235 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.235 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.235 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.235 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.236 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.236 INFO:teuthology.orchestra.run.smithi071.stdout: "orchestrator": { 2024-03-04T09:09:02.236 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "orchestrator", 2024-03-04T09:09:02.236 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.236 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.236 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.236 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.237 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.237 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.237 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.237 INFO:teuthology.orchestra.run.smithi071.stdout: "cephadm", 2024-03-04T09:09:02.237 INFO:teuthology.orchestra.run.smithi071.stdout: "rook", 2024-03-04T09:09:02.237 INFO:teuthology.orchestra.run.smithi071.stdout: "test_orchestrator" 2024-03-04T09:09:02.237 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.237 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Orchestrator backend", 2024-03-04T09:09:02.237 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.238 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.238 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.238 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.238 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.238 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "osd_perf_query", 2024-03-04T09:09:02.238 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.239 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.239 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.239 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.239 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.239 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.239 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.239 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.239 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.239 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.240 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.240 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.240 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.240 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.240 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.240 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.240 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.240 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.241 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.241 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.241 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.241 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.241 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.241 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.241 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.242 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.242 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.242 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.242 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.242 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.242 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.242 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.242 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.243 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.243 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.243 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.243 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.243 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.243 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.243 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.243 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.244 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.244 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.244 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.244 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.244 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.244 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.244 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.244 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.245 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.245 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.245 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.245 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.245 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.245 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.245 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.245 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.246 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.246 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.246 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.246 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.246 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.246 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.246 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.246 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.247 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.247 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.247 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.247 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.247 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.247 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.247 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.248 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "osd_support", 2024-03-04T09:09:02.248 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.248 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.248 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.248 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.248 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.248 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.248 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.249 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.249 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.249 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.249 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.249 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.249 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.249 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.249 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.250 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.250 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.250 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.250 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.250 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.250 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.250 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.250 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.251 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.251 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.251 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.251 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.251 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.251 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.251 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.251 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.252 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.252 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.252 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.252 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.252 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.252 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.252 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.253 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.253 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.253 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.253 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.253 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.253 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.253 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.253 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.253 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.254 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.254 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.254 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.254 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.254 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.254 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.254 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.254 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.255 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.255 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.255 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.255 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.255 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.255 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.255 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.256 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.256 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.256 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.256 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.256 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.256 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.256 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.256 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.257 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.257 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "pg_autoscaler", 2024-03-04T09:09:02.257 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.257 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.257 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.257 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.258 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.258 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.258 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.258 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.258 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.258 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.258 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.258 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.258 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.259 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.259 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.259 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.259 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.259 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.259 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.259 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.259 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.259 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.260 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.260 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.260 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.260 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.260 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.260 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.260 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.261 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.261 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.261 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.261 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.261 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.261 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.261 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.261 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.262 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.262 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.262 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.262 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.262 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.262 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.262 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.262 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.263 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.263 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.263 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.263 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.263 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.263 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.263 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.263 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.263 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.264 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.264 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.264 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.264 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.264 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.264 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.264 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.265 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.265 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.265 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.265 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.265 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.265 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.265 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.265 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.266 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-03-04T09:09:02.266 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-03-04T09:09:02.266 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:02.266 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.266 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.266 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-03-04T09:09:02.266 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.267 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.267 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.267 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.267 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.267 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.267 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.267 INFO:teuthology.orchestra.run.smithi071.stdout: "threshold": { 2024-03-04T09:09:02.268 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "threshold", 2024-03-04T09:09:02.268 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-03-04T09:09:02.268 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.268 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.268 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "3.0", 2024-03-04T09:09:02.268 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1.0", 2024-03-04T09:09:02.268 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.268 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.269 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "scaling threshold", 2024-03-04T09:09:02.269 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-04T09:09:02.269 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.269 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.269 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.269 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.269 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.269 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "progress", 2024-03-04T09:09:02.270 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.270 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.270 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.270 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_pg_recovery_event": { 2024-03-04T09:09:02.270 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "allow_pg_recovery_event", 2024-03-04T09:09:02.270 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.270 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.270 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.271 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.271 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.271 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.271 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.271 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-04T09:09:02.271 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.271 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.271 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.272 INFO:teuthology.orchestra.run.smithi071.stdout: "enabled": { 2024-03-04T09:09:02.272 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "enabled", 2024-03-04T09:09:02.272 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.272 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.272 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.272 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:02.272 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.273 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.273 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.273 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.273 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.273 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.273 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.273 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.273 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.274 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.274 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.274 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.274 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.274 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.274 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.274 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.274 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.275 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.275 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.275 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.275 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.275 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.275 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.275 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.275 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.275 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.276 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.276 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.276 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.276 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.276 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.276 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.276 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.277 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.277 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.277 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.277 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.277 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.277 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.277 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.277 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.278 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.278 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.278 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.278 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.278 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.278 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.278 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.278 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.279 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.279 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.279 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.279 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.279 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.279 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.279 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.279 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.279 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.280 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.280 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.280 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.280 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.280 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.280 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.280 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.281 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.281 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.281 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.281 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.281 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.281 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.281 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.281 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.282 INFO:teuthology.orchestra.run.smithi071.stdout: "max_completed_events": { 2024-03-04T09:09:02.282 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_completed_events", 2024-03-04T09:09:02.282 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.282 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.282 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.282 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "50", 2024-03-04T09:09:02.282 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.282 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.283 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.283 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "number of past completed events to remember", 2024-03-04T09:09:02.283 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.283 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.283 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.283 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-03-04T09:09:02.283 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-03-04T09:09:02.284 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:02.284 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.284 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.284 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5", 2024-03-04T09:09:02.284 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.284 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.284 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.284 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how long the module is going to sleep", 2024-03-04T09:09:02.285 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.285 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.285 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.285 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.285 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.285 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "prometheus", 2024-03-04T09:09:02.286 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.286 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.286 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.286 INFO:teuthology.orchestra.run.smithi071.stdout: "cache": { 2024-03-04T09:09:02.286 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cache", 2024-03-04T09:09:02.286 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.286 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.286 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.286 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:02.287 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.287 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.287 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.287 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.287 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.287 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.287 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.287 INFO:teuthology.orchestra.run.smithi071.stdout: "exclude_perf_counters": { 2024-03-04T09:09:02.288 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "exclude_perf_counters", 2024-03-04T09:09:02.288 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.288 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.288 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.288 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:02.288 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.288 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.288 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.289 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-04T09:09:02.289 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-04T09:09:02.289 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.289 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.289 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.289 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.289 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.290 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.290 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.290 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.290 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.290 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.290 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.290 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.290 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.291 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.291 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.291 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.291 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.291 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.291 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.291 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.291 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.291 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.292 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.292 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.292 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.292 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.292 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.292 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.292 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.292 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.293 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.293 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.293 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.293 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.293 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.293 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.293 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.294 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.294 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.294 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.294 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.294 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.294 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.294 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.294 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.295 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.295 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.295 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.295 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.295 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.295 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.295 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.295 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.295 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.296 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.296 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.296 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.296 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.296 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.296 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.296 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.296 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.297 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.297 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.297 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.297 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.297 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.297 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.297 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.297 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_stats_pools": { 2024-03-04T09:09:02.298 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rbd_stats_pools", 2024-03-04T09:09:02.298 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.298 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.298 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.298 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.298 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.298 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.298 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.298 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.299 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.299 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.299 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.299 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-04T09:09:02.299 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-04T09:09:02.299 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.300 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.300 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.300 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "300", 2024-03-04T09:09:02.300 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.300 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.300 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.300 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.300 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.300 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.301 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.301 INFO:teuthology.orchestra.run.smithi071.stdout: "scrape_interval": { 2024-03-04T09:09:02.301 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "scrape_interval", 2024-03-04T09:09:02.301 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-03-04T09:09:02.301 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.301 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.301 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "15.0", 2024-03-04T09:09:02.301 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.302 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.302 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.302 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.302 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.302 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.302 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.302 INFO:teuthology.orchestra.run.smithi071.stdout: "server_addr": { 2024-03-04T09:09:02.303 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_addr", 2024-03-04T09:09:02.303 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.303 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.303 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.303 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "::", 2024-03-04T09:09:02.303 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.303 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.303 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.303 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-04T09:09:02.304 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.304 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.304 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.304 INFO:teuthology.orchestra.run.smithi071.stdout: "server_port": { 2024-03-04T09:09:02.304 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_port", 2024-03-04T09:09:02.304 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.304 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.305 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.305 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "9283", 2024-03-04T09:09:02.305 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.305 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.305 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.305 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-04T09:09:02.305 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.305 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.306 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.306 INFO:teuthology.orchestra.run.smithi071.stdout: "stale_cache_strategy": { 2024-03-04T09:09:02.306 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "stale_cache_strategy", 2024-03-04T09:09:02.306 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.306 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.306 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.306 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "log", 2024-03-04T09:09:02.307 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.307 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.307 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.307 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.307 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.307 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.307 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.307 INFO:teuthology.orchestra.run.smithi071.stdout: "standby_behaviour": { 2024-03-04T09:09:02.308 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "standby_behaviour", 2024-03-04T09:09:02.308 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.308 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.308 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.308 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "default", 2024-03-04T09:09:02.308 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.308 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.308 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.309 INFO:teuthology.orchestra.run.smithi071.stdout: "default", 2024-03-04T09:09:02.309 INFO:teuthology.orchestra.run.smithi071.stdout: "error" 2024-03-04T09:09:02.309 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.309 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.309 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.309 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.309 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.310 INFO:teuthology.orchestra.run.smithi071.stdout: "standby_error_status_code": { 2024-03-04T09:09:02.310 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "standby_error_status_code", 2024-03-04T09:09:02.310 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.310 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.310 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.310 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "500", 2024-03-04T09:09:02.310 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "400", 2024-03-04T09:09:02.310 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "599", 2024-03-04T09:09:02.310 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.311 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.311 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.311 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.311 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.311 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.311 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.312 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rbd_support", 2024-03-04T09:09:02.312 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.312 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.312 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.312 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.312 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.312 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.312 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.313 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.313 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.313 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.313 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.313 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.313 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.313 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.313 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.313 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.314 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.314 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.314 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.314 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.314 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.314 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.314 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.315 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.315 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.315 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.315 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.315 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.315 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.315 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.315 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.316 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.316 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.316 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.316 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.316 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.316 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.316 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.317 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.317 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.317 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.317 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.317 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.317 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.317 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.317 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.318 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.318 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.318 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.318 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.318 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.318 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.318 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.318 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.318 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.319 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.319 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.319 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.319 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.319 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.319 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.319 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.320 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.320 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.320 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.320 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.320 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.320 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.320 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.320 INFO:teuthology.orchestra.run.smithi071.stdout: "max_concurrent_snap_create": { 2024-03-04T09:09:02.321 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_concurrent_snap_create", 2024-03-04T09:09:02.321 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.321 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.321 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.321 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-03-04T09:09:02.321 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.321 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.321 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.322 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.322 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.322 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.322 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.322 INFO:teuthology.orchestra.run.smithi071.stdout: "mirror_snapshot_schedule": { 2024-03-04T09:09:02.322 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mirror_snapshot_schedule", 2024-03-04T09:09:02.322 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.322 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.323 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.323 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.323 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.323 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.323 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.323 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.323 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.323 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.324 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.324 INFO:teuthology.orchestra.run.smithi071.stdout: "trash_purge_schedule": { 2024-03-04T09:09:02.324 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "trash_purge_schedule", 2024-03-04T09:09:02.324 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.324 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.324 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.324 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.325 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.325 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.325 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.325 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.325 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.325 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.325 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.325 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.325 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.326 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "restful", 2024-03-04T09:09:02.326 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.326 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.326 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.326 INFO:teuthology.orchestra.run.smithi071.stdout: "enable_auth": { 2024-03-04T09:09:02.326 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "enable_auth", 2024-03-04T09:09:02.326 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.327 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.327 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.327 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:02.327 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.327 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.327 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.327 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.328 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.328 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.328 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.328 INFO:teuthology.orchestra.run.smithi071.stdout: "key_file": { 2024-03-04T09:09:02.328 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "key_file", 2024-03-04T09:09:02.328 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.328 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.328 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.329 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.329 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.329 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.329 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.329 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.329 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.329 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.329 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.330 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.330 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.330 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.330 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.330 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.330 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.330 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.330 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.331 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.331 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.331 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.331 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.331 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.331 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.331 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.331 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.331 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.332 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.332 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.332 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.332 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.332 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.332 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.332 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.332 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.333 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.333 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.333 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.333 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.333 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.333 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.333 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.333 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.334 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.334 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.334 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.334 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.334 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.334 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.334 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.335 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.335 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.335 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.335 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.335 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.335 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.335 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.335 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.335 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.336 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.336 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.336 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.336 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.336 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.336 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.336 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.337 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.337 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.337 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.337 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.337 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.337 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.337 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.337 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.337 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.338 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.338 INFO:teuthology.orchestra.run.smithi071.stdout: "server_addr": { 2024-03-04T09:09:02.338 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_addr", 2024-03-04T09:09:02.338 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.338 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.338 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.338 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.339 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.339 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.339 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.339 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.339 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.339 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.339 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.339 INFO:teuthology.orchestra.run.smithi071.stdout: "server_port": { 2024-03-04T09:09:02.340 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_port", 2024-03-04T09:09:02.340 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.340 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.340 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.340 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.340 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.340 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.340 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.341 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.341 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.341 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.341 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.341 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.341 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.342 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rgw", 2024-03-04T09:09:02.342 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.342 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.342 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.342 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.342 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.342 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.342 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.342 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.343 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.343 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.343 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.343 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.343 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.343 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.343 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.343 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.344 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.344 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.344 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.344 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.344 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.344 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.344 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.345 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.345 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.345 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.345 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.345 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.345 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.345 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.345 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.346 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.346 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.346 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.346 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.346 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.346 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.346 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.346 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.347 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.347 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.347 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.347 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.347 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.347 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.347 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.347 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.348 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.348 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.348 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.348 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.348 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.348 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.348 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.348 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.349 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.349 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.349 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.349 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.349 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.349 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.349 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.350 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.350 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.350 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.350 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.350 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.350 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.350 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.350 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.350 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.351 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.351 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rook", 2024-03-04T09:09:02.351 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.351 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.351 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.351 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.351 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.352 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.352 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.352 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.352 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.355 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.355 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.355 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.355 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.355 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.355 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.356 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.356 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.356 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.356 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.356 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.356 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.356 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.356 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.357 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.357 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.357 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.357 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.357 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.357 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.357 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.357 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.358 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.358 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.358 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.358 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.358 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.358 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.358 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.358 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.359 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.359 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.359 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.359 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.359 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.359 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.359 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.359 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.360 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.360 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.360 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.360 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.360 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.360 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.360 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.360 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.361 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.361 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.361 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.361 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.361 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.361 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.361 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.361 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.362 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.362 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.362 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.362 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.362 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.362 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.362 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.362 INFO:teuthology.orchestra.run.smithi071.stdout: "storage_class": { 2024-03-04T09:09:02.362 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "storage_class", 2024-03-04T09:09:02.363 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.363 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.363 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.363 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "local", 2024-03-04T09:09:02.363 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.363 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.363 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.363 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-04T09:09:02.363 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.364 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.364 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.364 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.364 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.364 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "selftest", 2024-03-04T09:09:02.364 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.365 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.365 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.365 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.365 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.365 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.365 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.365 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.365 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.365 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.366 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.366 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.366 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.366 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.366 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.366 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.366 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.366 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.367 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.367 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.367 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.367 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.367 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.367 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.367 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.367 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.368 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.368 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.368 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.368 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.368 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.368 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.368 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.368 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.369 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.369 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.369 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.369 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.369 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.369 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.369 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.369 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.370 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.370 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.370 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.370 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.370 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.370 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.370 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.370 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.371 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.371 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.371 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.371 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.371 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.371 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.371 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.371 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.372 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.372 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.372 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.372 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.372 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.372 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.372 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.372 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.373 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.373 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.373 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.373 INFO:teuthology.orchestra.run.smithi071.stdout: "roption1": { 2024-03-04T09:09:02.373 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "roption1", 2024-03-04T09:09:02.373 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.373 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.374 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.374 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.374 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.374 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.374 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.374 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.374 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.374 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.374 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.375 INFO:teuthology.orchestra.run.smithi071.stdout: "roption2": { 2024-03-04T09:09:02.375 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "roption2", 2024-03-04T09:09:02.375 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.375 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.375 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.375 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "xyz", 2024-03-04T09:09:02.375 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.375 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.376 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.376 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.376 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.376 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.376 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.376 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption1": { 2024-03-04T09:09:02.376 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption1", 2024-03-04T09:09:02.377 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.377 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.377 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.377 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.377 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.377 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.377 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.377 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.377 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.378 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.378 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.378 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption2": { 2024-03-04T09:09:02.378 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption2", 2024-03-04T09:09:02.378 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.378 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.378 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.379 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.379 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.379 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.379 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.379 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.379 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.379 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.379 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.380 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption3": { 2024-03-04T09:09:02.380 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption3", 2024-03-04T09:09:02.380 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-03-04T09:09:02.380 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.380 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.380 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.380 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.380 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.381 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.381 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.381 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.381 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.381 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.381 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption4": { 2024-03-04T09:09:02.381 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption4", 2024-03-04T09:09:02.381 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.382 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.382 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.382 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.382 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.382 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.382 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.382 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.382 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.383 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.383 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.383 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption5": { 2024-03-04T09:09:02.383 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption5", 2024-03-04T09:09:02.383 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.383 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.383 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.383 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.384 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.384 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.384 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.384 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.384 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.384 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.384 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.385 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption6": { 2024-03-04T09:09:02.385 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption6", 2024-03-04T09:09:02.385 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.385 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.385 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.385 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:02.385 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.385 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.386 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.386 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.386 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.386 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.386 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.386 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption7": { 2024-03-04T09:09:02.386 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption7", 2024-03-04T09:09:02.386 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.387 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.387 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.387 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.387 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1", 2024-03-04T09:09:02.387 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "42", 2024-03-04T09:09:02.387 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.387 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.387 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.387 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.388 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.388 INFO:teuthology.orchestra.run.smithi071.stdout: "testkey": { 2024-03-04T09:09:02.388 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "testkey", 2024-03-04T09:09:02.388 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.388 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.388 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.389 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.389 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.389 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.389 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.389 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.389 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.389 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.389 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.390 INFO:teuthology.orchestra.run.smithi071.stdout: "testlkey": { 2024-03-04T09:09:02.390 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "testlkey", 2024-03-04T09:09:02.390 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.390 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.390 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.390 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.390 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.390 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.391 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.391 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.391 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.391 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.391 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.391 INFO:teuthology.orchestra.run.smithi071.stdout: "testnewline": { 2024-03-04T09:09:02.391 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "testnewline", 2024-03-04T09:09:02.391 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.392 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.392 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.392 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.392 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.392 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.392 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.392 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.392 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.393 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.393 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.393 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.393 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.393 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "snap_schedule", 2024-03-04T09:09:02.393 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.393 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.394 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.394 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_m_granularity": { 2024-03-04T09:09:02.394 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "allow_m_granularity", 2024-03-04T09:09:02.394 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.394 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.394 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.394 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.394 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.395 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.395 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.395 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "allow minute scheduled snapshots", 2024-03-04T09:09:02.395 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.395 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.395 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.395 INFO:teuthology.orchestra.run.smithi071.stdout: "dump_on_update": { 2024-03-04T09:09:02.395 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "dump_on_update", 2024-03-04T09:09:02.396 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.396 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.396 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.396 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.396 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.396 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.396 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.396 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "dump database to debug log on update", 2024-03-04T09:09:02.397 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.397 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.397 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.397 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.397 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.397 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.397 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.397 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.398 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.398 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.398 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.398 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.398 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.398 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.398 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.398 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.399 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.399 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.399 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.399 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.399 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.399 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.399 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.399 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.400 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.400 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.400 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.400 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.400 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.400 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.400 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.400 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.401 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.401 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.401 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.401 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.401 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.401 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.401 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.401 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.402 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.402 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.402 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.402 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.402 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.402 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.402 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.402 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.403 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.403 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.403 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.403 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.403 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.403 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.403 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.403 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.404 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.404 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.404 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.404 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.404 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.404 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.404 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.404 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.404 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.405 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.405 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.405 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.405 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.405 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.405 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.406 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "stats", 2024-03-04T09:09:02.406 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.406 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.406 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.406 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.406 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.406 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.406 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.406 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.407 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.407 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.407 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.407 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.407 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.407 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.407 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.407 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.408 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.408 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.408 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.408 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.408 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.408 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.408 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.408 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.409 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.409 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.409 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.409 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.409 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.409 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.409 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.409 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.410 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.410 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.410 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.410 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.410 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.410 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.410 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.410 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.411 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.411 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.411 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.411 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.411 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.411 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.411 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.411 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.411 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.412 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.412 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.412 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.412 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.412 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.412 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.412 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.413 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.413 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.413 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.413 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.413 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.413 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.413 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.413 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.414 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.414 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.414 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.414 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.414 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.414 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.414 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.415 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "status", 2024-03-04T09:09:02.415 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.415 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.415 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.415 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.415 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.415 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.415 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.416 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.416 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.416 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.416 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.416 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.416 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.416 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.416 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.416 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.417 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.417 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.417 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.417 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.417 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.417 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.417 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.418 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.418 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.418 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.418 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.418 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.418 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.418 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.418 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.419 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.419 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.419 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.419 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.419 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.419 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.419 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.419 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.420 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.420 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.420 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.420 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.420 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.420 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.420 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.420 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.421 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.421 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.421 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.421 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.421 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.421 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.421 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.421 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.421 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.422 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.422 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.422 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.422 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.422 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.422 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.422 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.422 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.423 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.423 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.423 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.423 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.423 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.423 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.423 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.424 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "telegraf", 2024-03-04T09:09:02.424 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.424 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.424 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.424 INFO:teuthology.orchestra.run.smithi071.stdout: "address": { 2024-03-04T09:09:02.424 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "address", 2024-03-04T09:09:02.424 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.424 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.425 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.425 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-04T09:09:02.425 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.425 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.425 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.425 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.425 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.425 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.426 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.426 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-03-04T09:09:02.426 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-03-04T09:09:02.426 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:02.426 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.426 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.426 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "15", 2024-03-04T09:09:02.426 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.427 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.427 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.427 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.427 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.427 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.427 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.427 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.428 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.428 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.428 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.428 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.428 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.428 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.428 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.428 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.428 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.429 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.429 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.429 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.429 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.429 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.429 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.429 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.429 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.430 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.430 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.430 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.430 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.430 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.430 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.430 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.430 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.431 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.431 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.431 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.431 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.431 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.431 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.431 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.432 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.432 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.432 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.432 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.432 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.432 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.432 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.432 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.433 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.433 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.433 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.433 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.433 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.433 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.433 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.433 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.434 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.434 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.434 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.434 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.434 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.434 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.434 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.434 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.435 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.435 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.435 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.435 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.435 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.435 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.435 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.435 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.436 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.436 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.436 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.436 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "telemetry", 2024-03-04T09:09:02.436 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.436 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.437 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.437 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_basic": { 2024-03-04T09:09:02.437 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_basic", 2024-03-04T09:09:02.437 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.437 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.437 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.437 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:02.437 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.437 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.438 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.438 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-04T09:09:02.438 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.438 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.438 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.438 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_crash": { 2024-03-04T09:09:02.438 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_crash", 2024-03-04T09:09:02.439 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.439 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.439 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.439 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:02.439 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.439 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.439 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.439 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-04T09:09:02.439 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.440 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.440 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.440 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_device": { 2024-03-04T09:09:02.440 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_device", 2024-03-04T09:09:02.440 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.440 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.440 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.441 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:02.441 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.441 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.441 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.441 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-04T09:09:02.441 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.441 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.441 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.442 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_ident": { 2024-03-04T09:09:02.442 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_ident", 2024-03-04T09:09:02.442 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.442 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.442 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.442 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.442 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.442 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.443 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.443 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-04T09:09:02.443 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.443 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.443 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.443 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_perf": { 2024-03-04T09:09:02.443 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_perf", 2024-03-04T09:09:02.444 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.444 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.444 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.444 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.444 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.444 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.444 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.444 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-04T09:09:02.445 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.445 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.445 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.445 INFO:teuthology.orchestra.run.smithi071.stdout: "contact": { 2024-03-04T09:09:02.445 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "contact", 2024-03-04T09:09:02.445 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.445 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.445 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.446 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.446 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.446 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.446 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.446 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.446 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.446 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.446 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.447 INFO:teuthology.orchestra.run.smithi071.stdout: "description": { 2024-03-04T09:09:02.447 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "description", 2024-03-04T09:09:02.447 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.447 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.447 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.447 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.447 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.447 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.448 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.448 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.448 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.448 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.448 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.448 INFO:teuthology.orchestra.run.smithi071.stdout: "device_url": { 2024-03-04T09:09:02.448 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "device_url", 2024-03-04T09:09:02.449 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.449 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.449 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.449 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-04T09:09:02.449 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.449 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.449 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.449 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.450 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.450 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.450 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.450 INFO:teuthology.orchestra.run.smithi071.stdout: "enabled": { 2024-03-04T09:09:02.450 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "enabled", 2024-03-04T09:09:02.450 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.450 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.450 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.451 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.451 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.451 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.451 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.451 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.451 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.451 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.451 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.452 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.452 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-03-04T09:09:02.452 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-03-04T09:09:02.452 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.452 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.452 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.452 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "24", 2024-03-04T09:09:02.452 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "8", 2024-03-04T09:09:02.453 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.453 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.453 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.453 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.453 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.453 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.453 INFO:teuthology.orchestra.run.smithi071.stdout: "last_opt_revision": { 2024-03-04T09:09:02.453 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "last_opt_revision", 2024-03-04T09:09:02.454 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.454 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.454 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.454 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "1", 2024-03-04T09:09:02.454 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.454 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.454 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.454 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.455 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.455 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.455 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.455 INFO:teuthology.orchestra.run.smithi071.stdout: "leaderboard": { 2024-03-04T09:09:02.455 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "leaderboard", 2024-03-04T09:09:02.455 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.455 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.455 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.456 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.456 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.456 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.456 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.456 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.456 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.456 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.456 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.457 INFO:teuthology.orchestra.run.smithi071.stdout: "leaderboard_description": { 2024-03-04T09:09:02.457 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "leaderboard_description", 2024-03-04T09:09:02.457 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.457 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.457 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.457 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.457 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.458 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.458 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.458 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.458 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.458 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.458 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.458 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.458 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.459 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.459 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.459 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.459 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.459 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.459 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.459 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.459 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.460 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.460 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.460 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.460 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.460 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.460 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.460 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.460 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.460 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.461 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.461 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.461 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.461 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.461 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.461 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.461 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.461 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.462 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.462 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.462 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.462 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.462 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.462 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.462 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.463 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.463 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.463 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.463 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.463 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.463 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.463 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.463 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.464 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.464 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.464 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.464 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.464 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.464 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.464 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.464 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.464 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.465 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.465 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.465 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.465 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.465 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.465 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.465 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.466 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.466 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.466 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.466 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.466 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.466 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.466 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.466 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.467 INFO:teuthology.orchestra.run.smithi071.stdout: "organization": { 2024-03-04T09:09:02.467 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "organization", 2024-03-04T09:09:02.467 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.467 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.467 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.467 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.467 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.467 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.467 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.468 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.468 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.468 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.468 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.468 INFO:teuthology.orchestra.run.smithi071.stdout: "proxy": { 2024-03-04T09:09:02.468 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "proxy", 2024-03-04T09:09:02.468 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.469 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.469 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.469 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.469 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.469 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.469 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.469 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.469 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.469 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.470 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.470 INFO:teuthology.orchestra.run.smithi071.stdout: "url": { 2024-03-04T09:09:02.470 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "url", 2024-03-04T09:09:02.470 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.470 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.470 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.470 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-04T09:09:02.470 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.471 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.471 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.471 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.471 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.471 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.471 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.471 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.471 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.472 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.472 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "test_orchestrator", 2024-03-04T09:09:02.472 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.472 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.472 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.472 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.472 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.473 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.473 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.473 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.473 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.473 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.473 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.473 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.473 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.473 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.474 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.474 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.474 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.474 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.474 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.474 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.474 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.474 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.475 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.475 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.475 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.475 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.475 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.475 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.475 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.475 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.476 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.476 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.476 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.476 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.476 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.476 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.476 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.477 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.477 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.477 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.477 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.477 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.477 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.477 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.477 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.477 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.478 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.478 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.478 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.478 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.478 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.478 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.478 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.478 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.478 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.478 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.479 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.479 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.479 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.479 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.479 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.479 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.479 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.480 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.480 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.480 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.480 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.480 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.480 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.480 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.480 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.481 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "volumes", 2024-03-04T09:09:02.481 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.481 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.481 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.481 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.481 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.481 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.482 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.482 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.482 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.482 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.482 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.482 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.482 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.482 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.482 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.483 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.483 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.483 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.483 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.483 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.483 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.483 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.484 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.484 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.484 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.484 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.484 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.484 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.484 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.485 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.485 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.485 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.485 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.485 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.485 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.485 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.485 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.486 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.486 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.486 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.486 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.486 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.486 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.486 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.486 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.487 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.487 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.487 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.487 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.487 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.487 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.487 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.487 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.488 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.488 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.488 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.488 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.488 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.488 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.488 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.488 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.489 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.489 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.489 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.489 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.489 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.489 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.489 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.489 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.490 INFO:teuthology.orchestra.run.smithi071.stdout: "max_concurrent_clones": { 2024-03-04T09:09:02.490 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_concurrent_clones", 2024-03-04T09:09:02.490 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.490 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.490 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.490 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "4", 2024-03-04T09:09:02.490 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.490 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.491 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.491 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-04T09:09:02.491 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.491 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.491 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.491 INFO:teuthology.orchestra.run.smithi071.stdout: "periodic_async_work": { 2024-03-04T09:09:02.491 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "periodic_async_work", 2024-03-04T09:09:02.492 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.492 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.492 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.492 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.492 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.492 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.492 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.492 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Periodically check for async work", 2024-03-04T09:09:02.492 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.493 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.493 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.493 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot_clone_delay": { 2024-03-04T09:09:02.493 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "snapshot_clone_delay", 2024-03-04T09:09:02.493 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.493 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.493 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.493 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-03-04T09:09:02.494 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.494 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.494 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.494 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-04T09:09:02.494 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.494 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.494 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.494 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot_clone_no_wait": { 2024-03-04T09:09:02.495 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "snapshot_clone_no_wait", 2024-03-04T09:09:02.495 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.495 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.495 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.495 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-03-04T09:09:02.495 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.495 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.495 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.496 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-04T09:09:02.496 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.496 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.496 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.496 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.496 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.497 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "zabbix", 2024-03-04T09:09:02.497 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-03-04T09:09:02.497 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-03-04T09:09:02.497 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-03-04T09:09:02.497 INFO:teuthology.orchestra.run.smithi071.stdout: "discovery_interval": { 2024-03-04T09:09:02.497 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "discovery_interval", 2024-03-04T09:09:02.497 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-03-04T09:09:02.497 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.498 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.498 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "100", 2024-03-04T09:09:02.498 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.498 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.498 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.498 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.498 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.498 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.498 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.499 INFO:teuthology.orchestra.run.smithi071.stdout: "identifier": { 2024-03-04T09:09:02.499 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "identifier", 2024-03-04T09:09:02.499 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.499 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.499 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.499 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.499 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.500 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.500 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.500 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.500 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.500 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.500 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.500 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-03-04T09:09:02.501 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-03-04T09:09:02.501 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-03-04T09:09:02.501 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.501 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.501 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-03-04T09:09:02.501 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.501 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.501 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.502 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.502 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.502 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.502 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.502 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-03-04T09:09:02.502 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-03-04T09:09:02.502 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.503 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.503 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.503 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.503 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.503 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.503 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.503 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.503 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.503 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.504 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.504 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.504 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.504 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.504 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.504 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.504 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.504 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.505 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-03-04T09:09:02.505 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-03-04T09:09:02.505 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.505 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.505 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.505 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.505 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.506 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.506 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.506 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.506 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.506 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.506 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.506 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-03-04T09:09:02.507 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-03-04T09:09:02.507 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.507 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.507 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.507 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-03-04T09:09:02.507 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.507 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.507 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-03-04T09:09:02.508 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-03-04T09:09:02.508 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-03-04T09:09:02.508 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-03-04T09:09:02.508 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-03-04T09:09:02.508 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-03-04T09:09:02.508 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-03-04T09:09:02.508 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.508 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.509 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.509 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.509 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.509 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-03-04T09:09:02.509 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-03-04T09:09:02.509 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-03-04T09:09:02.509 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.509 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-03-04T09:09:02.510 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-03-04T09:09:02.510 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.510 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.510 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.510 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.510 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.510 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.510 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.511 INFO:teuthology.orchestra.run.smithi071.stdout: "zabbix_host": { 2024-03-04T09:09:02.511 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "zabbix_host", 2024-03-04T09:09:02.511 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.511 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.511 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.511 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-03-04T09:09:02.511 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.511 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.511 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.512 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.512 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.512 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.512 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.512 INFO:teuthology.orchestra.run.smithi071.stdout: "zabbix_port": { 2024-03-04T09:09:02.512 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "zabbix_port", 2024-03-04T09:09:02.512 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-03-04T09:09:02.513 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.513 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.513 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10051", 2024-03-04T09:09:02.513 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.513 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.513 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.513 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.513 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.514 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.514 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.514 INFO:teuthology.orchestra.run.smithi071.stdout: "zabbix_sender": { 2024-03-04T09:09:02.514 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "zabbix_sender", 2024-03-04T09:09:02.514 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-03-04T09:09:02.514 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-03-04T09:09:02.514 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-03-04T09:09:02.514 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-04T09:09:02.514 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-03-04T09:09:02.515 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-03-04T09:09:02.515 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-03-04T09:09:02.515 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-03-04T09:09:02.515 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-03-04T09:09:02.515 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-03-04T09:09:02.515 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-03-04T09:09:02.515 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.515 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.516 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.516 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.516 INFO:teuthology.orchestra.run.smithi071.stdout: "services": { 2024-03-04T09:09:02.516 INFO:teuthology.orchestra.run.smithi071.stdout: "dashboard": "https://172.21.15.71:8443/" 2024-03-04T09:09:02.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.516 INFO:teuthology.orchestra.run.smithi071.stdout: "always_on_modules": { 2024-03-04T09:09:02.516 INFO:teuthology.orchestra.run.smithi071.stdout: "octopus": [ 2024-03-04T09:09:02.516 INFO:teuthology.orchestra.run.smithi071.stdout: "balancer", 2024-03-04T09:09:02.517 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-03-04T09:09:02.517 INFO:teuthology.orchestra.run.smithi071.stdout: "devicehealth", 2024-03-04T09:09:02.517 INFO:teuthology.orchestra.run.smithi071.stdout: "orchestrator", 2024-03-04T09:09:02.517 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscaler", 2024-03-04T09:09:02.517 INFO:teuthology.orchestra.run.smithi071.stdout: "progress", 2024-03-04T09:09:02.517 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_support", 2024-03-04T09:09:02.517 INFO:teuthology.orchestra.run.smithi071.stdout: "status", 2024-03-04T09:09:02.517 INFO:teuthology.orchestra.run.smithi071.stdout: "telemetry", 2024-03-04T09:09:02.518 INFO:teuthology.orchestra.run.smithi071.stdout: "volumes" 2024-03-04T09:09:02.518 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.518 INFO:teuthology.orchestra.run.smithi071.stdout: "pacific": [ 2024-03-04T09:09:02.518 INFO:teuthology.orchestra.run.smithi071.stdout: "balancer", 2024-03-04T09:09:02.518 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-03-04T09:09:02.518 INFO:teuthology.orchestra.run.smithi071.stdout: "devicehealth", 2024-03-04T09:09:02.518 INFO:teuthology.orchestra.run.smithi071.stdout: "orchestrator", 2024-03-04T09:09:02.518 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscaler", 2024-03-04T09:09:02.518 INFO:teuthology.orchestra.run.smithi071.stdout: "progress", 2024-03-04T09:09:02.519 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_support", 2024-03-04T09:09:02.519 INFO:teuthology.orchestra.run.smithi071.stdout: "status", 2024-03-04T09:09:02.519 INFO:teuthology.orchestra.run.smithi071.stdout: "telemetry", 2024-03-04T09:09:02.519 INFO:teuthology.orchestra.run.smithi071.stdout: "volumes" 2024-03-04T09:09:02.519 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.519 INFO:teuthology.orchestra.run.smithi071.stdout: "quincy": [ 2024-03-04T09:09:02.519 INFO:teuthology.orchestra.run.smithi071.stdout: "balancer", 2024-03-04T09:09:02.519 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-03-04T09:09:02.520 INFO:teuthology.orchestra.run.smithi071.stdout: "devicehealth", 2024-03-04T09:09:02.520 INFO:teuthology.orchestra.run.smithi071.stdout: "orchestrator", 2024-03-04T09:09:02.520 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscaler", 2024-03-04T09:09:02.520 INFO:teuthology.orchestra.run.smithi071.stdout: "progress", 2024-03-04T09:09:02.520 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_support", 2024-03-04T09:09:02.520 INFO:teuthology.orchestra.run.smithi071.stdout: "status", 2024-03-04T09:09:02.520 INFO:teuthology.orchestra.run.smithi071.stdout: "telemetry", 2024-03-04T09:09:02.520 INFO:teuthology.orchestra.run.smithi071.stdout: "volumes" 2024-03-04T09:09:02.520 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.521 INFO:teuthology.orchestra.run.smithi071.stdout: "reef": [ 2024-03-04T09:09:02.521 INFO:teuthology.orchestra.run.smithi071.stdout: "balancer", 2024-03-04T09:09:02.521 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-03-04T09:09:02.521 INFO:teuthology.orchestra.run.smithi071.stdout: "devicehealth", 2024-03-04T09:09:02.521 INFO:teuthology.orchestra.run.smithi071.stdout: "orchestrator", 2024-03-04T09:09:02.521 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscaler", 2024-03-04T09:09:02.521 INFO:teuthology.orchestra.run.smithi071.stdout: "progress", 2024-03-04T09:09:02.521 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_support", 2024-03-04T09:09:02.522 INFO:teuthology.orchestra.run.smithi071.stdout: "status", 2024-03-04T09:09:02.522 INFO:teuthology.orchestra.run.smithi071.stdout: "telemetry", 2024-03-04T09:09:02.522 INFO:teuthology.orchestra.run.smithi071.stdout: "volumes" 2024-03-04T09:09:02.522 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-03-04T09:09:02.522 INFO:teuthology.orchestra.run.smithi071.stdout: "squid": [ 2024-03-04T09:09:02.522 INFO:teuthology.orchestra.run.smithi071.stdout: "balancer", 2024-03-04T09:09:02.522 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-03-04T09:09:02.522 INFO:teuthology.orchestra.run.smithi071.stdout: "devicehealth", 2024-03-04T09:09:02.523 INFO:teuthology.orchestra.run.smithi071.stdout: "orchestrator", 2024-03-04T09:09:02.523 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscaler", 2024-03-04T09:09:02.523 INFO:teuthology.orchestra.run.smithi071.stdout: "progress", 2024-03-04T09:09:02.523 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_support", 2024-03-04T09:09:02.523 INFO:teuthology.orchestra.run.smithi071.stdout: "status", 2024-03-04T09:09:02.523 INFO:teuthology.orchestra.run.smithi071.stdout: "telemetry", 2024-03-04T09:09:02.523 INFO:teuthology.orchestra.run.smithi071.stdout: "volumes" 2024-03-04T09:09:02.523 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-03-04T09:09:02.524 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.524 INFO:teuthology.orchestra.run.smithi071.stdout: "last_failure_osd_epoch": 96, 2024-03-04T09:09:02.524 INFO:teuthology.orchestra.run.smithi071.stdout: "active_clients": [ 2024-03-04T09:09:02.524 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.524 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "devicehealth", 2024-03-04T09:09:02.524 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-03-04T09:09:02.524 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.524 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-03-04T09:09:02.524 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:0", 2024-03-04T09:09:02.525 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2506279430 2024-03-04T09:09:02.525 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.525 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-03-04T09:09:02.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.525 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "libcephsqlite", 2024-03-04T09:09:02.525 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-03-04T09:09:02.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.526 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-03-04T09:09:02.526 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:0", 2024-03-04T09:09:02.526 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3668144808 2024-03-04T09:09:02.526 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.526 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-03-04T09:09:02.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.526 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rbd_support", 2024-03-04T09:09:02.526 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-03-04T09:09:02.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.527 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-03-04T09:09:02.527 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:0", 2024-03-04T09:09:02.527 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 71876011 2024-03-04T09:09:02.527 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.527 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-03-04T09:09:02.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.528 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "snap_schedule", 2024-03-04T09:09:02.528 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-03-04T09:09:02.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.528 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-03-04T09:09:02.528 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:0", 2024-03-04T09:09:02.528 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 758132905 2024-03-04T09:09:02.528 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.528 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-03-04T09:09:02.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.529 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "snap_schedule", 2024-03-04T09:09:02.529 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-03-04T09:09:02.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.529 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-03-04T09:09:02.529 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:0", 2024-03-04T09:09:02.529 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1454501450 2024-03-04T09:09:02.529 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.530 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-03-04T09:09:02.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:02.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.530 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "volumes", 2024-03-04T09:09:02.530 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-03-04T09:09:02.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-03-04T09:09:02.530 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-03-04T09:09:02.530 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:0", 2024-03-04T09:09:02.530 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1088897835 2024-03-04T09:09:02.531 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.531 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-03-04T09:09:02.531 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:02.531 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-03-04T09:09:02.531 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-03-04T09:09:02.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.363+0000 7f9591483640 1 -- 172.21.15.71:0/3246742560 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f95740a1ef0 msgr2=0x7f95740a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:02.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.363+0000 7f9591483640 1 --2- 172.21.15.71:0/3246742560 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f95740a1ef0 0x7f95740a43b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9580000c00 tx=0x7f958002e040 comp rx=0 tx=0).stop 2024-03-04T09:09:02.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.363+0000 7f9591483640 1 -- 172.21.15.71:0/3246742560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f958c101580 msgr2=0x7f958c197d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:02.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.363+0000 7f9591483640 1 --2- 172.21.15.71:0/3246742560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f958c101580 0x7f958c197d80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f957c00c920 tx=0x7f957c00cde0 comp rx=0 tx=0).stop 2024-03-04T09:09:02.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.363+0000 7f9591483640 1 -- 172.21.15.71:0/3246742560 shutdown_connections 2024-03-04T09:09:02.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.363+0000 7f9591483640 1 --2- 172.21.15.71:0/3246742560 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f958c107640 0x7f958c19c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:02.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.363+0000 7f9591483640 1 --2- 172.21.15.71:0/3246742560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f958c101f40 0x7f958c1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:02.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.363+0000 7f9591483640 1 --2- 172.21.15.71:0/3246742560 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f95740a1ef0 0x7f95740a43b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:02.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.363+0000 7f9591483640 1 --2- 172.21.15.71:0/3246742560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f958c101580 0x7f958c197d80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:02.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.364+0000 7f9591483640 1 -- 172.21.15.71:0/3246742560 >> 172.21.15.71:0/3246742560 conn(0x7f958c0789d0 msgr2=0x7f958c100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:02.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.364+0000 7f9591483640 1 -- 172.21.15.71:0/3246742560 shutdown_connections 2024-03-04T09:09:02.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:00.364+0000 7f9591483640 1 -- 172.21.15.71:0/3246742560 wait complete. 2024-03-04T09:09:02.535 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:00 smithi071 ceph-mon[32789]: from='client.15489 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:09:02.535 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:00 smithi071 ceph-mon[32789]: from='client.15495 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:09:02.535 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:00 smithi071 ceph-mon[32789]: pgmap v311: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:09:02.535 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:00 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3627395402' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:09:02.535 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:00 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/382274372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:09:02.535 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:00 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/75774603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:09:02.535 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:00 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/4262240804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:09:02.535 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:00 smithi158 ceph-mon[37241]: from='client.15489 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:09:02.536 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:00 smithi158 ceph-mon[37241]: from='client.15495 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:09:02.536 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:00 smithi158 ceph-mon[37241]: pgmap v311: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:09:02.536 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:00 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3627395402' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:09:02.536 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:00 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/382274372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:09:02.536 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:00 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/75774603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:09:02.536 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:00 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/4262240804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:09:02.536 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:01 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3246742560' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-04T09:09:02.537 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:00 smithi150 ceph-mon[36807]: from='client.15489 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:09:02.537 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:00 smithi150 ceph-mon[36807]: from='client.15495 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:09:02.537 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:00 smithi150 ceph-mon[36807]: pgmap v311: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:09:02.537 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:00 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3627395402' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:09:02.537 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:00 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/382274372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:09:02.537 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:00 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/75774603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:09:02.538 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:00 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/4262240804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:09:02.538 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:01 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3246742560' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-04T09:09:02.538 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:02 smithi150 ceph-mon[36807]: pgmap v312: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 19 op/s 2024-03-04T09:09:02.544 DEBUG:tasks.check_counter:Getting stats from a 2024-03-04T09:09:02.545 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-04T09:09:02.545 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:01 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3246742560' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-04T09:09:02.545 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:02 smithi071 ceph-mon[32789]: pgmap v312: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 19 op/s 2024-03-04T09:09:02.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:02 smithi158 ceph-mon[37241]: pgmap v312: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 19 op/s 2024-03-04T09:09:02.799 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:09:04.228 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:04 smithi071 ceph-mon[32789]: pgmap v313: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 KiB/s wr, 18 op/s 2024-03-04T09:09:04.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:04 smithi150 ceph-mon[36807]: pgmap v313: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 KiB/s wr, 18 op/s 2024-03-04T09:09:04.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:04 smithi158 ceph-mon[37241]: pgmap v313: 129 pgs: 129 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 KiB/s wr, 18 op/s 2024-03-04T09:09:04.654 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-03-04T09:09:04.654 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-03-04T09:09:04.654 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 4, 2024-03-04T09:09:04.654 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 4, 2024-03-04T09:09:04.654 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 2992, 2024-03-04T09:09:04.654 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 448, 2024-03-04T09:09:04.655 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 4, 2024-03-04T09:09:04.655 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 0, 2024-03-04T09:09:04.655 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.001539286, 2024-03-04T09:09:04.655 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.000029463, 2024-03-04T09:09:04.655 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.000609715, 2024-03-04T09:09:04.655 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:09:04.655 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:04.655 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 3, 2024-03-04T09:09:04.655 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000694128, 2024-03-04T09:09:04.656 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000231376 2024-03-04T09:09:04.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:04.656 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:04.656 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-04T09:09:04.656 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-04T09:09:04.656 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-04T09:09:04.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:04.656 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-04T09:09:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 448 2024-03-04T09:09:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-03-04T09:09:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 215, 2024-03-04T09:09:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 205, 2024-03-04T09:09:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 1430352, 2024-03-04T09:09:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 122912, 2024-03-04T09:09:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 2, 2024-03-04T09:09:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 1, 2024-03-04T09:09:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.035456726, 2024-03-04T09:09:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.011349740, 2024-03-04T09:09:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.028032289, 2024-03-04T09:09:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.004053082, 2024-03-04T09:09:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 204, 2024-03-04T09:09:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.008008522, 2024-03-04T09:09:04.659 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000039257 2024-03-04T09:09:04.659 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:04.659 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:04.659 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-04T09:09:04.659 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-04T09:09:04.659 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-04T09:09:04.659 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:04.659 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 1430352, 2024-03-04T09:09:04.659 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 122912 2024-03-04T09:09:04.660 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:04.660 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-03-04T09:09:04.660 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 2, 2024-03-04T09:09:04.660 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 156, 2024-03-04T09:09:04.660 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 320, 2024-03-04T09:09:04.660 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 182384, 2024-03-04T09:09:04.660 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 5, 2024-03-04T09:09:04.660 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 3, 2024-03-04T09:09:04.661 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.011727039, 2024-03-04T09:09:04.661 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.007775251, 2024-03-04T09:09:04.661 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.001865367, 2024-03-04T09:09:04.661 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:09:04.661 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:04.661 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 153, 2024-03-04T09:09:04.661 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.011888798, 2024-03-04T09:09:04.661 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000077704 2024-03-04T09:09:04.662 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:04.662 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:04.662 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-04T09:09:04.662 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-04T09:09:04.662 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-04T09:09:04.662 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:04.662 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-04T09:09:04.662 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 182384 2024-03-04T09:09:04.662 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:04.663 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-03-04T09:09:04.663 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 0, 2024-03-04T09:09:04.663 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-03-04T09:09:04.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:04.663 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-03-04T09:09:04.663 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-03-04T09:09:04.663 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-03-04T09:09:04.663 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-03-04T09:09:04.664 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-03-04T09:09:04.664 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-03-04T09:09:04.664 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-03-04T09:09:04.664 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-04T09:09:04.664 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-03-04T09:09:04.664 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-04T09:09:04.664 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-03-04T09:09:04.664 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-03-04T09:09:04.664 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-03-04T09:09:04.665 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-04T09:09:04.665 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-03-04T09:09:04.665 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-03-04T09:09:04.665 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-03-04T09:09:04.665 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-03-04T09:09:04.665 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-03-04T09:09:04.665 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-04T09:09:04.665 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-03-04T09:09:04.666 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-04T09:09:04.666 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-03-04T09:09:04.666 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-03-04T09:09:04.666 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-03-04T09:09:04.666 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-03-04T09:09:04.666 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-03-04T09:09:04.666 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-04T09:09:04.666 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-03-04T09:09:04.666 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-03-04T09:09:04.667 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-03-04T09:09:04.667 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-03-04T09:09:04.667 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-03-04T09:09:04.667 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-03-04T09:09:04.667 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-03-04T09:09:04.667 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-03-04T09:09:04.667 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-03-04T09:09:04.667 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 149057, 2024-03-04T09:09:04.668 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 32, 2024-03-04T09:09:04.668 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-03-04T09:09:04.668 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-03-04T09:09:04.668 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-03-04T09:09:04.668 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-03-04T09:09:04.668 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-03-04T09:09:04.668 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-03-04T09:09:04.669 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-03-04T09:09:04.669 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-03-04T09:09:04.669 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 0, 2024-03-04T09:09:04.669 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 0, 2024-03-04T09:09:04.669 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-03-04T09:09:04.669 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-03-04T09:09:04.669 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-03-04T09:09:04.669 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-03-04T09:09:04.669 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 928, 2024-03-04T09:09:04.670 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 2, 2024-03-04T09:09:04.670 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-03-04T09:09:04.670 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-03-04T09:09:04.670 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-03-04T09:09:04.670 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-03-04T09:09:04.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:04.670 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-04T09:09:04.670 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-03-04T09:09:04.670 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-03-04T09:09:04.671 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-03-04T09:09:04.671 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 217, 2024-03-04T09:09:04.671 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 1400026, 2024-03-04T09:09:04.671 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-03-04T09:09:04.671 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 217, 2024-03-04T09:09:04.671 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-03-04T09:09:04.671 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-03-04T09:09:04.671 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 217, 2024-03-04T09:09:04.672 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 1400026, 2024-03-04T09:09:04.672 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-03-04T09:09:04.672 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-04T09:09:04.672 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-04T09:09:04.672 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-04T09:09:04.672 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:04.672 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:04.672 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-03-04T09:09:05.334 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:05 smithi071 ceph-mon[32789]: pgmap v314: 129 pgs: 129 active+clean; 440 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 21 op/s 2024-03-04T09:09:05.359 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-03-04T09:09:05.359 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-03-04T09:09:05.359 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-03-04T09:09:05.359 DEBUG:tasks.check_counter:Getting stats from d 2024-03-04T09:09:05.360 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-04T09:09:05.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:05 smithi150 ceph-mon[36807]: pgmap v314: 129 pgs: 129 active+clean; 440 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 21 op/s 2024-03-04T09:09:05.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:05 smithi158 ceph-mon[37241]: pgmap v314: 129 pgs: 129 active+clean; 440 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 21 op/s 2024-03-04T09:09:05.627 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:09:07.414 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:07 smithi071 ceph-mon[32789]: osdmap e103: 12 total, 12 up, 12 in 2024-03-04T09:09:07.414 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:07 smithi071 ceph-mon[32789]: pgmap v316: 129 pgs: 129 active+clean; 440 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-03-04T09:09:07.414 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-03-04T09:09:07.414 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-03-04T09:09:07.414 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 4, 2024-03-04T09:09:07.415 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 4, 2024-03-04T09:09:07.415 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 2992, 2024-03-04T09:09:07.415 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 448, 2024-03-04T09:09:07.415 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 4, 2024-03-04T09:09:07.415 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 0, 2024-03-04T09:09:07.415 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.001466968, 2024-03-04T09:09:07.415 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.000029154, 2024-03-04T09:09:07.415 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.000622807, 2024-03-04T09:09:07.416 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:09:07.416 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:07.416 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 3, 2024-03-04T09:09:07.416 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000696357, 2024-03-04T09:09:07.416 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000232119 2024-03-04T09:09:07.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:07.416 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:07.416 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-04T09:09:07.417 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-04T09:09:07.417 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-04T09:09:07.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:07.417 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-04T09:09:07.417 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 448 2024-03-04T09:09:07.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:07.417 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-03-04T09:09:07.417 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 2, 2024-03-04T09:09:07.418 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 161, 2024-03-04T09:09:07.418 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 320, 2024-03-04T09:09:07.418 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 183264, 2024-03-04T09:09:07.418 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 5, 2024-03-04T09:09:07.418 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 3, 2024-03-04T09:09:07.418 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.012619423, 2024-03-04T09:09:07.418 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.008422715, 2024-03-04T09:09:07.418 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.001978705, 2024-03-04T09:09:07.418 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:09:07.419 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:07.419 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 158, 2024-03-04T09:09:07.419 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.013256879, 2024-03-04T09:09:07.419 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000083904 2024-03-04T09:09:07.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:07.419 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:07.419 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-04T09:09:07.419 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-04T09:09:07.420 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-04T09:09:07.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:07.420 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-04T09:09:07.420 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 183264 2024-03-04T09:09:07.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:07.420 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-03-04T09:09:07.420 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 221, 2024-03-04T09:09:07.420 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 211, 2024-03-04T09:09:07.420 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 1431984, 2024-03-04T09:09:07.421 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 126656, 2024-03-04T09:09:07.421 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 2, 2024-03-04T09:09:07.421 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 1, 2024-03-04T09:09:07.421 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.039101116, 2024-03-04T09:09:07.421 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.011632091, 2024-03-04T09:09:07.421 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.031637702, 2024-03-04T09:09:07.421 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.003962590, 2024-03-04T09:09:07.421 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:07.421 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 210, 2024-03-04T09:09:07.422 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.008016591, 2024-03-04T09:09:07.422 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000038174 2024-03-04T09:09:07.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:07.422 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:07.422 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-04T09:09:07.422 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-04T09:09:07.422 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-04T09:09:07.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:07.423 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 1431984, 2024-03-04T09:09:07.423 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 126656 2024-03-04T09:09:07.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:07.423 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-03-04T09:09:07.423 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 0, 2024-03-04T09:09:07.423 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-03-04T09:09:07.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:07.423 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-03-04T09:09:07.424 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-03-04T09:09:07.424 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-03-04T09:09:07.424 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-03-04T09:09:07.424 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-03-04T09:09:07.424 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-03-04T09:09:07.424 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-03-04T09:09:07.424 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-04T09:09:07.424 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-03-04T09:09:07.425 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-04T09:09:07.425 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-03-04T09:09:07.425 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-03-04T09:09:07.425 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-03-04T09:09:07.425 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-04T09:09:07.425 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-03-04T09:09:07.425 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-03-04T09:09:07.425 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-03-04T09:09:07.425 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-03-04T09:09:07.426 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-03-04T09:09:07.426 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-04T09:09:07.426 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-03-04T09:09:07.426 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-04T09:09:07.426 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-03-04T09:09:07.426 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-03-04T09:09:07.426 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-03-04T09:09:07.426 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-03-04T09:09:07.426 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-03-04T09:09:07.427 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-04T09:09:07.427 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-03-04T09:09:07.427 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-03-04T09:09:07.427 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-03-04T09:09:07.427 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-03-04T09:09:07.427 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-03-04T09:09:07.427 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-03-04T09:09:07.427 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-03-04T09:09:07.427 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-03-04T09:09:07.428 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-03-04T09:09:07.428 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 149017, 2024-03-04T09:09:07.428 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 31, 2024-03-04T09:09:07.428 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-03-04T09:09:07.428 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-03-04T09:09:07.428 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-03-04T09:09:07.428 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-03-04T09:09:07.429 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-03-04T09:09:07.429 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-03-04T09:09:07.429 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-03-04T09:09:07.429 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-03-04T09:09:07.429 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 0, 2024-03-04T09:09:07.429 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 0, 2024-03-04T09:09:07.429 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-03-04T09:09:07.429 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-03-04T09:09:07.430 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-03-04T09:09:07.430 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-03-04T09:09:07.430 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 928, 2024-03-04T09:09:07.430 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 2, 2024-03-04T09:09:07.430 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-03-04T09:09:07.430 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-03-04T09:09:07.430 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-03-04T09:09:07.430 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-03-04T09:09:07.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:07.431 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-04T09:09:07.431 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-03-04T09:09:07.431 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-03-04T09:09:07.431 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-03-04T09:09:07.431 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 223, 2024-03-04T09:09:07.431 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 1400806, 2024-03-04T09:09:07.431 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-03-04T09:09:07.431 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 223, 2024-03-04T09:09:07.432 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-03-04T09:09:07.432 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-03-04T09:09:07.432 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 223, 2024-03-04T09:09:07.432 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 1400806, 2024-03-04T09:09:07.432 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-03-04T09:09:07.432 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-04T09:09:07.432 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-04T09:09:07.432 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-04T09:09:07.433 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:07.433 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:07.433 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-03-04T09:09:07.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:07 smithi150 ceph-mon[36807]: osdmap e103: 12 total, 12 up, 12 in 2024-03-04T09:09:07.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:07 smithi150 ceph-mon[36807]: pgmap v316: 129 pgs: 129 active+clean; 440 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-03-04T09:09:07.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:07 smithi158 ceph-mon[37241]: osdmap e103: 12 total, 12 up, 12 in 2024-03-04T09:09:07.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:07 smithi158 ceph-mon[37241]: pgmap v316: 129 pgs: 129 active+clean; 440 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-03-04T09:09:08.067 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-03-04T09:09:08.067 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-03-04T09:09:08.067 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-03-04T09:09:08.068 DEBUG:tasks.check_counter:Getting stats from g 2024-03-04T09:09:08.068 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-04T09:09:08.321 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:09:08.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:08 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:09:08.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:08 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:09:08.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:08 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:09:09.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:09 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:09.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:09 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:09.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:09 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:09.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:09 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:09.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:09 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:09.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:09 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:09.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:09 smithi158 ceph-mon[37241]: pgmap v317: 129 pgs: 129 active+clean; 440 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-03-04T09:09:09.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:09 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:09.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:09 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:09.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:09 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:09.780 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:09 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:09.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:09 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:09.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:09 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:09.781 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:09 smithi150 ceph-mon[36807]: pgmap v317: 129 pgs: 129 active+clean; 440 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-03-04T09:09:09.810 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:09 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:09.810 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:09 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:09.811 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:09 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:09.811 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:09 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:09.811 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:09 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:09.811 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:09 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:09.811 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:09 smithi071 ceph-mon[32789]: pgmap v317: 129 pgs: 129 active+clean; 440 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-03-04T09:09:10.095 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-03-04T09:09:10.095 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-03-04T09:09:10.096 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 0, 2024-03-04T09:09:10.096 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 2, 2024-03-04T09:09:10.096 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 0, 2024-03-04T09:09:10.096 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 0, 2024-03-04T09:09:10.096 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 4, 2024-03-04T09:09:10.096 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 0, 2024-03-04T09:09:10.096 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.001237182, 2024-03-04T09:09:10.096 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.000000000, 2024-03-04T09:09:10.097 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.000331273, 2024-03-04T09:09:10.097 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:09:10.097 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:10.097 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-04T09:09:10.097 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-04T09:09:10.097 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-04T09:09:10.097 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:10.097 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:10.097 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-04T09:09:10.098 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-04T09:09:10.098 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-04T09:09:10.098 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:10.098 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-04T09:09:10.098 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 0 2024-03-04T09:09:10.098 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:10.098 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-03-04T09:09:10.098 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 6, 2024-03-04T09:09:10.098 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 160, 2024-03-04T09:09:10.099 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 3312, 2024-03-04T09:09:10.099 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 183184, 2024-03-04T09:09:10.099 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 5, 2024-03-04T09:09:10.099 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 3, 2024-03-04T09:09:10.099 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.012338354, 2024-03-04T09:09:10.099 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.007783427, 2024-03-04T09:09:10.099 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.001910006, 2024-03-04T09:09:10.099 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:09:10.100 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:10.100 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 158, 2024-03-04T09:09:10.100 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.014655883, 2024-03-04T09:09:10.100 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000092758 2024-03-04T09:09:10.100 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:10.100 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:10.100 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-04T09:09:10.100 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-04T09:09:10.100 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-04T09:09:10.101 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:10.101 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-03-04T09:09:10.101 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 183184 2024-03-04T09:09:10.101 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:10.101 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-03-04T09:09:10.101 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 217, 2024-03-04T09:09:10.101 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 207, 2024-03-04T09:09:10.101 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 1430896, 2024-03-04T09:09:10.101 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 124160, 2024-03-04T09:09:10.102 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 2, 2024-03-04T09:09:10.102 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 1, 2024-03-04T09:09:10.102 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.037388755, 2024-03-04T09:09:10.102 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.012394675, 2024-03-04T09:09:10.102 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.029473113, 2024-03-04T09:09:10.102 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.003830888, 2024-03-04T09:09:10.102 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:10.102 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 206, 2024-03-04T09:09:10.103 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.007728774, 2024-03-04T09:09:10.103 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000037518 2024-03-04T09:09:10.103 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:10.103 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:10.103 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-04T09:09:10.103 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-04T09:09:10.103 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-04T09:09:10.103 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:10.103 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 1430896, 2024-03-04T09:09:10.104 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 124160 2024-03-04T09:09:10.104 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:10.104 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-03-04T09:09:10.104 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 0, 2024-03-04T09:09:10.104 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-03-04T09:09:10.104 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:10.104 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-03-04T09:09:10.104 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-03-04T09:09:10.105 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-03-04T09:09:10.105 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-03-04T09:09:10.105 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-03-04T09:09:10.107 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-03-04T09:09:10.108 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-03-04T09:09:10.108 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-04T09:09:10.108 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-03-04T09:09:10.108 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-04T09:09:10.108 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-03-04T09:09:10.108 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-03-04T09:09:10.108 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-03-04T09:09:10.109 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-04T09:09:10.109 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-03-04T09:09:10.109 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-03-04T09:09:10.109 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-03-04T09:09:10.109 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-03-04T09:09:10.109 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-03-04T09:09:10.109 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-04T09:09:10.109 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-03-04T09:09:10.110 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-04T09:09:10.110 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-03-04T09:09:10.110 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-03-04T09:09:10.110 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-03-04T09:09:10.110 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-03-04T09:09:10.110 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-03-04T09:09:10.110 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-04T09:09:10.110 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-03-04T09:09:10.111 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-03-04T09:09:10.111 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-03-04T09:09:10.111 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-03-04T09:09:10.111 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-03-04T09:09:10.111 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-03-04T09:09:10.111 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-03-04T09:09:10.112 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-03-04T09:09:10.112 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-03-04T09:09:10.112 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 149047, 2024-03-04T09:09:10.112 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 31, 2024-03-04T09:09:10.112 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-03-04T09:09:10.112 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-03-04T09:09:10.112 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-03-04T09:09:10.112 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-03-04T09:09:10.113 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-03-04T09:09:10.113 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-03-04T09:09:10.113 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-03-04T09:09:10.113 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-03-04T09:09:10.113 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 0, 2024-03-04T09:09:10.113 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 0, 2024-03-04T09:09:10.113 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-03-04T09:09:10.113 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-03-04T09:09:10.114 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-03-04T09:09:10.114 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-03-04T09:09:10.114 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 928, 2024-03-04T09:09:10.114 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 2, 2024-03-04T09:09:10.114 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-03-04T09:09:10.114 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-03-04T09:09:10.114 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-03-04T09:09:10.114 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-03-04T09:09:10.114 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:10.115 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-04T09:09:10.115 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-03-04T09:09:10.115 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-03-04T09:09:10.115 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-03-04T09:09:10.115 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 219, 2024-03-04T09:09:10.115 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 1400286, 2024-03-04T09:09:10.115 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-03-04T09:09:10.115 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 219, 2024-03-04T09:09:10.116 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-03-04T09:09:10.116 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-03-04T09:09:10.116 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 219, 2024-03-04T09:09:10.116 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 1400286, 2024-03-04T09:09:10.116 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-03-04T09:09:10.116 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-04T09:09:10.116 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-04T09:09:10.117 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-04T09:09:10.117 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:10.117 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:10.117 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-03-04T09:09:11.055 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-03-04T09:09:11.055 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-03-04T09:09:11.055 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-03-04T09:09:11.055 DEBUG:tasks.check_counter:Getting stats from j 2024-03-04T09:09:11.056 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-04T09:09:11.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:11 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:11.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:11 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:11.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:11 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:11.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:11 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:11.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:11 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:11.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:11 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:11.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:11 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:11.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:11 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:11.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:11 smithi071 ceph-mon[32789]: Adjusting osd_memory_target on smithi158 to 174.1M 2024-03-04T09:09:11.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:11 smithi071 ceph-mon[32789]: Unable to set osd_memory_target on smithi158 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-03-04T09:09:11.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:11 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:11.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:11 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:11.424 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:11 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:11.424 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:11 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:11.424 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:11 smithi071 ceph-mon[32789]: Adjusting osd_memory_target on smithi150 to 174.1M 2024-03-04T09:09:11.424 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:11 smithi071 ceph-mon[32789]: Unable to set osd_memory_target on smithi150 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-03-04T09:09:11.424 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:11 smithi071 ceph-mon[32789]: pgmap v318: 129 pgs: 129 active+clean; 422 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 28 KiB/s wr, 5 op/s 2024-03-04T09:09:11.486 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:09:11.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:11 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:11.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:11 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:11.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:11 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:11.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:11 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:11.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:11 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:11.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:11 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:11.532 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:11 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:11.532 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:11 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:11.532 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:11 smithi150 ceph-mon[36807]: Adjusting osd_memory_target on smithi158 to 174.1M 2024-03-04T09:09:11.532 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:11 smithi150 ceph-mon[36807]: Unable to set osd_memory_target on smithi158 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-03-04T09:09:11.533 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:11 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:11.533 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:11 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:11.533 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:11 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:11.533 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:11 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:11.534 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:11 smithi150 ceph-mon[36807]: Adjusting osd_memory_target on smithi150 to 174.1M 2024-03-04T09:09:11.534 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:11 smithi150 ceph-mon[36807]: Unable to set osd_memory_target on smithi150 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-03-04T09:09:11.534 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:11 smithi150 ceph-mon[36807]: pgmap v318: 129 pgs: 129 active+clean; 422 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 28 KiB/s wr, 5 op/s 2024-03-04T09:09:11.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:11 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:11.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:11 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:11.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:11 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:11.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:11 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:11.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:11 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:11.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:11 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:11.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:11 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:11.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:11 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:11.554 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:11 smithi158 ceph-mon[37241]: Adjusting osd_memory_target on smithi158 to 174.1M 2024-03-04T09:09:11.554 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:11 smithi158 ceph-mon[37241]: Unable to set osd_memory_target on smithi158 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-03-04T09:09:11.554 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:11 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:11.554 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:11 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:11.555 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:11 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:11.555 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:11 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:11.555 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:11 smithi158 ceph-mon[37241]: Adjusting osd_memory_target on smithi150 to 174.1M 2024-03-04T09:09:11.555 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:11 smithi158 ceph-mon[37241]: Unable to set osd_memory_target on smithi150 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-03-04T09:09:11.555 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:11 smithi158 ceph-mon[37241]: pgmap v318: 129 pgs: 129 active+clean; 422 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 28 KiB/s wr, 5 op/s 2024-03-04T09:09:13.254 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-03-04T09:09:13.255 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-03-04T09:09:13.255 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 0, 2024-03-04T09:09:13.255 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 2, 2024-03-04T09:09:13.255 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 0, 2024-03-04T09:09:13.255 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 0, 2024-03-04T09:09:13.255 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 4, 2024-03-04T09:09:13.256 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 0, 2024-03-04T09:09:13.256 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.001063805, 2024-03-04T09:09:13.256 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.000000000, 2024-03-04T09:09:13.256 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.000240330, 2024-03-04T09:09:13.256 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:09:13.256 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:13.256 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-04T09:09:13.256 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-04T09:09:13.257 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-04T09:09:13.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:13.257 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:13.257 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-04T09:09:13.257 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-04T09:09:13.257 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-04T09:09:13.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:13.257 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-04T09:09:13.258 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 0 2024-03-04T09:09:13.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:13.258 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-03-04T09:09:13.258 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 2, 2024-03-04T09:09:13.258 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 163, 2024-03-04T09:09:13.258 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 320, 2024-03-04T09:09:13.258 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 183616, 2024-03-04T09:09:13.258 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 5, 2024-03-04T09:09:13.259 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 3, 2024-03-04T09:09:13.259 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.011563536, 2024-03-04T09:09:13.259 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.007807511, 2024-03-04T09:09:13.259 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.001767309, 2024-03-04T09:09:13.259 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:09:13.259 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:13.259 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 160, 2024-03-04T09:09:13.259 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.013498132, 2024-03-04T09:09:13.260 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000084363 2024-03-04T09:09:13.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:13.260 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:13.260 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-04T09:09:13.260 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-04T09:09:13.260 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-04T09:09:13.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:13.260 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-04T09:09:13.261 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 183616 2024-03-04T09:09:13.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:13.261 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-03-04T09:09:13.261 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 227, 2024-03-04T09:09:13.261 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 215, 2024-03-04T09:09:13.261 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 1435520, 2024-03-04T09:09:13.261 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 128352, 2024-03-04T09:09:13.261 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 2, 2024-03-04T09:09:13.261 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 1, 2024-03-04T09:09:13.262 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.038297803, 2024-03-04T09:09:13.262 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.012486435, 2024-03-04T09:09:13.262 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.029881201, 2024-03-04T09:09:13.262 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.003929395, 2024-03-04T09:09:13.262 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:13.262 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 215, 2024-03-04T09:09:13.262 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.010149726, 2024-03-04T09:09:13.262 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000047208 2024-03-04T09:09:13.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:13.263 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:13.263 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-04T09:09:13.263 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-04T09:09:13.263 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-04T09:09:13.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:13.263 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 1435520, 2024-03-04T09:09:13.263 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 128352 2024-03-04T09:09:13.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:13.264 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-03-04T09:09:13.264 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 0, 2024-03-04T09:09:13.264 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-03-04T09:09:13.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:13.264 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-03-04T09:09:13.264 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-03-04T09:09:13.264 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-03-04T09:09:13.265 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-03-04T09:09:13.265 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-03-04T09:09:13.265 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-03-04T09:09:13.265 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-03-04T09:09:13.265 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-04T09:09:13.265 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-03-04T09:09:13.265 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-04T09:09:13.265 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-03-04T09:09:13.266 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-03-04T09:09:13.266 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-03-04T09:09:13.266 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-04T09:09:13.266 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-03-04T09:09:13.266 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-03-04T09:09:13.266 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-03-04T09:09:13.266 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-03-04T09:09:13.266 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-03-04T09:09:13.267 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-04T09:09:13.267 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-03-04T09:09:13.267 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-04T09:09:13.267 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-03-04T09:09:13.267 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-03-04T09:09:13.267 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-03-04T09:09:13.267 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-03-04T09:09:13.267 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-03-04T09:09:13.267 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-04T09:09:13.268 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-03-04T09:09:13.268 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-03-04T09:09:13.268 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-03-04T09:09:13.268 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-03-04T09:09:13.268 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-03-04T09:09:13.268 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-03-04T09:09:13.268 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-03-04T09:09:13.268 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-03-04T09:09:13.269 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-03-04T09:09:13.269 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 148977, 2024-03-04T09:09:13.269 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 30, 2024-03-04T09:09:13.269 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-03-04T09:09:13.269 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-03-04T09:09:13.269 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-03-04T09:09:13.269 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-03-04T09:09:13.269 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-03-04T09:09:13.270 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-03-04T09:09:13.270 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-03-04T09:09:13.270 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-03-04T09:09:13.270 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 0, 2024-03-04T09:09:13.270 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 0, 2024-03-04T09:09:13.270 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-03-04T09:09:13.270 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-03-04T09:09:13.270 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-03-04T09:09:13.270 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-03-04T09:09:13.271 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 928, 2024-03-04T09:09:13.271 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 2, 2024-03-04T09:09:13.271 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-03-04T09:09:13.271 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-03-04T09:09:13.271 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-03-04T09:09:13.271 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-03-04T09:09:13.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-03-04T09:09:13.271 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-04T09:09:13.272 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-03-04T09:09:13.272 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-03-04T09:09:13.272 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-03-04T09:09:13.272 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 225, 2024-03-04T09:09:13.272 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 1401066, 2024-03-04T09:09:13.272 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-03-04T09:09:13.272 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 225, 2024-03-04T09:09:13.272 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-03-04T09:09:13.273 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-03-04T09:09:13.273 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 225, 2024-03-04T09:09:13.273 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 1401066, 2024-03-04T09:09:13.273 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-03-04T09:09:13.273 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-03-04T09:09:13.273 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-03-04T09:09:13.273 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-03-04T09:09:13.273 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:13.274 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-03-04T09:09:13.274 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-03-04T09:09:13.925 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-03-04T09:09:13.925 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-03-04T09:09:13.925 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-03-04T09:09:13.925 DEBUG:tasks.check_counter:Getting stats from b 2024-03-04T09:09:13.925 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-04T09:09:14.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:14 smithi071 ceph-mon[32789]: pgmap v319: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 28 KiB/s wr, 5 op/s 2024-03-04T09:09:14.174 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T09:09:14.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:14 smithi150 ceph-mon[36807]: pgmap v319: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 28 KiB/s wr, 5 op/s 2024-03-04T09:09:14.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:14 smithi158 ceph-mon[37241]: pgmap v319: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 28 KiB/s wr, 5 op/s 2024-03-04T09:09:15.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:15 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:15.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:15 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:15.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:15 smithi071 ceph-mon[32789]: osdmap e104: 12 total, 12 up, 12 in 2024-03-04T09:09:15.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:15 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:15.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:15 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:15.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:15 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:15.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:15 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:15.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:15 smithi071 ceph-mon[32789]: Adjusting osd_memory_target on smithi071 to 176.2M 2024-03-04T09:09:15.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:15 smithi071 ceph-mon[32789]: Unable to set osd_memory_target on smithi071 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-03-04T09:09:15.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:15 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:09:15.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:15 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:09:15.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:15 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:15.424 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:15 smithi071 ceph-mon[32789]: pgmap v321: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 807 B/s rd, 230 B/s wr, 2 op/s 2024-03-04T09:09:15.440 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:15 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:15.441 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:15 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:15.441 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:15 smithi150 ceph-mon[36807]: osdmap e104: 12 total, 12 up, 12 in 2024-03-04T09:09:15.441 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:15 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:15.441 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:15 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:15.442 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:15 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:15.442 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:15 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:15.442 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:15 smithi150 ceph-mon[36807]: Adjusting osd_memory_target on smithi071 to 176.2M 2024-03-04T09:09:15.442 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:15 smithi150 ceph-mon[36807]: Unable to set osd_memory_target on smithi071 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-03-04T09:09:15.442 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:15 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:09:15.443 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:15 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:09:15.443 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:15 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:15.443 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:15 smithi150 ceph-mon[36807]: pgmap v321: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 807 B/s rd, 230 B/s wr, 2 op/s 2024-03-04T09:09:15.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:15 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:16.082 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-03-04T09:09:16.082 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-03-04T09:09:16.082 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 47415, 2024-03-04T09:09:16.082 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 46832, 2024-03-04T09:09:16.083 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 11998210, 2024-03-04T09:09:16.083 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 266619967, 2024-03-04T09:09:16.083 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 12, 2024-03-04T09:09:16.083 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 5, 2024-03-04T09:09:16.083 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 4.202151265, 2024-03-04T09:09:16.083 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 1.779251363, 2024-03-04T09:09:16.083 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.799553556, 2024-03-04T09:09:16.083 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 1.319781018, 2024-03-04T09:09:16.084 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:16.084 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 46830, 2024-03-04T09:09:16.084 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 8.368740635, 2024-03-04T09:09:16.084 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000178704 2024-03-04T09:09:16.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.084 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:16.084 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 909, 2024-03-04T09:09:16.084 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000148064, 2024-03-04T09:09:16.084 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000162 2024-03-04T09:09:16.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.085 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 327008, 2024-03-04T09:09:16.085 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 101344 2024-03-04T09:09:16.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.085 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-03-04T09:09:16.085 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 45262, 2024-03-04T09:09:16.085 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 45166, 2024-03-04T09:09:16.085 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 16057850, 2024-03-04T09:09:16.086 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 256451805, 2024-03-04T09:09:16.086 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 7, 2024-03-04T09:09:16.086 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 6, 2024-03-04T09:09:16.086 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 3.806811186, 2024-03-04T09:09:16.086 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 1.468880752, 2024-03-04T09:09:16.086 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.765336252, 2024-03-04T09:09:16.086 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 1.281288387, 2024-03-04T09:09:16.086 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:16.087 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 45165, 2024-03-04T09:09:16.087 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 7.575551981, 2024-03-04T09:09:16.087 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000167730 2024-03-04T09:09:16.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.087 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:16.087 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 106, 2024-03-04T09:09:16.087 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000013743, 2024-03-04T09:09:16.087 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000129 2024-03-04T09:09:16.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.088 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1512688, 2024-03-04T09:09:16.088 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 138288 2024-03-04T09:09:16.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.088 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-03-04T09:09:16.088 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 330361, 2024-03-04T09:09:16.088 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 504977, 2024-03-04T09:09:16.088 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 94880730, 2024-03-04T09:09:16.088 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 430828059, 2024-03-04T09:09:16.089 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 16, 2024-03-04T09:09:16.089 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 7, 2024-03-04T09:09:16.089 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 14.829900348, 2024-03-04T09:09:16.089 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 7.472281828, 2024-03-04T09:09:16.089 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 8.107910160, 2024-03-04T09:09:16.089 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.962841856, 2024-03-04T09:09:16.089 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:16.089 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 504975, 2024-03-04T09:09:16.089 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 17.427903932, 2024-03-04T09:09:16.090 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000034512 2024-03-04T09:09:16.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.090 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:16.090 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 443634, 2024-03-04T09:09:16.090 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.079185550, 2024-03-04T09:09:16.090 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000178 2024-03-04T09:09:16.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.090 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 103104, 2024-03-04T09:09:16.091 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 482448 2024-03-04T09:09:16.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.091 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-03-04T09:09:16.091 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2024-03-04T09:09:16.091 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-03-04T09:09:16.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.091 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2024-03-04T09:09:16.091 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-03-04T09:09:16.091 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-03-04T09:09:16.092 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 100245, 2024-03-04T09:09:16.092 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 75.547764046, 2024-03-04T09:09:16.092 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000753631 2024-03-04T09:09:16.092 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-03-04T09:09:16.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.092 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2024-03-04T09:09:16.092 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-03-04T09:09:16.092 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-03-04T09:09:16.093 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 32593, 2024-03-04T09:09:16.093 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 8.820834394, 2024-03-04T09:09:16.093 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000270635 2024-03-04T09:09:16.093 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-03-04T09:09:16.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.093 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2024-03-04T09:09:16.093 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 192359, 2024-03-04T09:09:16.093 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 192359, 2024-03-04T09:09:16.094 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2024-03-04T09:09:16.094 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 192359, 2024-03-04T09:09:16.094 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 300.993509426, 2024-03-04T09:09:16.094 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001564748 2024-03-04T09:09:16.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.094 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2024-03-04T09:09:16.094 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 0, 2024-03-04T09:09:16.095 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 0, 2024-03-04T09:09:16.095 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2024-03-04T09:09:16.095 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 153, 2024-03-04T09:09:16.095 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 12, 2024-03-04T09:09:16.095 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 21, 2024-03-04T09:09:16.095 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 2056, 2024-03-04T09:09:16.095 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 1140, 2024-03-04T09:09:16.095 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 916, 2024-03-04T09:09:16.096 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 0, 2024-03-04T09:09:16.096 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 428, 2024-03-04T09:09:16.096 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 0, 2024-03-04T09:09:16.096 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 50, 2024-03-04T09:09:16.096 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 51, 2024-03-04T09:09:16.096 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 2, 2024-03-04T09:09:16.096 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 261115, 2024-03-04T09:09:16.096 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 162771, 2024-03-04T09:09:16.096 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 0, 2024-03-04T09:09:16.097 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 0, 2024-03-04T09:09:16.097 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2024-03-04T09:09:16.097 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2024-03-04T09:09:16.097 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 2156, 2024-03-04T09:09:16.097 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 0, 2024-03-04T09:09:16.097 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 0, 2024-03-04T09:09:16.097 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 0, 2024-03-04T09:09:16.097 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 0, 2024-03-04T09:09:16.098 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 0, 2024-03-04T09:09:16.098 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 0, 2024-03-04T09:09:16.098 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2024-03-04T09:09:16.098 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 0, 2024-03-04T09:09:16.098 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2024-03-04T09:09:16.098 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 68, 2024-03-04T09:09:16.098 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 21196213, 2024-03-04T09:09:16.098 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 7, 2024-03-04T09:09:16.098 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 0, 2024-03-04T09:09:16.099 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2024-03-04T09:09:16.099 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2024-03-04T09:09:16.099 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2024-03-04T09:09:16.099 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 0, 2024-03-04T09:09:16.099 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 0, 2024-03-04T09:09:16.099 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 0, 2024-03-04T09:09:16.099 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 0, 2024-03-04T09:09:16.099 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 0, 2024-03-04T09:09:16.100 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 56381, 2024-03-04T09:09:16.100 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 140888, 2024-03-04T09:09:16.100 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 0, 2024-03-04T09:09:16.100 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-04T09:09:16.100 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-04T09:09:16.100 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 97672, 2024-03-04T09:09:16.100 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 23998, 2024-03-04T09:09:16.100 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 12089, 2024-03-04T09:09:16.101 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 124203 2024-03-04T09:09:16.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.101 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2024-03-04T09:09:16.101 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 0, 2024-03-04T09:09:16.101 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 0, 2024-03-04T09:09:16.101 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 0, 2024-03-04T09:09:16.101 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 0, 2024-03-04T09:09:16.101 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 0, 2024-03-04T09:09:16.102 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 0, 2024-03-04T09:09:16.102 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2024-03-04T09:09:16.102 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2024-03-04T09:09:16.102 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 30592, 2024-03-04T09:09:16.102 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 30592, 2024-03-04T09:09:16.102 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2024-03-04T09:09:16.102 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2024-03-04T09:09:16.102 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2024-03-04T09:09:16.102 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2024-03-04T09:09:16.103 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2024-03-04T09:09:16.103 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2024-03-04T09:09:16.103 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2024-03-04T09:09:16.103 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 0, 2024-03-04T09:09:16.103 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 0, 2024-03-04T09:09:16.103 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2024-03-04T09:09:16.103 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 33, 2024-03-04T09:09:16.103 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2024-03-04T09:09:16.104 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0 2024-03-04T09:09:16.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.104 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2024-03-04T09:09:16.104 INFO:teuthology.orchestra.run.smithi150.stdout: "evlrg": 0, 2024-03-04T09:09:16.104 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 138732, 2024-03-04T09:09:16.104 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 49161, 2024-03-04T09:09:16.104 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 49161, 2024-03-04T09:09:16.104 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 89571, 2024-03-04T09:09:16.105 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2024-03-04T09:09:16.105 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 12290, 2024-03-04T09:09:16.105 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 199, 2024-03-04T09:09:16.105 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 71, 2024-03-04T09:09:16.105 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 71, 2024-03-04T09:09:16.105 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 128, 2024-03-04T09:09:16.105 INFO:teuthology.orchestra.run.smithi150.stdout: "segmjr": 8, 2024-03-04T09:09:16.105 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2024-03-04T09:09:16.106 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 20, 2024-03-04T09:09:16.106 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 143482111, 2024-03-04T09:09:16.106 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 550044378, 2024-03-04T09:09:16.106 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 0, 2024-03-04T09:09:16.106 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2024-03-04T09:09:16.106 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 50623, 2024-03-04T09:09:16.106 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 169.892926962, 2024-03-04T09:09:16.106 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.003356042 2024-03-04T09:09:16.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.107 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 0 2024-03-04T09:09:16.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.107 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2024-03-04T09:09:16.107 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 483, 2024-03-04T09:09:16.107 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 31075, 2024-03-04T09:09:16.107 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 30592, 2024-03-04T09:09:16.107 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 47, 2024-03-04T09:09:16.107 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 149, 2024-03-04T09:09:16.107 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 102, 2024-03-04T09:09:16.108 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 2056, 2024-03-04T09:09:16.108 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 62688, 2024-03-04T09:09:16.108 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 60632, 2024-03-04T09:09:16.108 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 51, 2024-03-04T09:09:16.108 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 32020, 2024-03-04T09:09:16.108 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 31969, 2024-03-04T09:09:16.108 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 76272, 2024-03-04T09:09:16.108 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 231804 2024-03-04T09:09:16.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.109 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2024-03-04T09:09:16.109 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 253038, 2024-03-04T09:09:16.109 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 0, 2024-03-04T09:09:16.109 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 192359, 2024-03-04T09:09:16.109 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 4262, 2024-03-04T09:09:16.109 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 0, 2024-03-04T09:09:16.109 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2024-03-04T09:09:16.109 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 30660, 2024-03-04T09:09:16.110 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 32.045355056, 2024-03-04T09:09:16.110 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001045184 2024-03-04T09:09:16.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.110 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2024-03-04T09:09:16.110 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 358, 2024-03-04T09:09:16.110 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.479271474, 2024-03-04T09:09:16.110 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001338747 2024-03-04T09:09:16.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.111 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2024-03-04T09:09:16.111 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:16.111 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:16.111 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:16.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.111 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2024-03-04T09:09:16.111 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:16.111 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:16.112 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:16.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.112 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2024-03-04T09:09:16.112 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 59894, 2024-03-04T09:09:16.112 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 37.203834207, 2024-03-04T09:09:16.112 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000621161 2024-03-04T09:09:16.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.112 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2024-03-04T09:09:16.113 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:16.113 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:16.113 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:16.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.113 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2024-03-04T09:09:16.113 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2, 2024-03-04T09:09:16.113 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000391973, 2024-03-04T09:09:16.113 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000195986 2024-03-04T09:09:16.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.114 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2024-03-04T09:09:16.114 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:16.114 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:16.114 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:16.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.114 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2024-03-04T09:09:16.114 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:16.114 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:16.115 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:16.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.115 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2024-03-04T09:09:16.115 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:16.115 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:16.115 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:16.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.115 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2024-03-04T09:09:16.116 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 11, 2024-03-04T09:09:16.116 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.002963704, 2024-03-04T09:09:16.116 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000269427 2024-03-04T09:09:16.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.117 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2024-03-04T09:09:16.117 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 20, 2024-03-04T09:09:16.117 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.018829670, 2024-03-04T09:09:16.117 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000941483 2024-03-04T09:09:16.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.117 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2024-03-04T09:09:16.117 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:16.117 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:16.118 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:16.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.118 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2024-03-04T09:09:16.118 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 11, 2024-03-04T09:09:16.118 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.197787671, 2024-03-04T09:09:16.118 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.017980697 2024-03-04T09:09:16.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.118 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2024-03-04T09:09:16.119 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:16.119 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:16.119 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:16.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.119 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2024-03-04T09:09:16.119 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 63399, 2024-03-04T09:09:16.119 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 177.690892274, 2024-03-04T09:09:16.119 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.002802739 2024-03-04T09:09:16.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.120 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2024-03-04T09:09:16.120 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 6410, 2024-03-04T09:09:16.120 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 8.527734770, 2024-03-04T09:09:16.120 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001330379 2024-03-04T09:09:16.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.120 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2024-03-04T09:09:16.120 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:16.120 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:16.121 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:16.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.121 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2024-03-04T09:09:16.121 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:16.121 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:16.121 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:16.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.121 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2024-03-04T09:09:16.122 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:16.122 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:16.122 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:16.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.122 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2024-03-04T09:09:16.122 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 4, 2024-03-04T09:09:16.122 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.023927877, 2024-03-04T09:09:16.123 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.005981969 2024-03-04T09:09:16.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.123 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2024-03-04T09:09:16.123 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:16.123 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:16.123 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:16.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.123 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2024-03-04T09:09:16.123 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 6, 2024-03-04T09:09:16.124 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.034005619, 2024-03-04T09:09:16.124 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.005667603 2024-03-04T09:09:16.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.124 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2024-03-04T09:09:16.124 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:16.124 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:16.124 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:16.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.125 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2024-03-04T09:09:16.125 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 984, 2024-03-04T09:09:16.125 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.500253637, 2024-03-04T09:09:16.125 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000508387 2024-03-04T09:09:16.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.125 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2024-03-04T09:09:16.125 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:16.125 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:16.126 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:16.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.126 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2024-03-04T09:09:16.126 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 8, 2024-03-04T09:09:16.126 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.018823594, 2024-03-04T09:09:16.126 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.002352949 2024-03-04T09:09:16.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.126 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2024-03-04T09:09:16.127 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:16.127 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:16.127 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:16.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.127 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2024-03-04T09:09:16.127 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 30592, 2024-03-04T09:09:16.127 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 44.249437900, 2024-03-04T09:09:16.127 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001446438 2024-03-04T09:09:16.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.128 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2024-03-04T09:09:16.128 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2024-03-04T09:09:16.128 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2024-03-04T09:09:16.128 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:16.128 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:16.128 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:16.128 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-03-04T09:09:16.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.129 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2024-03-04T09:09:16.129 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 2, 2024-03-04T09:09:16.129 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 6, 2024-03-04T09:09:16.129 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 4, 2024-03-04T09:09:16.129 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 2, 2024-03-04T09:09:16.129 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2024-03-04T09:09:16.129 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 10733, 2024-03-04T09:09:16.130 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 5366, 2024-03-04T09:09:16.130 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 622, 2024-03-04T09:09:16.130 INFO:teuthology.orchestra.run.smithi150.stdout: "mdthresh_evicted": 0 2024-03-04T09:09:16.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.130 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-03-04T09:09:16.131 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-03-04T09:09:16.131 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-03-04T09:09:16.131 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-03-04T09:09:16.131 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-03-04T09:09:16.131 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-03-04T09:09:16.131 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-03-04T09:09:16.131 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-04T09:09:16.131 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-03-04T09:09:16.131 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-04T09:09:16.132 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-03-04T09:09:16.132 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-03-04T09:09:16.132 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-03-04T09:09:16.132 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-04T09:09:16.132 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-03-04T09:09:16.132 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-03-04T09:09:16.132 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-03-04T09:09:16.132 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-03-04T09:09:16.133 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-03-04T09:09:16.133 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-04T09:09:16.133 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-03-04T09:09:16.133 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-04T09:09:16.133 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-03-04T09:09:16.133 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-03-04T09:09:16.133 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-03-04T09:09:16.133 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-03-04T09:09:16.133 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-03-04T09:09:16.134 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-04T09:09:16.134 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-03-04T09:09:16.134 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-03-04T09:09:16.134 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-03-04T09:09:16.134 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-03-04T09:09:16.134 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-03-04T09:09:16.134 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-03-04T09:09:16.134 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-03-04T09:09:16.135 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-03-04T09:09:16.135 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-03-04T09:09:16.135 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 319434, 2024-03-04T09:09:16.135 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 345, 2024-03-04T09:09:16.135 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 528, 2024-03-04T09:09:16.135 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 6, 2024-03-04T09:09:16.135 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-03-04T09:09:16.135 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-03-04T09:09:16.135 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-03-04T09:09:16.136 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-03-04T09:09:16.136 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-03-04T09:09:16.136 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-03-04T09:09:16.136 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 9416, 2024-03-04T09:09:16.136 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 180, 2024-03-04T09:09:16.136 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-03-04T09:09:16.136 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-03-04T09:09:16.136 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-03-04T09:09:16.137 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-03-04T09:09:16.137 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 2881867, 2024-03-04T09:09:16.137 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 30420, 2024-03-04T09:09:16.137 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-03-04T09:09:16.137 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-03-04T09:09:16.137 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-03-04T09:09:16.137 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-03-04T09:09:16.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.137 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2024-03-04T09:09:16.138 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2024-03-04T09:09:16.138 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2024-03-04T09:09:16.138 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 115123, 2024-03-04T09:09:16.138 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 553430769, 2024-03-04T09:09:16.138 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 31, 2024-03-04T09:09:16.138 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 115123, 2024-03-04T09:09:16.138 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2024-03-04T09:09:16.138 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 115123, 2024-03-04T09:09:16.139 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 1862.466834950, 2024-03-04T09:09:16.139 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.016178060 2024-03-04T09:09:16.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.139 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2024-03-04T09:09:16.139 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2024-03-04T09:09:16.139 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 115123, 2024-03-04T09:09:16.139 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 116112 2024-03-04T09:09:16.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.139 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 115123, 2024-03-04T09:09:16.140 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 791, 2024-03-04T09:09:16.140 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 114332, 2024-03-04T09:09:16.140 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2024-03-04T09:09:16.140 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2024-03-04T09:09:16.140 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 4, 2024-03-04T09:09:16.140 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 51, 2024-03-04T09:09:16.140 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 791, 2024-03-04T09:09:16.140 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 51414, 2024-03-04T09:09:16.141 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 346, 2024-03-04T09:09:16.141 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2024-03-04T09:09:16.141 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2024-03-04T09:09:16.141 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2024-03-04T09:09:16.141 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 12, 2024-03-04T09:09:16.141 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 61949, 2024-03-04T09:09:16.141 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2024-03-04T09:09:16.141 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2024-03-04T09:09:16.141 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2024-03-04T09:09:16.142 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 0, 2024-03-04T09:09:16.142 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 98, 2024-03-04T09:09:16.142 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2024-03-04T09:09:16.142 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2024-03-04T09:09:16.142 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2024-03-04T09:09:16.142 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2024-03-04T09:09:16.142 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2024-03-04T09:09:16.142 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2024-03-04T09:09:16.143 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2024-03-04T09:09:16.143 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2024-03-04T09:09:16.143 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2024-03-04T09:09:16.143 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 0, 2024-03-04T09:09:16.143 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2024-03-04T09:09:16.143 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2024-03-04T09:09:16.143 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2024-03-04T09:09:16.143 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2024-03-04T09:09:16.144 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2024-03-04T09:09:16.144 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2024-03-04T09:09:16.144 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2024-03-04T09:09:16.144 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2024-03-04T09:09:16.144 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2024-03-04T09:09:16.144 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2024-03-04T09:09:16.144 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2024-03-04T09:09:16.144 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2024-03-04T09:09:16.145 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2024-03-04T09:09:16.145 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2024-03-04T09:09:16.145 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2024-03-04T09:09:16.145 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2024-03-04T09:09:16.145 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 104, 2024-03-04T09:09:16.145 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2024-03-04T09:09:16.145 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 18, 2024-03-04T09:09:16.145 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 12, 2024-03-04T09:09:16.146 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 12, 2024-03-04T09:09:16.146 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 0, 2024-03-04T09:09:16.146 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2024-03-04T09:09:16.146 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 1086, 2024-03-04T09:09:16.146 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 0, 2024-03-04T09:09:16.146 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 361 2024-03-04T09:09:16.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.146 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2024-03-04T09:09:16.147 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 1, 2024-03-04T09:09:16.147 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2024-03-04T09:09:16.147 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2024-03-04T09:09:16.147 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2024-03-04T09:09:16.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.147 INFO:teuthology.orchestra.run.smithi150.stdout: "osd-slow-ops": { 2024-03-04T09:09:16.147 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 0 2024-03-04T09:09:16.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.147 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2024-03-04T09:09:16.148 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2024-03-04T09:09:16.148 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 48, 2024-03-04T09:09:16.148 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2024-03-04T09:09:16.148 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 24, 2024-03-04T09:09:16.148 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed_ops": 61184, 2024-03-04T09:09:16.148 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 30592, 2024-03-04T09:09:16.148 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2024-03-04T09:09:16.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.149 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-04T09:09:16.149 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-03-04T09:09:16.149 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-03-04T09:09:16.149 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-03-04T09:09:16.149 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 423034, 2024-03-04T09:09:16.149 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 89701738, 2024-03-04T09:09:16.149 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-03-04T09:09:16.150 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 423034, 2024-03-04T09:09:16.150 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-03-04T09:09:16.150 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-03-04T09:09:16.150 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 423034, 2024-03-04T09:09:16.150 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 89701738, 2024-03-04T09:09:16.150 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-03-04T09:09:16.150 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:16.150 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:16.150 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:16.151 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-03-04T09:09:16.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.151 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2024-03-04T09:09:16.151 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-03-04T09:09:16.151 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-03-04T09:09:16.151 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-03-04T09:09:16.151 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-03-04T09:09:16.151 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-03-04T09:09:16.152 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-03-04T09:09:16.152 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-03-04T09:09:16.152 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 115123, 2024-03-04T09:09:16.152 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 556767321, 2024-03-04T09:09:16.152 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 115123, 2024-03-04T09:09:16.152 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 556767321, 2024-03-04T09:09:16.152 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-03-04T09:09:16.152 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:16.153 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:16.153 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:16.153 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-03-04T09:09:16.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.153 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2024-03-04T09:09:16.153 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-03-04T09:09:16.153 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2024-03-04T09:09:16.153 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-03-04T09:09:16.153 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-03-04T09:09:16.154 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-03-04T09:09:16.154 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-03-04T09:09:16.154 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-03-04T09:09:16.154 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 115123, 2024-03-04T09:09:16.154 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 115123, 2024-03-04T09:09:16.154 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 115123, 2024-03-04T09:09:16.154 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 115123, 2024-03-04T09:09:16.154 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-03-04T09:09:16.154 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:16.155 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:16.155 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:16.155 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-03-04T09:09:16.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.155 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2024-03-04T09:09:16.155 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-03-04T09:09:16.155 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-03-04T09:09:16.155 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-03-04T09:09:16.156 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 30592, 2024-03-04T09:09:16.156 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 3336552, 2024-03-04T09:09:16.156 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-03-04T09:09:16.156 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 30592, 2024-03-04T09:09:16.156 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-03-04T09:09:16.156 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-03-04T09:09:16.156 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 791, 2024-03-04T09:09:16.156 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 3336552, 2024-03-04T09:09:16.156 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-03-04T09:09:16.157 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:16.157 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:16.157 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:16.157 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-03-04T09:09:16.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:16.157 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x56416fd4c6e0": { 2024-03-04T09:09:16.157 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-03-04T09:09:16.157 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-03-04T09:09:16.158 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-03-04T09:09:16.158 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 138732, 2024-03-04T09:09:16.158 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 545850074, 2024-03-04T09:09:16.158 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-03-04T09:09:16.158 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 138732, 2024-03-04T09:09:16.158 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-03-04T09:09:16.158 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-03-04T09:09:16.158 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 50623, 2024-03-04T09:09:16.158 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 545850074, 2024-03-04T09:09:16.159 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-03-04T09:09:16.159 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:16.159 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:16.159 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:16.159 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-03-04T09:09:16.159 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-03-04T09:09:16.159 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-03-04T09:09:16.842 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-03-04T09:09:16.843 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-03-04T09:09:16.843 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-03-04T09:09:16.843 DEBUG:tasks.check_counter:Getting stats from e 2024-03-04T09:09:16.844 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-04T09:09:17.109 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T09:09:18.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:18 smithi150 ceph-mon[36807]: pgmap v322: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-03-04T09:09:18.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:18 smithi071 ceph-mon[32789]: pgmap v322: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-03-04T09:09:18.802 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:18 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:18.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:18 smithi158 ceph-mon[37241]: osdmap e104: 12 total, 12 up, 12 in 2024-03-04T09:09:18.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:18 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:18.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:18 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:18.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:18 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:18.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:18 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:09:18.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:18 smithi158 ceph-mon[37241]: Adjusting osd_memory_target on smithi071 to 176.2M 2024-03-04T09:09:18.803 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:18 smithi158 ceph-mon[37241]: Unable to set osd_memory_target on smithi071 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-03-04T09:09:18.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:18 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:09:18.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:18 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:09:18.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:18 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' 2024-03-04T09:09:18.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:18 smithi158 ceph-mon[37241]: pgmap v321: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 807 B/s rd, 230 B/s wr, 2 op/s 2024-03-04T09:09:18.804 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:18 smithi158 ceph-mon[37241]: pgmap v322: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-03-04T09:09:18.850 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-03-04T09:09:18.850 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-03-04T09:09:18.850 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 0, 2024-03-04T09:09:18.850 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 2, 2024-03-04T09:09:18.850 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 0, 2024-03-04T09:09:18.851 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 0, 2024-03-04T09:09:18.851 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-03-04T09:09:18.851 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 0, 2024-03-04T09:09:18.851 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.001189280, 2024-03-04T09:09:18.851 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.000000000, 2024-03-04T09:09:18.851 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.000309856, 2024-03-04T09:09:18.851 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:09:18.851 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:18.851 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:18.852 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:18.852 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:18.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:18.852 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:18.852 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:18.852 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:18.852 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:18.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:18.853 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-04T09:09:18.853 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-03-04T09:09:18.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:18.853 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-03-04T09:09:18.853 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 223, 2024-03-04T09:09:18.853 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 213, 2024-03-04T09:09:18.853 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 1432528, 2024-03-04T09:09:18.853 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 127904, 2024-03-04T09:09:18.853 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 2, 2024-03-04T09:09:18.854 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-03-04T09:09:18.854 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.034333827, 2024-03-04T09:09:18.854 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.008762998, 2024-03-04T09:09:18.854 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.030908064, 2024-03-04T09:09:18.854 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.003794495, 2024-03-04T09:09:18.854 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:18.854 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 212, 2024-03-04T09:09:18.854 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.006200541, 2024-03-04T09:09:18.855 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000029247 2024-03-04T09:09:18.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:18.855 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:18.855 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:18.855 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:18.856 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:18.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:18.856 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1432528, 2024-03-04T09:09:18.856 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 127904 2024-03-04T09:09:18.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:18.857 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-03-04T09:09:18.857 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 6, 2024-03-04T09:09:18.857 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 165, 2024-03-04T09:09:18.857 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 3312, 2024-03-04T09:09:18.857 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 184064, 2024-03-04T09:09:18.858 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 5, 2024-03-04T09:09:18.858 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 3, 2024-03-04T09:09:18.858 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.008228664, 2024-03-04T09:09:18.858 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.004604990, 2024-03-04T09:09:18.858 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.002148649, 2024-03-04T09:09:18.858 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:09:18.859 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:18.859 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 163, 2024-03-04T09:09:18.859 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.013934642, 2024-03-04T09:09:18.859 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000085488 2024-03-04T09:09:18.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:18.860 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:18.860 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:18.860 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:18.860 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:18.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:18.860 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-03-04T09:09:18.860 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 184064 2024-03-04T09:09:18.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:18.861 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-03-04T09:09:18.861 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 0, 2024-03-04T09:09:18.861 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-03-04T09:09:18.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:18.861 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-03-04T09:09:18.861 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-03-04T09:09:18.861 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-03-04T09:09:18.861 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-03-04T09:09:18.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-03-04T09:09:18.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-03-04T09:09:18.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-03-04T09:09:18.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-04T09:09:18.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-03-04T09:09:18.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-04T09:09:18.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-03-04T09:09:18.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-03-04T09:09:18.862 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-03-04T09:09:18.863 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-04T09:09:18.863 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-03-04T09:09:18.863 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-03-04T09:09:18.863 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-03-04T09:09:18.863 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-03-04T09:09:18.863 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-03-04T09:09:18.863 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-04T09:09:18.863 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-03-04T09:09:18.863 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-04T09:09:18.864 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-03-04T09:09:18.864 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-03-04T09:09:18.864 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-03-04T09:09:18.864 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-03-04T09:09:18.864 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-03-04T09:09:18.864 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-04T09:09:18.864 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-03-04T09:09:18.864 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-03-04T09:09:18.865 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-03-04T09:09:18.865 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-03-04T09:09:18.865 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-03-04T09:09:18.865 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-03-04T09:09:18.865 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-03-04T09:09:18.865 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-03-04T09:09:18.865 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-03-04T09:09:18.865 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 149057, 2024-03-04T09:09:18.866 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 32, 2024-03-04T09:09:18.866 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-03-04T09:09:18.866 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-03-04T09:09:18.866 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-03-04T09:09:18.866 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-03-04T09:09:18.866 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-03-04T09:09:18.866 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-03-04T09:09:18.866 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-03-04T09:09:18.866 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-03-04T09:09:18.867 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 0, 2024-03-04T09:09:18.867 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 0, 2024-03-04T09:09:18.867 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-03-04T09:09:18.867 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-03-04T09:09:18.867 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-03-04T09:09:18.867 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-03-04T09:09:18.867 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 928, 2024-03-04T09:09:18.867 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 2, 2024-03-04T09:09:18.868 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-03-04T09:09:18.868 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-03-04T09:09:18.868 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-03-04T09:09:18.868 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-03-04T09:09:18.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:18.868 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-04T09:09:18.868 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-03-04T09:09:18.868 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-03-04T09:09:18.869 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-03-04T09:09:18.869 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 225, 2024-03-04T09:09:18.869 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 1401066, 2024-03-04T09:09:18.869 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-03-04T09:09:18.869 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 225, 2024-03-04T09:09:18.869 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-03-04T09:09:18.869 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-03-04T09:09:18.869 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 225, 2024-03-04T09:09:18.869 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 1401066, 2024-03-04T09:09:18.870 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-03-04T09:09:18.870 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:18.870 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:18.870 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:18.870 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-03-04T09:09:18.870 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-03-04T09:09:18.870 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-03-04T09:09:19.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:19 smithi071 ceph-mon[32789]: pgmap v323: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-03-04T09:09:19.712 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:19 smithi150 ceph-mon[36807]: pgmap v323: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-03-04T09:09:19.738 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-03-04T09:09:19.738 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-03-04T09:09:19.738 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-03-04T09:09:19.738 DEBUG:tasks.check_counter:Getting stats from h 2024-03-04T09:09:19.738 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-04T09:09:19.755 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:19 smithi158 ceph-mon[37241]: pgmap v323: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-03-04T09:09:19.989 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T09:09:21.776 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-03-04T09:09:21.777 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-03-04T09:09:21.777 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 4, 2024-03-04T09:09:21.777 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 4, 2024-03-04T09:09:21.777 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 2992, 2024-03-04T09:09:21.777 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 448, 2024-03-04T09:09:21.777 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-03-04T09:09:21.777 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 0, 2024-03-04T09:09:21.777 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.001597137, 2024-03-04T09:09:21.778 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.000039328, 2024-03-04T09:09:21.778 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.000689620, 2024-03-04T09:09:21.778 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:09:21.778 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:21.778 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3, 2024-03-04T09:09:21.778 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000748613, 2024-03-04T09:09:21.778 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000249537 2024-03-04T09:09:21.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:21.779 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:21.779 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:21.779 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:21.779 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:21.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:21.779 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-04T09:09:21.779 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 448 2024-03-04T09:09:21.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:21.780 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-03-04T09:09:21.780 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 221, 2024-03-04T09:09:21.780 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 211, 2024-03-04T09:09:21.780 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 1431984, 2024-03-04T09:09:21.780 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 126656, 2024-03-04T09:09:21.780 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 2, 2024-03-04T09:09:21.780 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-03-04T09:09:21.780 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.033229619, 2024-03-04T09:09:21.781 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.008662198, 2024-03-04T09:09:21.781 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.030259614, 2024-03-04T09:09:21.781 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.003830719, 2024-03-04T09:09:21.781 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:21.781 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 210, 2024-03-04T09:09:21.781 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.007521494, 2024-03-04T09:09:21.781 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000035816 2024-03-04T09:09:21.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:21.781 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:21.782 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:21.782 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:21.782 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:21.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:21.782 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1431984, 2024-03-04T09:09:21.782 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 126656 2024-03-04T09:09:21.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:21.783 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-03-04T09:09:21.783 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 2, 2024-03-04T09:09:21.783 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 162, 2024-03-04T09:09:21.783 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 320, 2024-03-04T09:09:21.783 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 183440, 2024-03-04T09:09:21.783 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 5, 2024-03-04T09:09:21.783 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 3, 2024-03-04T09:09:21.783 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.007530210, 2024-03-04T09:09:21.784 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.004408381, 2024-03-04T09:09:21.784 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.001610308, 2024-03-04T09:09:21.784 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:09:21.784 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:21.785 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 159, 2024-03-04T09:09:21.785 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.011271730, 2024-03-04T09:09:21.785 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000070891 2024-03-04T09:09:21.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:21.785 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:21.785 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:21.785 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:21.785 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:21.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:21.786 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-04T09:09:21.786 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 183440 2024-03-04T09:09:21.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:21.786 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-03-04T09:09:21.786 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 0, 2024-03-04T09:09:21.786 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-03-04T09:09:21.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:21.787 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-03-04T09:09:21.787 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-03-04T09:09:21.787 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-03-04T09:09:21.787 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-03-04T09:09:21.787 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-03-04T09:09:21.787 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-03-04T09:09:21.787 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-03-04T09:09:21.787 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-04T09:09:21.788 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-03-04T09:09:21.788 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-04T09:09:21.788 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-03-04T09:09:21.788 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-03-04T09:09:21.788 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-03-04T09:09:21.788 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-04T09:09:21.788 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-03-04T09:09:21.789 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-03-04T09:09:21.789 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-03-04T09:09:21.789 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-03-04T09:09:21.789 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-03-04T09:09:21.789 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-04T09:09:21.789 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-03-04T09:09:21.789 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-04T09:09:21.789 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-03-04T09:09:21.790 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-03-04T09:09:21.790 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-03-04T09:09:21.790 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-03-04T09:09:21.790 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-03-04T09:09:21.790 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-04T09:09:21.790 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-03-04T09:09:21.790 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-03-04T09:09:21.790 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-03-04T09:09:21.791 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-03-04T09:09:21.791 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-03-04T09:09:21.791 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-03-04T09:09:21.791 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-03-04T09:09:21.791 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-03-04T09:09:21.791 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-03-04T09:09:21.791 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 148967, 2024-03-04T09:09:21.791 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 29, 2024-03-04T09:09:21.792 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-03-04T09:09:21.792 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-03-04T09:09:21.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-03-04T09:09:21.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-03-04T09:09:21.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-03-04T09:09:21.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-03-04T09:09:21.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-03-04T09:09:21.792 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-03-04T09:09:21.793 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 0, 2024-03-04T09:09:21.793 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 0, 2024-03-04T09:09:21.793 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-03-04T09:09:21.793 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-03-04T09:09:21.793 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-03-04T09:09:21.793 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-03-04T09:09:21.794 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 928, 2024-03-04T09:09:21.794 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 2, 2024-03-04T09:09:21.794 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-03-04T09:09:21.794 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-03-04T09:09:21.794 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-03-04T09:09:21.794 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-03-04T09:09:21.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:21.795 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-04T09:09:21.795 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-03-04T09:09:21.795 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-03-04T09:09:21.795 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-03-04T09:09:21.795 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 223, 2024-03-04T09:09:21.795 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 1400806, 2024-03-04T09:09:21.795 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-03-04T09:09:21.795 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 223, 2024-03-04T09:09:21.796 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-03-04T09:09:21.796 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-03-04T09:09:21.796 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 223, 2024-03-04T09:09:21.796 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 1400806, 2024-03-04T09:09:21.796 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-03-04T09:09:21.796 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:21.796 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:21.797 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:21.797 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-03-04T09:09:21.797 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-03-04T09:09:21.797 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-03-04T09:09:22.347 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:22 smithi150 ceph-mon[36807]: pgmap v324: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 716 B/s rd, 102 B/s wr, 1 op/s 2024-03-04T09:09:22.373 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.h 2024-03-04T09:09:22.374 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.h 2024-03-04T09:09:22.374 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.h 2024-03-04T09:09:22.374 DEBUG:tasks.check_counter:Getting stats from k 2024-03-04T09:09:22.374 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-04T09:09:22.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:22 smithi071 ceph-mon[32789]: pgmap v324: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 716 B/s rd, 102 B/s wr, 1 op/s 2024-03-04T09:09:22.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:22 smithi158 ceph-mon[37241]: pgmap v324: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 716 B/s rd, 102 B/s wr, 1 op/s 2024-03-04T09:09:22.619 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T09:09:24.366 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:24 smithi150 ceph-mon[36807]: pgmap v325: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 716 B/s rd, 102 B/s wr, 1 op/s 2024-03-04T09:09:24.385 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-03-04T09:09:24.385 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-03-04T09:09:24.385 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 226, 2024-03-04T09:09:24.385 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 216, 2024-03-04T09:09:24.385 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 1433344, 2024-03-04T09:09:24.385 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 129776, 2024-03-04T09:09:24.385 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 4, 2024-03-04T09:09:24.386 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 0, 2024-03-04T09:09:24.386 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.037387491, 2024-03-04T09:09:24.386 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.008773802, 2024-03-04T09:09:24.386 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.033841554, 2024-03-04T09:09:24.386 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.003877219, 2024-03-04T09:09:24.386 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:24.386 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 215, 2024-03-04T09:09:24.386 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.006251681, 2024-03-04T09:09:24.386 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000029077 2024-03-04T09:09:24.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:24.387 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:24.387 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:24.387 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:24.387 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:24.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:24.387 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 1433344, 2024-03-04T09:09:24.387 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 129776 2024-03-04T09:09:24.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:24.388 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-03-04T09:09:24.388 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 2, 2024-03-04T09:09:24.388 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 166, 2024-03-04T09:09:24.388 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 320, 2024-03-04T09:09:24.388 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 184144, 2024-03-04T09:09:24.388 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 5, 2024-03-04T09:09:24.388 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 3, 2024-03-04T09:09:24.389 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.008030994, 2024-03-04T09:09:24.389 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.004990655, 2024-03-04T09:09:24.389 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.001627013, 2024-03-04T09:09:24.389 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:09:24.389 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:24.389 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 163, 2024-03-04T09:09:24.389 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.008823069, 2024-03-04T09:09:24.389 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000054129 2024-03-04T09:09:24.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:24.390 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:24.390 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:24.390 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:24.390 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:24.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:24.390 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-04T09:09:24.390 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 184144 2024-03-04T09:09:24.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:24.391 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-03-04T09:09:24.391 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 4, 2024-03-04T09:09:24.391 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 4, 2024-03-04T09:09:24.391 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 2992, 2024-03-04T09:09:24.391 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 448, 2024-03-04T09:09:24.391 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 2, 2024-03-04T09:09:24.391 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 1, 2024-03-04T09:09:24.391 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.000728477, 2024-03-04T09:09:24.392 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.000033657, 2024-03-04T09:09:24.392 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.000686994, 2024-03-04T09:09:24.392 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:09:24.392 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:24.392 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3, 2024-03-04T09:09:24.392 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000718652, 2024-03-04T09:09:24.392 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000239550 2024-03-04T09:09:24.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:24.393 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:24.393 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:24.393 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:24.393 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:24.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:24.393 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-04T09:09:24.393 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 448 2024-03-04T09:09:24.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:24.394 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-03-04T09:09:24.394 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 0, 2024-03-04T09:09:24.394 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-03-04T09:09:24.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:24.394 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-03-04T09:09:24.394 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-03-04T09:09:24.394 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-03-04T09:09:24.394 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-03-04T09:09:24.394 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-03-04T09:09:24.395 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-03-04T09:09:24.395 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-03-04T09:09:24.395 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-04T09:09:24.395 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-03-04T09:09:24.395 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-04T09:09:24.395 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-03-04T09:09:24.395 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-03-04T09:09:24.395 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-03-04T09:09:24.396 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-04T09:09:24.396 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-03-04T09:09:24.396 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-03-04T09:09:24.396 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-03-04T09:09:24.396 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-03-04T09:09:24.396 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-03-04T09:09:24.396 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-04T09:09:24.396 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-03-04T09:09:24.397 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-04T09:09:24.397 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-03-04T09:09:24.397 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-03-04T09:09:24.397 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-03-04T09:09:24.397 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-03-04T09:09:24.397 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-03-04T09:09:24.397 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-04T09:09:24.397 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-03-04T09:09:24.397 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-03-04T09:09:24.398 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-03-04T09:09:24.398 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-03-04T09:09:24.398 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-03-04T09:09:24.398 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-03-04T09:09:24.398 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-03-04T09:09:24.398 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-03-04T09:09:24.398 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-03-04T09:09:24.398 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 149034, 2024-03-04T09:09:24.399 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 32, 2024-03-04T09:09:24.399 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-03-04T09:09:24.399 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-03-04T09:09:24.399 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-03-04T09:09:24.399 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-03-04T09:09:24.399 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-03-04T09:09:24.399 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-03-04T09:09:24.399 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-03-04T09:09:24.399 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-03-04T09:09:24.400 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 0, 2024-03-04T09:09:24.400 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 0, 2024-03-04T09:09:24.400 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-03-04T09:09:24.400 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-03-04T09:09:24.400 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-03-04T09:09:24.400 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-03-04T09:09:24.400 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 928, 2024-03-04T09:09:24.400 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 2, 2024-03-04T09:09:24.401 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-03-04T09:09:24.401 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-03-04T09:09:24.401 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-03-04T09:09:24.401 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-03-04T09:09:24.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-03-04T09:09:24.401 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-04T09:09:24.401 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-03-04T09:09:24.401 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-03-04T09:09:24.401 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-03-04T09:09:24.402 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 228, 2024-03-04T09:09:24.402 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 1401456, 2024-03-04T09:09:24.402 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-03-04T09:09:24.402 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 228, 2024-03-04T09:09:24.402 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-03-04T09:09:24.402 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-03-04T09:09:24.402 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 228, 2024-03-04T09:09:24.402 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 1401456, 2024-03-04T09:09:24.403 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-03-04T09:09:24.403 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-03-04T09:09:24.403 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-03-04T09:09:24.403 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-03-04T09:09:24.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-03-04T09:09:24.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-03-04T09:09:24.403 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-03-04T09:09:24.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:24 smithi071 ceph-mon[32789]: pgmap v325: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 716 B/s rd, 102 B/s wr, 1 op/s 2024-03-04T09:09:24.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:24 smithi158 ceph-mon[37241]: pgmap v325: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 716 B/s rd, 102 B/s wr, 1 op/s 2024-03-04T09:09:25.074 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-03-04T09:09:25.074 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-03-04T09:09:25.075 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-03-04T09:09:25.075 DEBUG:tasks.check_counter:Getting stats from c 2024-03-04T09:09:25.075 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-04T09:09:25.328 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T09:09:26.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:26 smithi071 ceph-mon[32789]: pgmap v326: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 596 B/s rd, 1 op/s 2024-03-04T09:09:26.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:26 smithi150 ceph-mon[36807]: pgmap v326: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 596 B/s rd, 1 op/s 2024-03-04T09:09:26.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:26 smithi158 ceph-mon[37241]: pgmap v326: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 596 B/s rd, 1 op/s 2024-03-04T09:09:27.272 INFO:teuthology.orchestra.run.smithi158.stdout:{ 2024-03-04T09:09:27.273 INFO:teuthology.orchestra.run.smithi158.stdout: "AsyncMessenger::Worker-0": { 2024-03-04T09:09:27.273 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_messages": 223, 2024-03-04T09:09:27.273 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages": 213, 2024-03-04T09:09:27.273 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_bytes": 1432528, 2024-03-04T09:09:27.273 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_bytes": 127904, 2024-03-04T09:09:27.273 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_created_connections": 4, 2024-03-04T09:09:27.273 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_active_connections": 0, 2024-03-04T09:09:27.274 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_total_time": 0.040639749, 2024-03-04T09:09:27.274 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_send_time": 0.012238933, 2024-03-04T09:09:27.274 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_recv_time": 0.029980757, 2024-03-04T09:09:27.274 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_fast_dispatch_time": 0.004904973, 2024-03-04T09:09:27.274 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:27.274 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 212, 2024-03-04T09:09:27.274 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.010461434, 2024-03-04T09:09:27.274 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000049346 2024-03-04T09:09:27.274 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:27.275 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:27.275 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:27.275 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:27.275 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:27.275 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:27.275 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_encrypted_bytes": 1432528, 2024-03-04T09:09:27.275 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_encrypted_bytes": 127904 2024-03-04T09:09:27.275 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:27.276 INFO:teuthology.orchestra.run.smithi158.stdout: "AsyncMessenger::Worker-1": { 2024-03-04T09:09:27.276 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_messages": 6, 2024-03-04T09:09:27.276 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages": 166, 2024-03-04T09:09:27.276 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_bytes": 3312, 2024-03-04T09:09:27.276 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_bytes": 184240, 2024-03-04T09:09:27.276 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_created_connections": 5, 2024-03-04T09:09:27.276 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_active_connections": 3, 2024-03-04T09:09:27.276 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_total_time": 0.010485987, 2024-03-04T09:09:27.277 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_send_time": 0.006701065, 2024-03-04T09:09:27.277 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_recv_time": 0.001791244, 2024-03-04T09:09:27.277 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:09:27.277 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:27.277 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 164, 2024-03-04T09:09:27.277 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.017664987, 2024-03-04T09:09:27.277 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000107713 2024-03-04T09:09:27.277 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:27.277 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:27.278 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:27.278 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:27.278 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:27.278 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:27.278 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-03-04T09:09:27.278 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_encrypted_bytes": 184240 2024-03-04T09:09:27.278 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:27.278 INFO:teuthology.orchestra.run.smithi158.stdout: "AsyncMessenger::Worker-2": { 2024-03-04T09:09:27.278 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_messages": 0, 2024-03-04T09:09:27.279 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages": 2, 2024-03-04T09:09:27.279 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_bytes": 0, 2024-03-04T09:09:27.279 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_bytes": 0, 2024-03-04T09:09:27.279 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_created_connections": 2, 2024-03-04T09:09:27.279 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_active_connections": 1, 2024-03-04T09:09:27.279 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_total_time": 0.000262533, 2024-03-04T09:09:27.279 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_send_time": 0.000000000, 2024-03-04T09:09:27.279 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_recv_time": 0.000206673, 2024-03-04T09:09:27.280 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:09:27.280 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:27.280 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:27.280 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:27.280 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:27.280 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:27.280 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:27.280 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:27.281 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:27.281 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:27.281 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:27.281 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-04T09:09:27.281 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_encrypted_bytes": 0 2024-03-04T09:09:27.281 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:27.281 INFO:teuthology.orchestra.run.smithi158.stdout: "cct": { 2024-03-04T09:09:27.281 INFO:teuthology.orchestra.run.smithi158.stdout: "total_workers": 0, 2024-03-04T09:09:27.282 INFO:teuthology.orchestra.run.smithi158.stdout: "unhealthy_workers": 0 2024-03-04T09:09:27.282 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:27.282 INFO:teuthology.orchestra.run.smithi158.stdout: "mempool": { 2024-03-04T09:09:27.282 INFO:teuthology.orchestra.run.smithi158.stdout: "bloom_filter_bytes": 0, 2024-03-04T09:09:27.282 INFO:teuthology.orchestra.run.smithi158.stdout: "bloom_filter_items": 0, 2024-03-04T09:09:27.282 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_alloc_bytes": 0, 2024-03-04T09:09:27.282 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_alloc_items": 0, 2024-03-04T09:09:27.282 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_data_bytes": 0, 2024-03-04T09:09:27.282 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_data_items": 0, 2024-03-04T09:09:27.283 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-04T09:09:27.283 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_onode_items": 0, 2024-03-04T09:09:27.283 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-04T09:09:27.283 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_meta_items": 0, 2024-03-04T09:09:27.283 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_other_bytes": 0, 2024-03-04T09:09:27.283 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_other_items": 0, 2024-03-04T09:09:27.283 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-04T09:09:27.283 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_buffer_items": 0, 2024-03-04T09:09:27.284 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_extent_bytes": 0, 2024-03-04T09:09:27.284 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_extent_items": 0, 2024-03-04T09:09:27.284 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_blob_bytes": 0, 2024-03-04T09:09:27.284 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_blob_items": 0, 2024-03-04T09:09:27.284 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-04T09:09:27.284 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_shared_blob_items": 0, 2024-03-04T09:09:27.284 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-04T09:09:27.284 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_inline_bl_items": 0, 2024-03-04T09:09:27.284 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_fsck_bytes": 0, 2024-03-04T09:09:27.285 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_fsck_items": 0, 2024-03-04T09:09:27.285 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_txc_bytes": 0, 2024-03-04T09:09:27.285 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_txc_items": 0, 2024-03-04T09:09:27.285 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-04T09:09:27.285 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_deferred_items": 0, 2024-03-04T09:09:27.285 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_bytes": 0, 2024-03-04T09:09:27.285 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_items": 0, 2024-03-04T09:09:27.285 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_bytes": 0, 2024-03-04T09:09:27.286 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_items": 0, 2024-03-04T09:09:27.286 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_reader_bytes": 0, 2024-03-04T09:09:27.286 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_reader_items": 0, 2024-03-04T09:09:27.286 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_writer_bytes": 0, 2024-03-04T09:09:27.286 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_writer_items": 0, 2024-03-04T09:09:27.286 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_anon_bytes": 149017, 2024-03-04T09:09:27.286 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_anon_items": 31, 2024-03-04T09:09:27.286 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_meta_bytes": 0, 2024-03-04T09:09:27.286 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_meta_items": 0, 2024-03-04T09:09:27.287 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_bytes": 0, 2024-03-04T09:09:27.287 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_items": 0, 2024-03-04T09:09:27.287 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_mapbl_bytes": 0, 2024-03-04T09:09:27.287 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_mapbl_items": 0, 2024-03-04T09:09:27.287 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_pglog_bytes": 0, 2024-03-04T09:09:27.287 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_pglog_items": 0, 2024-03-04T09:09:27.287 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_bytes": 0, 2024-03-04T09:09:27.287 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_items": 0, 2024-03-04T09:09:27.288 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_mapping_bytes": 0, 2024-03-04T09:09:27.288 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_mapping_items": 0, 2024-03-04T09:09:27.288 INFO:teuthology.orchestra.run.smithi158.stdout: "pgmap_bytes": 0, 2024-03-04T09:09:27.288 INFO:teuthology.orchestra.run.smithi158.stdout: "pgmap_items": 0, 2024-03-04T09:09:27.288 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_co_bytes": 928, 2024-03-04T09:09:27.288 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_co_items": 2, 2024-03-04T09:09:27.288 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_1_bytes": 0, 2024-03-04T09:09:27.288 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_1_items": 0, 2024-03-04T09:09:27.289 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_2_bytes": 0, 2024-03-04T09:09:27.289 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_2_items": 0 2024-03-04T09:09:27.289 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:27.289 INFO:teuthology.orchestra.run.smithi158.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-04T09:09:27.289 INFO:teuthology.orchestra.run.smithi158.stdout: "val": 0, 2024-03-04T09:09:27.289 INFO:teuthology.orchestra.run.smithi158.stdout: "max": 104857600, 2024-03-04T09:09:27.289 INFO:teuthology.orchestra.run.smithi158.stdout: "get_started": 0, 2024-03-04T09:09:27.289 INFO:teuthology.orchestra.run.smithi158.stdout: "get": 225, 2024-03-04T09:09:27.290 INFO:teuthology.orchestra.run.smithi158.stdout: "get_sum": 1401066, 2024-03-04T09:09:27.290 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_fail": 0, 2024-03-04T09:09:27.290 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_success": 225, 2024-03-04T09:09:27.290 INFO:teuthology.orchestra.run.smithi158.stdout: "take": 0, 2024-03-04T09:09:27.290 INFO:teuthology.orchestra.run.smithi158.stdout: "take_sum": 0, 2024-03-04T09:09:27.290 INFO:teuthology.orchestra.run.smithi158.stdout: "put": 225, 2024-03-04T09:09:27.290 INFO:teuthology.orchestra.run.smithi158.stdout: "put_sum": 1401066, 2024-03-04T09:09:27.290 INFO:teuthology.orchestra.run.smithi158.stdout: "wait": { 2024-03-04T09:09:27.290 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:27.291 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:27.291 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:27.291 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-03-04T09:09:27.291 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-03-04T09:09:27.291 INFO:teuthology.orchestra.run.smithi158.stdout:} 2024-03-04T09:09:28.009 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.c 2024-03-04T09:09:28.009 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.c 2024-03-04T09:09:28.009 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.c 2024-03-04T09:09:28.009 DEBUG:tasks.check_counter:Getting stats from f 2024-03-04T09:09:28.009 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-04T09:09:28.256 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T09:09:28.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:28 smithi158 ceph-mon[37241]: pgmap v327: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-03-04T09:09:28.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:28 smithi071 ceph-mon[32789]: pgmap v327: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-03-04T09:09:28.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:28 smithi150 ceph-mon[36807]: pgmap v327: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-03-04T09:09:29.973 INFO:teuthology.orchestra.run.smithi158.stdout:{ 2024-03-04T09:09:29.974 INFO:teuthology.orchestra.run.smithi158.stdout: "AsyncMessenger::Worker-0": { 2024-03-04T09:09:29.974 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_messages": 0, 2024-03-04T09:09:29.974 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages": 2, 2024-03-04T09:09:29.974 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_bytes": 0, 2024-03-04T09:09:29.974 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_bytes": 0, 2024-03-04T09:09:29.974 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_created_connections": 4, 2024-03-04T09:09:29.975 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_active_connections": 0, 2024-03-04T09:09:29.975 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_total_time": 0.001551331, 2024-03-04T09:09:29.975 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_send_time": 0.000000000, 2024-03-04T09:09:29.975 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_recv_time": 0.000367166, 2024-03-04T09:09:29.975 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:09:29.975 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:29.975 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:29.976 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:29.976 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:29.976 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:29.976 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:29.976 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:29.976 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:29.976 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:29.977 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:29.977 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-04T09:09:29.977 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_encrypted_bytes": 0 2024-03-04T09:09:29.977 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:29.977 INFO:teuthology.orchestra.run.smithi158.stdout: "AsyncMessenger::Worker-1": { 2024-03-04T09:09:29.977 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_messages": 229, 2024-03-04T09:09:29.977 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages": 217, 2024-03-04T09:09:29.977 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_bytes": 1436064, 2024-03-04T09:09:29.978 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_bytes": 129600, 2024-03-04T09:09:29.978 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_created_connections": 2, 2024-03-04T09:09:29.978 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_active_connections": 1, 2024-03-04T09:09:29.978 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_total_time": 0.044314398, 2024-03-04T09:09:29.978 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_send_time": 0.012833755, 2024-03-04T09:09:29.978 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_recv_time": 0.034585151, 2024-03-04T09:09:29.978 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_fast_dispatch_time": 0.005253193, 2024-03-04T09:09:29.979 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:29.979 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 217, 2024-03-04T09:09:29.979 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.011616590, 2024-03-04T09:09:29.979 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000053532 2024-03-04T09:09:29.979 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:29.979 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:29.979 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:29.980 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:29.980 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:29.980 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:29.980 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_encrypted_bytes": 1436064, 2024-03-04T09:09:29.980 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_encrypted_bytes": 129600 2024-03-04T09:09:29.980 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:29.980 INFO:teuthology.orchestra.run.smithi158.stdout: "AsyncMessenger::Worker-2": { 2024-03-04T09:09:29.980 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_messages": 2, 2024-03-04T09:09:29.981 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages": 164, 2024-03-04T09:09:29.981 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_bytes": 320, 2024-03-04T09:09:29.981 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_bytes": 183792, 2024-03-04T09:09:29.981 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_created_connections": 5, 2024-03-04T09:09:29.981 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_active_connections": 3, 2024-03-04T09:09:29.981 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_total_time": 0.010637502, 2024-03-04T09:09:29.981 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_send_time": 0.006834512, 2024-03-04T09:09:29.982 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_recv_time": 0.001722965, 2024-03-04T09:09:29.982 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:09:29.982 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:29.982 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 161, 2024-03-04T09:09:29.982 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.017800552, 2024-03-04T09:09:29.982 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000110562 2024-03-04T09:09:29.982 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:29.983 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:29.983 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:29.983 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:29.983 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:29.983 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:29.983 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-04T09:09:29.983 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_encrypted_bytes": 183792 2024-03-04T09:09:29.983 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:29.984 INFO:teuthology.orchestra.run.smithi158.stdout: "cct": { 2024-03-04T09:09:29.984 INFO:teuthology.orchestra.run.smithi158.stdout: "total_workers": 0, 2024-03-04T09:09:29.984 INFO:teuthology.orchestra.run.smithi158.stdout: "unhealthy_workers": 0 2024-03-04T09:09:29.984 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:29.984 INFO:teuthology.orchestra.run.smithi158.stdout: "mempool": { 2024-03-04T09:09:29.984 INFO:teuthology.orchestra.run.smithi158.stdout: "bloom_filter_bytes": 0, 2024-03-04T09:09:29.984 INFO:teuthology.orchestra.run.smithi158.stdout: "bloom_filter_items": 0, 2024-03-04T09:09:29.985 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_alloc_bytes": 0, 2024-03-04T09:09:29.985 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_alloc_items": 0, 2024-03-04T09:09:29.985 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_data_bytes": 0, 2024-03-04T09:09:29.985 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_data_items": 0, 2024-03-04T09:09:29.985 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-04T09:09:29.985 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_onode_items": 0, 2024-03-04T09:09:29.985 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-04T09:09:29.986 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_meta_items": 0, 2024-03-04T09:09:29.986 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_other_bytes": 0, 2024-03-04T09:09:29.986 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_other_items": 0, 2024-03-04T09:09:29.986 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-04T09:09:29.986 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_buffer_items": 0, 2024-03-04T09:09:29.986 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_extent_bytes": 0, 2024-03-04T09:09:29.986 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_extent_items": 0, 2024-03-04T09:09:29.986 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_blob_bytes": 0, 2024-03-04T09:09:29.987 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_blob_items": 0, 2024-03-04T09:09:29.987 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-04T09:09:29.987 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_shared_blob_items": 0, 2024-03-04T09:09:29.987 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-04T09:09:29.987 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_inline_bl_items": 0, 2024-03-04T09:09:29.987 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_fsck_bytes": 0, 2024-03-04T09:09:29.987 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_fsck_items": 0, 2024-03-04T09:09:29.987 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_txc_bytes": 0, 2024-03-04T09:09:29.988 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_txc_items": 0, 2024-03-04T09:09:29.988 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-04T09:09:29.988 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_deferred_items": 0, 2024-03-04T09:09:29.988 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_bytes": 0, 2024-03-04T09:09:29.988 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_items": 0, 2024-03-04T09:09:29.988 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_bytes": 0, 2024-03-04T09:09:29.988 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_items": 0, 2024-03-04T09:09:29.989 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_reader_bytes": 0, 2024-03-04T09:09:29.989 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_reader_items": 0, 2024-03-04T09:09:29.989 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_writer_bytes": 0, 2024-03-04T09:09:29.989 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_writer_items": 0, 2024-03-04T09:09:29.989 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_anon_bytes": 149034, 2024-03-04T09:09:29.989 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_anon_items": 32, 2024-03-04T09:09:29.989 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_meta_bytes": 0, 2024-03-04T09:09:29.990 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_meta_items": 0, 2024-03-04T09:09:29.990 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_bytes": 0, 2024-03-04T09:09:29.990 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_items": 0, 2024-03-04T09:09:29.990 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_mapbl_bytes": 0, 2024-03-04T09:09:29.990 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_mapbl_items": 0, 2024-03-04T09:09:29.990 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_pglog_bytes": 0, 2024-03-04T09:09:29.990 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_pglog_items": 0, 2024-03-04T09:09:29.991 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_bytes": 0, 2024-03-04T09:09:29.991 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_items": 0, 2024-03-04T09:09:29.991 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_mapping_bytes": 0, 2024-03-04T09:09:29.991 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_mapping_items": 0, 2024-03-04T09:09:29.991 INFO:teuthology.orchestra.run.smithi158.stdout: "pgmap_bytes": 0, 2024-03-04T09:09:29.991 INFO:teuthology.orchestra.run.smithi158.stdout: "pgmap_items": 0, 2024-03-04T09:09:29.991 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_co_bytes": 928, 2024-03-04T09:09:29.991 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_co_items": 2, 2024-03-04T09:09:29.992 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_1_bytes": 0, 2024-03-04T09:09:29.992 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_1_items": 0, 2024-03-04T09:09:29.992 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_2_bytes": 0, 2024-03-04T09:09:29.992 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_2_items": 0 2024-03-04T09:09:29.992 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:29.992 INFO:teuthology.orchestra.run.smithi158.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-04T09:09:29.993 INFO:teuthology.orchestra.run.smithi158.stdout: "val": 0, 2024-03-04T09:09:29.993 INFO:teuthology.orchestra.run.smithi158.stdout: "max": 104857600, 2024-03-04T09:09:29.993 INFO:teuthology.orchestra.run.smithi158.stdout: "get_started": 0, 2024-03-04T09:09:29.993 INFO:teuthology.orchestra.run.smithi158.stdout: "get": 227, 2024-03-04T09:09:29.993 INFO:teuthology.orchestra.run.smithi158.stdout: "get_sum": 1401326, 2024-03-04T09:09:29.993 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_fail": 0, 2024-03-04T09:09:29.993 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_success": 227, 2024-03-04T09:09:29.993 INFO:teuthology.orchestra.run.smithi158.stdout: "take": 0, 2024-03-04T09:09:29.994 INFO:teuthology.orchestra.run.smithi158.stdout: "take_sum": 0, 2024-03-04T09:09:29.994 INFO:teuthology.orchestra.run.smithi158.stdout: "put": 227, 2024-03-04T09:09:29.994 INFO:teuthology.orchestra.run.smithi158.stdout: "put_sum": 1401326, 2024-03-04T09:09:29.994 INFO:teuthology.orchestra.run.smithi158.stdout: "wait": { 2024-03-04T09:09:29.994 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:29.994 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:29.994 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:29.995 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-03-04T09:09:29.995 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-03-04T09:09:29.995 INFO:teuthology.orchestra.run.smithi158.stdout:} 2024-03-04T09:09:30.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:30 smithi071 ceph-mon[32789]: pgmap v328: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1 op/s 2024-03-04T09:09:30.445 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:30 smithi158 ceph-mon[37241]: pgmap v328: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1 op/s 2024-03-04T09:09:30.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:30 smithi150 ceph-mon[36807]: pgmap v328: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1 op/s 2024-03-04T09:09:30.797 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-03-04T09:09:30.797 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-03-04T09:09:30.797 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-03-04T09:09:30.797 DEBUG:tasks.check_counter:Getting stats from i 2024-03-04T09:09:30.798 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-04T09:09:31.036 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T09:09:32.365 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:32 smithi158 ceph-mon[37241]: pgmap v329: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-03-04T09:09:32.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:32 smithi071 ceph-mon[32789]: pgmap v329: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-03-04T09:09:32.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:32 smithi150 ceph-mon[36807]: pgmap v329: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-03-04T09:09:32.770 INFO:teuthology.orchestra.run.smithi158.stdout:{ 2024-03-04T09:09:32.770 INFO:teuthology.orchestra.run.smithi158.stdout: "AsyncMessenger::Worker-0": { 2024-03-04T09:09:32.770 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_messages": 1860, 2024-03-04T09:09:32.771 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages": 1846, 2024-03-04T09:09:32.771 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_bytes": 194869760, 2024-03-04T09:09:32.771 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_bytes": 644270, 2024-03-04T09:09:32.771 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_created_connections": 7, 2024-03-04T09:09:32.771 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_active_connections": 3, 2024-03-04T09:09:32.771 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_total_time": 0.283034121, 2024-03-04T09:09:32.771 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_send_time": 0.056588996, 2024-03-04T09:09:32.771 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_recv_time": 0.170381664, 2024-03-04T09:09:32.771 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_fast_dispatch_time": 0.049447687, 2024-03-04T09:09:32.772 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:32.772 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 1846, 2024-03-04T09:09:32.772 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.072070457, 2024-03-04T09:09:32.772 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000039041 2024-03-04T09:09:32.772 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.772 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:32.772 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.772 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.773 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.773 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.773 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_encrypted_bytes": 1459808, 2024-03-04T09:09:32.773 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_encrypted_bytes": 131968 2024-03-04T09:09:32.773 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.773 INFO:teuthology.orchestra.run.smithi158.stdout: "AsyncMessenger::Worker-1": { 2024-03-04T09:09:32.773 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_messages": 693, 2024-03-04T09:09:32.773 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages": 854, 2024-03-04T09:09:32.774 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_bytes": 143374476, 2024-03-04T09:09:32.774 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_bytes": 661628, 2024-03-04T09:09:32.774 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_created_connections": 9, 2024-03-04T09:09:32.774 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_active_connections": 7, 2024-03-04T09:09:32.774 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_total_time": 0.120730472, 2024-03-04T09:09:32.774 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_send_time": 0.021296108, 2024-03-04T09:09:32.774 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_recv_time": 0.077061467, 2024-03-04T09:09:32.775 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_fast_dispatch_time": 0.015857367, 2024-03-04T09:09:32.775 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:32.775 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 851, 2024-03-04T09:09:32.775 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.037321133, 2024-03-04T09:09:32.775 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000043855 2024-03-04T09:09:32.776 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.776 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:32.776 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.776 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.776 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.776 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.776 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-04T09:09:32.777 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_encrypted_bytes": 441824 2024-03-04T09:09:32.777 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.777 INFO:teuthology.orchestra.run.smithi158.stdout: "AsyncMessenger::Worker-2": { 2024-03-04T09:09:32.777 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_messages": 910, 2024-03-04T09:09:32.777 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages": 912, 2024-03-04T09:09:32.777 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_bytes": 209956157, 2024-03-04T09:09:32.777 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_bytes": 289387, 2024-03-04T09:09:32.777 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_created_connections": 6, 2024-03-04T09:09:32.777 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_active_connections": 5, 2024-03-04T09:09:32.778 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_total_time": 0.158375229, 2024-03-04T09:09:32.778 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_send_time": 0.021577292, 2024-03-04T09:09:32.778 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_recv_time": 0.109670579, 2024-03-04T09:09:32.778 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_fast_dispatch_time": 0.020818042, 2024-03-04T09:09:32.778 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:32.778 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 910, 2024-03-04T09:09:32.778 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.027974589, 2024-03-04T09:09:32.778 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000030741 2024-03-04T09:09:32.778 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.779 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:32.779 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.779 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.779 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.779 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.779 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-04T09:09:32.779 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_encrypted_bytes": 0 2024-03-04T09:09:32.779 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.779 INFO:teuthology.orchestra.run.smithi158.stdout: "cct": { 2024-03-04T09:09:32.780 INFO:teuthology.orchestra.run.smithi158.stdout: "total_workers": 1, 2024-03-04T09:09:32.780 INFO:teuthology.orchestra.run.smithi158.stdout: "unhealthy_workers": 0 2024-03-04T09:09:32.780 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.780 INFO:teuthology.orchestra.run.smithi158.stdout: "finisher-MDSRank": { 2024-03-04T09:09:32.780 INFO:teuthology.orchestra.run.smithi158.stdout: "queue_len": 0, 2024-03-04T09:09:32.780 INFO:teuthology.orchestra.run.smithi158.stdout: "complete_latency": { 2024-03-04T09:09:32.780 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 4971, 2024-03-04T09:09:32.781 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.135007868, 2024-03-04T09:09:32.781 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000027159 2024-03-04T09:09:32.781 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-03-04T09:09:32.781 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.781 INFO:teuthology.orchestra.run.smithi158.stdout: "finisher-PurgeQueue": { 2024-03-04T09:09:32.781 INFO:teuthology.orchestra.run.smithi158.stdout: "queue_len": 0, 2024-03-04T09:09:32.781 INFO:teuthology.orchestra.run.smithi158.stdout: "complete_latency": { 2024-03-04T09:09:32.781 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.782 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.782 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.782 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-03-04T09:09:32.782 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.782 INFO:teuthology.orchestra.run.smithi158.stdout: "mds": { 2024-03-04T09:09:32.782 INFO:teuthology.orchestra.run.smithi158.stdout: "request": 0, 2024-03-04T09:09:32.782 INFO:teuthology.orchestra.run.smithi158.stdout: "reply": 0, 2024-03-04T09:09:32.782 INFO:teuthology.orchestra.run.smithi158.stdout: "reply_latency": { 2024-03-04T09:09:32.782 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.783 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.783 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.783 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.783 INFO:teuthology.orchestra.run.smithi158.stdout: "slow_reply": 0, 2024-03-04T09:09:32.783 INFO:teuthology.orchestra.run.smithi158.stdout: "forward": 0, 2024-03-04T09:09:32.783 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_fetch_complete": 0, 2024-03-04T09:09:32.783 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_fetch_keys": 0, 2024-03-04T09:09:32.783 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_commit": 0, 2024-03-04T09:09:32.783 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_split": 0, 2024-03-04T09:09:32.784 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_merge": 0, 2024-03-04T09:09:32.784 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes": 31147, 2024-03-04T09:09:32.784 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes_top": 7234, 2024-03-04T09:09:32.784 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes_bottom": 23913, 2024-03-04T09:09:32.784 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes_pin_tail": 0, 2024-03-04T09:09:32.784 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes_pinned": 20813, 2024-03-04T09:09:32.784 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes_expired": 0, 2024-03-04T09:09:32.784 INFO:teuthology.orchestra.run.smithi158.stdout: "inodes_with_caps": 0, 2024-03-04T09:09:32.784 INFO:teuthology.orchestra.run.smithi158.stdout: "caps": 0, 2024-03-04T09:09:32.785 INFO:teuthology.orchestra.run.smithi158.stdout: "subtrees": 2, 2024-03-04T09:09:32.785 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse": 0, 2024-03-04T09:09:32.785 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse_hit": 0, 2024-03-04T09:09:32.785 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse_forward": 0, 2024-03-04T09:09:32.785 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse_discover": 0, 2024-03-04T09:09:32.785 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse_dir_fetch": 0, 2024-03-04T09:09:32.785 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse_remote_ino": 0, 2024-03-04T09:09:32.785 INFO:teuthology.orchestra.run.smithi158.stdout: "traverse_lock": 0, 2024-03-04T09:09:32.786 INFO:teuthology.orchestra.run.smithi158.stdout: "load_cent": 0, 2024-03-04T09:09:32.786 INFO:teuthology.orchestra.run.smithi158.stdout: "q": 0, 2024-03-04T09:09:32.786 INFO:teuthology.orchestra.run.smithi158.stdout: "exported": 0, 2024-03-04T09:09:32.786 INFO:teuthology.orchestra.run.smithi158.stdout: "exported_inodes": 0, 2024-03-04T09:09:32.786 INFO:teuthology.orchestra.run.smithi158.stdout: "imported": 0, 2024-03-04T09:09:32.786 INFO:teuthology.orchestra.run.smithi158.stdout: "imported_inodes": 0, 2024-03-04T09:09:32.786 INFO:teuthology.orchestra.run.smithi158.stdout: "openino_dir_fetch": 0, 2024-03-04T09:09:32.786 INFO:teuthology.orchestra.run.smithi158.stdout: "openino_backtrace_fetch": 0, 2024-03-04T09:09:32.787 INFO:teuthology.orchestra.run.smithi158.stdout: "openino_peer_discover": 0, 2024-03-04T09:09:32.787 INFO:teuthology.orchestra.run.smithi158.stdout: "root_rfiles": 68, 2024-03-04T09:09:32.787 INFO:teuthology.orchestra.run.smithi158.stdout: "root_rbytes": 21196213, 2024-03-04T09:09:32.787 INFO:teuthology.orchestra.run.smithi158.stdout: "root_rsnaps": 7, 2024-03-04T09:09:32.787 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_backtrace_fetch": 0, 2024-03-04T09:09:32.787 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_set_tag": 0, 2024-03-04T09:09:32.787 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_backtrace_repaired": 0, 2024-03-04T09:09:32.787 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_inotable_repaired": 0, 2024-03-04T09:09:32.788 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_dir_inodes": 0, 2024-03-04T09:09:32.788 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_dir_base_inodes": 0, 2024-03-04T09:09:32.788 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_dirfrag_rstats": 0, 2024-03-04T09:09:32.788 INFO:teuthology.orchestra.run.smithi158.stdout: "scrub_file_inodes": 0, 2024-03-04T09:09:32.788 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_inode_file_caps": 0, 2024-03-04T09:09:32.788 INFO:teuthology.orchestra.run.smithi158.stdout: "ceph_cap_op_revoke": 0, 2024-03-04T09:09:32.788 INFO:teuthology.orchestra.run.smithi158.stdout: "ceph_cap_op_grant": 0, 2024-03-04T09:09:32.788 INFO:teuthology.orchestra.run.smithi158.stdout: "ceph_cap_op_trunc": 0, 2024-03-04T09:09:32.789 INFO:teuthology.orchestra.run.smithi158.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-04T09:09:32.789 INFO:teuthology.orchestra.run.smithi158.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-04T09:09:32.789 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_client_caps": 0, 2024-03-04T09:09:32.789 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_client_caps_dirty": 0, 2024-03-04T09:09:32.790 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_client_cap_release": 0, 2024-03-04T09:09:32.790 INFO:teuthology.orchestra.run.smithi158.stdout: "process_request_cap_release": 0 2024-03-04T09:09:32.790 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.790 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_cache": { 2024-03-04T09:09:32.790 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_update": 0, 2024-03-04T09:09:32.790 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_update_receipt": 0, 2024-03-04T09:09:32.791 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_try_discover": 0, 2024-03-04T09:09:32.791 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_send_discover": 0, 2024-03-04T09:09:32.791 INFO:teuthology.orchestra.run.smithi158.stdout: "dir_handle_discover": 0, 2024-03-04T09:09:32.791 INFO:teuthology.orchestra.run.smithi158.stdout: "num_strays": 0, 2024-03-04T09:09:32.791 INFO:teuthology.orchestra.run.smithi158.stdout: "num_strays_delayed": 0, 2024-03-04T09:09:32.791 INFO:teuthology.orchestra.run.smithi158.stdout: "num_strays_enqueuing": 0, 2024-03-04T09:09:32.791 INFO:teuthology.orchestra.run.smithi158.stdout: "strays_created": 0, 2024-03-04T09:09:32.791 INFO:teuthology.orchestra.run.smithi158.stdout: "strays_enqueued": 0, 2024-03-04T09:09:32.791 INFO:teuthology.orchestra.run.smithi158.stdout: "strays_reintegrated": 0, 2024-03-04T09:09:32.792 INFO:teuthology.orchestra.run.smithi158.stdout: "strays_migrated": 0, 2024-03-04T09:09:32.792 INFO:teuthology.orchestra.run.smithi158.stdout: "num_recovering_processing": 0, 2024-03-04T09:09:32.792 INFO:teuthology.orchestra.run.smithi158.stdout: "num_recovering_enqueued": 0, 2024-03-04T09:09:32.792 INFO:teuthology.orchestra.run.smithi158.stdout: "num_recovering_prioritized": 0, 2024-03-04T09:09:32.792 INFO:teuthology.orchestra.run.smithi158.stdout: "recovery_started": 0, 2024-03-04T09:09:32.792 INFO:teuthology.orchestra.run.smithi158.stdout: "recovery_completed": 0, 2024-03-04T09:09:32.792 INFO:teuthology.orchestra.run.smithi158.stdout: "ireq_enqueue_scrub": 0, 2024-03-04T09:09:32.792 INFO:teuthology.orchestra.run.smithi158.stdout: "ireq_exportdir": 0, 2024-03-04T09:09:32.793 INFO:teuthology.orchestra.run.smithi158.stdout: "ireq_flush": 0, 2024-03-04T09:09:32.793 INFO:teuthology.orchestra.run.smithi158.stdout: "ireq_fragmentdir": 0, 2024-03-04T09:09:32.793 INFO:teuthology.orchestra.run.smithi158.stdout: "ireq_fragstats": 0, 2024-03-04T09:09:32.793 INFO:teuthology.orchestra.run.smithi158.stdout: "ireq_inodestats": 0 2024-03-04T09:09:32.793 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.793 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_log": { 2024-03-04T09:09:32.793 INFO:teuthology.orchestra.run.smithi158.stdout: "evlrg": 0, 2024-03-04T09:09:32.793 INFO:teuthology.orchestra.run.smithi158.stdout: "evadd": 0, 2024-03-04T09:09:32.794 INFO:teuthology.orchestra.run.smithi158.stdout: "evex": 0, 2024-03-04T09:09:32.794 INFO:teuthology.orchestra.run.smithi158.stdout: "evtrm": 0, 2024-03-04T09:09:32.794 INFO:teuthology.orchestra.run.smithi158.stdout: "ev": 138732, 2024-03-04T09:09:32.794 INFO:teuthology.orchestra.run.smithi158.stdout: "evexg": 0, 2024-03-04T09:09:32.794 INFO:teuthology.orchestra.run.smithi158.stdout: "evexd": 0, 2024-03-04T09:09:32.794 INFO:teuthology.orchestra.run.smithi158.stdout: "segadd": 0, 2024-03-04T09:09:32.795 INFO:teuthology.orchestra.run.smithi158.stdout: "segex": 0, 2024-03-04T09:09:32.795 INFO:teuthology.orchestra.run.smithi158.stdout: "segtrm": 0, 2024-03-04T09:09:32.795 INFO:teuthology.orchestra.run.smithi158.stdout: "seg": 128, 2024-03-04T09:09:32.795 INFO:teuthology.orchestra.run.smithi158.stdout: "segmjr": 13, 2024-03-04T09:09:32.795 INFO:teuthology.orchestra.run.smithi158.stdout: "segexg": 0, 2024-03-04T09:09:32.795 INFO:teuthology.orchestra.run.smithi158.stdout: "segexd": 0, 2024-03-04T09:09:32.795 INFO:teuthology.orchestra.run.smithi158.stdout: "expos": 143482111, 2024-03-04T09:09:32.795 INFO:teuthology.orchestra.run.smithi158.stdout: "wrpos": 550044378, 2024-03-04T09:09:32.796 INFO:teuthology.orchestra.run.smithi158.stdout: "rdpos": 550044242, 2024-03-04T09:09:32.796 INFO:teuthology.orchestra.run.smithi158.stdout: "jlat": { 2024-03-04T09:09:32.796 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.796 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.796 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.796 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.797 INFO:teuthology.orchestra.run.smithi158.stdout: "replayed": 138732 2024-03-04T09:09:32.797 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.797 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_mem": { 2024-03-04T09:09:32.797 INFO:teuthology.orchestra.run.smithi158.stdout: "ino": 483, 2024-03-04T09:09:32.797 INFO:teuthology.orchestra.run.smithi158.stdout: "ino+": 31075, 2024-03-04T09:09:32.797 INFO:teuthology.orchestra.run.smithi158.stdout: "ino-": 30592, 2024-03-04T09:09:32.797 INFO:teuthology.orchestra.run.smithi158.stdout: "dir": 47, 2024-03-04T09:09:32.797 INFO:teuthology.orchestra.run.smithi158.stdout: "dir+": 149, 2024-03-04T09:09:32.798 INFO:teuthology.orchestra.run.smithi158.stdout: "dir-": 102, 2024-03-04T09:09:32.798 INFO:teuthology.orchestra.run.smithi158.stdout: "dn": 31147, 2024-03-04T09:09:32.798 INFO:teuthology.orchestra.run.smithi158.stdout: "dn+": 31147, 2024-03-04T09:09:32.798 INFO:teuthology.orchestra.run.smithi158.stdout: "dn-": 0, 2024-03-04T09:09:32.798 INFO:teuthology.orchestra.run.smithi158.stdout: "cap": 0, 2024-03-04T09:09:32.798 INFO:teuthology.orchestra.run.smithi158.stdout: "cap+": 0, 2024-03-04T09:09:32.798 INFO:teuthology.orchestra.run.smithi158.stdout: "cap-": 0, 2024-03-04T09:09:32.798 INFO:teuthology.orchestra.run.smithi158.stdout: "rss": 78864, 2024-03-04T09:09:32.798 INFO:teuthology.orchestra.run.smithi158.stdout: "heap": 215420 2024-03-04T09:09:32.799 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.799 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_server": { 2024-03-04T09:09:32.799 INFO:teuthology.orchestra.run.smithi158.stdout: "dispatch_client_request": 0, 2024-03-04T09:09:32.799 INFO:teuthology.orchestra.run.smithi158.stdout: "dispatch_server_request": 0, 2024-03-04T09:09:32.799 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_client_request": 0, 2024-03-04T09:09:32.799 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_client_session": 0, 2024-03-04T09:09:32.799 INFO:teuthology.orchestra.run.smithi158.stdout: "handle_peer_request": 0, 2024-03-04T09:09:32.799 INFO:teuthology.orchestra.run.smithi158.stdout: "req_create_latency": { 2024-03-04T09:09:32.799 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.800 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.800 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.800 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.800 INFO:teuthology.orchestra.run.smithi158.stdout: "req_getattr_latency": { 2024-03-04T09:09:32.800 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.800 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.800 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.801 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.801 INFO:teuthology.orchestra.run.smithi158.stdout: "req_getfilelock_latency": { 2024-03-04T09:09:32.801 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.801 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.801 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.801 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.801 INFO:teuthology.orchestra.run.smithi158.stdout: "req_link_latency": { 2024-03-04T09:09:32.801 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.801 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.801 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.802 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.802 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lookup_latency": { 2024-03-04T09:09:32.802 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.802 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.802 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.802 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.802 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lookuphash_latency": { 2024-03-04T09:09:32.802 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.802 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.803 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.803 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.803 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lookupino_latency": { 2024-03-04T09:09:32.803 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.803 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.804 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.804 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.804 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lookupname_latency": { 2024-03-04T09:09:32.805 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.805 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.805 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.805 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.805 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lookupparent_latency": { 2024-03-04T09:09:32.805 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.805 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.806 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.806 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.806 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lookupsnap_latency": { 2024-03-04T09:09:32.806 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.806 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.806 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.806 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.806 INFO:teuthology.orchestra.run.smithi158.stdout: "req_lssnap_latency": { 2024-03-04T09:09:32.807 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.807 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.807 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.807 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.807 INFO:teuthology.orchestra.run.smithi158.stdout: "req_mkdir_latency": { 2024-03-04T09:09:32.807 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.808 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.808 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.808 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.808 INFO:teuthology.orchestra.run.smithi158.stdout: "req_mknod_latency": { 2024-03-04T09:09:32.809 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.809 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.809 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.809 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.809 INFO:teuthology.orchestra.run.smithi158.stdout: "req_mksnap_latency": { 2024-03-04T09:09:32.809 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.809 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.809 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.809 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.810 INFO:teuthology.orchestra.run.smithi158.stdout: "req_open_latency": { 2024-03-04T09:09:32.810 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.810 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.810 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.810 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.810 INFO:teuthology.orchestra.run.smithi158.stdout: "req_readdir_latency": { 2024-03-04T09:09:32.810 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.810 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.811 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.811 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.811 INFO:teuthology.orchestra.run.smithi158.stdout: "req_rename_latency": { 2024-03-04T09:09:32.811 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.811 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.811 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.811 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.811 INFO:teuthology.orchestra.run.smithi158.stdout: "req_renamesnap_latency": { 2024-03-04T09:09:32.811 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.812 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.812 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.812 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.812 INFO:teuthology.orchestra.run.smithi158.stdout: "req_snapdiff_latency": { 2024-03-04T09:09:32.812 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.812 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.812 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.812 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.813 INFO:teuthology.orchestra.run.smithi158.stdout: "req_rmdir_latency": { 2024-03-04T09:09:32.813 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.813 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.813 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.813 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.813 INFO:teuthology.orchestra.run.smithi158.stdout: "req_rmsnap_latency": { 2024-03-04T09:09:32.813 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.813 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.813 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.814 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.814 INFO:teuthology.orchestra.run.smithi158.stdout: "req_rmxattr_latency": { 2024-03-04T09:09:32.814 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.814 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.814 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.814 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.814 INFO:teuthology.orchestra.run.smithi158.stdout: "req_setattr_latency": { 2024-03-04T09:09:32.814 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.815 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.815 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.815 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.815 INFO:teuthology.orchestra.run.smithi158.stdout: "req_setdirlayout_latency": { 2024-03-04T09:09:32.815 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.815 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.815 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.815 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.815 INFO:teuthology.orchestra.run.smithi158.stdout: "req_setfilelock_latency": { 2024-03-04T09:09:32.816 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.816 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.816 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.816 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.816 INFO:teuthology.orchestra.run.smithi158.stdout: "req_setlayout_latency": { 2024-03-04T09:09:32.816 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.817 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.817 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.817 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.817 INFO:teuthology.orchestra.run.smithi158.stdout: "req_setxattr_latency": { 2024-03-04T09:09:32.817 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.817 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.817 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.817 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.817 INFO:teuthology.orchestra.run.smithi158.stdout: "req_symlink_latency": { 2024-03-04T09:09:32.818 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.818 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.818 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.818 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.818 INFO:teuthology.orchestra.run.smithi158.stdout: "req_unlink_latency": { 2024-03-04T09:09:32.818 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.818 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.819 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.819 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.819 INFO:teuthology.orchestra.run.smithi158.stdout: "cap_revoke_eviction": 0, 2024-03-04T09:09:32.819 INFO:teuthology.orchestra.run.smithi158.stdout: "cap_acquisition_throttle": 0, 2024-03-04T09:09:32.819 INFO:teuthology.orchestra.run.smithi158.stdout: "req_getvxattr_latency": { 2024-03-04T09:09:32.819 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.819 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.819 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.819 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-03-04T09:09:32.820 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.820 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_sessions": { 2024-03-04T09:09:32.820 INFO:teuthology.orchestra.run.smithi158.stdout: "session_count": 2, 2024-03-04T09:09:32.820 INFO:teuthology.orchestra.run.smithi158.stdout: "session_add": 6, 2024-03-04T09:09:32.820 INFO:teuthology.orchestra.run.smithi158.stdout: "session_remove": 4, 2024-03-04T09:09:32.820 INFO:teuthology.orchestra.run.smithi158.stdout: "sessions_open": 4, 2024-03-04T09:09:32.820 INFO:teuthology.orchestra.run.smithi158.stdout: "sessions_stale": 0, 2024-03-04T09:09:32.820 INFO:teuthology.orchestra.run.smithi158.stdout: "total_load": 0, 2024-03-04T09:09:32.821 INFO:teuthology.orchestra.run.smithi158.stdout: "average_load": 0, 2024-03-04T09:09:32.821 INFO:teuthology.orchestra.run.smithi158.stdout: "avg_session_uptime": 3583, 2024-03-04T09:09:32.821 INFO:teuthology.orchestra.run.smithi158.stdout: "mdthresh_evicted": 0 2024-03-04T09:09:32.821 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.821 INFO:teuthology.orchestra.run.smithi158.stdout: "mempool": { 2024-03-04T09:09:32.821 INFO:teuthology.orchestra.run.smithi158.stdout: "bloom_filter_bytes": 0, 2024-03-04T09:09:32.821 INFO:teuthology.orchestra.run.smithi158.stdout: "bloom_filter_items": 0, 2024-03-04T09:09:32.821 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_alloc_bytes": 0, 2024-03-04T09:09:32.821 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_alloc_items": 0, 2024-03-04T09:09:32.822 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_data_bytes": 0, 2024-03-04T09:09:32.822 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_data_items": 0, 2024-03-04T09:09:32.822 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-04T09:09:32.822 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_onode_items": 0, 2024-03-04T09:09:32.822 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-04T09:09:32.822 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_meta_items": 0, 2024-03-04T09:09:32.822 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_other_bytes": 0, 2024-03-04T09:09:32.822 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_other_items": 0, 2024-03-04T09:09:32.822 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-04T09:09:32.822 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_buffer_items": 0, 2024-03-04T09:09:32.823 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_extent_bytes": 0, 2024-03-04T09:09:32.823 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_extent_items": 0, 2024-03-04T09:09:32.823 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_blob_bytes": 0, 2024-03-04T09:09:32.823 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_blob_items": 0, 2024-03-04T09:09:32.823 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-04T09:09:32.823 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_shared_blob_items": 0, 2024-03-04T09:09:32.823 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-04T09:09:32.823 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_inline_bl_items": 0, 2024-03-04T09:09:32.824 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_fsck_bytes": 0, 2024-03-04T09:09:32.824 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_fsck_items": 0, 2024-03-04T09:09:32.824 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_txc_bytes": 0, 2024-03-04T09:09:32.824 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_txc_items": 0, 2024-03-04T09:09:32.824 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-04T09:09:32.825 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_deferred_items": 0, 2024-03-04T09:09:32.825 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_bytes": 0, 2024-03-04T09:09:32.825 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_items": 0, 2024-03-04T09:09:32.825 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_bytes": 0, 2024-03-04T09:09:32.825 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_items": 0, 2024-03-04T09:09:32.825 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_reader_bytes": 0, 2024-03-04T09:09:32.825 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_reader_items": 0, 2024-03-04T09:09:32.826 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_writer_bytes": 0, 2024-03-04T09:09:32.826 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_writer_items": 0, 2024-03-04T09:09:32.826 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_anon_bytes": 260857, 2024-03-04T09:09:32.826 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_anon_items": 657, 2024-03-04T09:09:32.826 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_meta_bytes": 0, 2024-03-04T09:09:32.826 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_meta_items": 0, 2024-03-04T09:09:32.826 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_bytes": 0, 2024-03-04T09:09:32.826 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_items": 0, 2024-03-04T09:09:32.827 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_mapbl_bytes": 0, 2024-03-04T09:09:32.827 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_mapbl_items": 0, 2024-03-04T09:09:32.827 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_pglog_bytes": 0, 2024-03-04T09:09:32.827 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_pglog_items": 0, 2024-03-04T09:09:32.827 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_bytes": 9416, 2024-03-04T09:09:32.827 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_items": 180, 2024-03-04T09:09:32.827 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_mapping_bytes": 0, 2024-03-04T09:09:32.827 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_mapping_items": 0, 2024-03-04T09:09:32.828 INFO:teuthology.orchestra.run.smithi158.stdout: "pgmap_bytes": 0, 2024-03-04T09:09:32.828 INFO:teuthology.orchestra.run.smithi158.stdout: "pgmap_items": 0, 2024-03-04T09:09:32.828 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_co_bytes": 23290756, 2024-03-04T09:09:32.828 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_co_items": 138306, 2024-03-04T09:09:32.828 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_1_bytes": 0, 2024-03-04T09:09:32.828 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_1_items": 0, 2024-03-04T09:09:32.828 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_2_bytes": 0, 2024-03-04T09:09:32.828 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_2_items": 0 2024-03-04T09:09:32.829 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.829 INFO:teuthology.orchestra.run.smithi158.stdout: "objecter": { 2024-03-04T09:09:32.829 INFO:teuthology.orchestra.run.smithi158.stdout: "op_active": 0, 2024-03-04T09:09:32.829 INFO:teuthology.orchestra.run.smithi158.stdout: "op_laggy": 0, 2024-03-04T09:09:32.829 INFO:teuthology.orchestra.run.smithi158.stdout: "op_send": 3212, 2024-03-04T09:09:32.829 INFO:teuthology.orchestra.run.smithi158.stdout: "op_send_bytes": 26, 2024-03-04T09:09:32.829 INFO:teuthology.orchestra.run.smithi158.stdout: "op_resend": 4, 2024-03-04T09:09:32.829 INFO:teuthology.orchestra.run.smithi158.stdout: "op_reply": 3212, 2024-03-04T09:09:32.830 INFO:teuthology.orchestra.run.smithi158.stdout: "op_latency": { 2024-03-04T09:09:32.830 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 3212, 2024-03-04T09:09:32.830 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 35.342616446, 2024-03-04T09:09:32.830 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.011003305 2024-03-04T09:09:32.830 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.830 INFO:teuthology.orchestra.run.smithi158.stdout: "op_inflight": 0, 2024-03-04T09:09:32.830 INFO:teuthology.orchestra.run.smithi158.stdout: "oplen_avg": { 2024-03-04T09:09:32.830 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 3212, 2024-03-04T09:09:32.831 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 3213 2024-03-04T09:09:32.831 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.831 INFO:teuthology.orchestra.run.smithi158.stdout: "op": 3212, 2024-03-04T09:09:32.831 INFO:teuthology.orchestra.run.smithi158.stdout: "op_r": 3212, 2024-03-04T09:09:32.831 INFO:teuthology.orchestra.run.smithi158.stdout: "op_w": 0, 2024-03-04T09:09:32.831 INFO:teuthology.orchestra.run.smithi158.stdout: "op_rmw": 0, 2024-03-04T09:09:32.831 INFO:teuthology.orchestra.run.smithi158.stdout: "op_pg": 0, 2024-03-04T09:09:32.831 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_stat": 1969, 2024-03-04T09:09:32.832 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_create": 0, 2024-03-04T09:09:32.832 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_read": 1240, 2024-03-04T09:09:32.832 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_write": 0, 2024-03-04T09:09:32.832 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_writefull": 0, 2024-03-04T09:09:32.832 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_writesame": 0, 2024-03-04T09:09:32.832 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_append": 0, 2024-03-04T09:09:32.832 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_zero": 0, 2024-03-04T09:09:32.833 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_truncate": 0, 2024-03-04T09:09:32.833 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_delete": 0, 2024-03-04T09:09:32.833 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_mapext": 0, 2024-03-04T09:09:32.833 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_sparse_read": 0, 2024-03-04T09:09:32.833 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_clonerange": 0, 2024-03-04T09:09:32.833 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_getxattr": 2, 2024-03-04T09:09:32.833 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_setxattr": 0, 2024-03-04T09:09:32.833 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_cmpxattr": 0, 2024-03-04T09:09:32.834 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_rmxattr": 0, 2024-03-04T09:09:32.834 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_resetxattrs": 0, 2024-03-04T09:09:32.834 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_call": 0, 2024-03-04T09:09:32.834 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_watch": 0, 2024-03-04T09:09:32.834 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_notify": 0, 2024-03-04T09:09:32.834 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_src_cmpxattr": 0, 2024-03-04T09:09:32.834 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_pgls": 0, 2024-03-04T09:09:32.834 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_pgls_filter": 0, 2024-03-04T09:09:32.834 INFO:teuthology.orchestra.run.smithi158.stdout: "osdop_other": 0, 2024-03-04T09:09:32.835 INFO:teuthology.orchestra.run.smithi158.stdout: "linger_active": 0, 2024-03-04T09:09:32.835 INFO:teuthology.orchestra.run.smithi158.stdout: "linger_send": 0, 2024-03-04T09:09:32.835 INFO:teuthology.orchestra.run.smithi158.stdout: "linger_resend": 0, 2024-03-04T09:09:32.835 INFO:teuthology.orchestra.run.smithi158.stdout: "linger_ping": 0, 2024-03-04T09:09:32.835 INFO:teuthology.orchestra.run.smithi158.stdout: "poolop_active": 0, 2024-03-04T09:09:32.835 INFO:teuthology.orchestra.run.smithi158.stdout: "poolop_send": 0, 2024-03-04T09:09:32.835 INFO:teuthology.orchestra.run.smithi158.stdout: "poolop_resend": 0, 2024-03-04T09:09:32.835 INFO:teuthology.orchestra.run.smithi158.stdout: "poolstat_active": 0, 2024-03-04T09:09:32.836 INFO:teuthology.orchestra.run.smithi158.stdout: "poolstat_send": 0, 2024-03-04T09:09:32.836 INFO:teuthology.orchestra.run.smithi158.stdout: "poolstat_resend": 0, 2024-03-04T09:09:32.836 INFO:teuthology.orchestra.run.smithi158.stdout: "statfs_active": 0, 2024-03-04T09:09:32.836 INFO:teuthology.orchestra.run.smithi158.stdout: "statfs_send": 0, 2024-03-04T09:09:32.836 INFO:teuthology.orchestra.run.smithi158.stdout: "statfs_resend": 0, 2024-03-04T09:09:32.836 INFO:teuthology.orchestra.run.smithi158.stdout: "command_active": 0, 2024-03-04T09:09:32.836 INFO:teuthology.orchestra.run.smithi158.stdout: "command_send": 0, 2024-03-04T09:09:32.836 INFO:teuthology.orchestra.run.smithi158.stdout: "command_resend": 0, 2024-03-04T09:09:32.837 INFO:teuthology.orchestra.run.smithi158.stdout: "map_epoch": 104, 2024-03-04T09:09:32.837 INFO:teuthology.orchestra.run.smithi158.stdout: "map_full": 0, 2024-03-04T09:09:32.837 INFO:teuthology.orchestra.run.smithi158.stdout: "map_inc": 13, 2024-03-04T09:09:32.837 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_sessions": 11, 2024-03-04T09:09:32.837 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_session_open": 11, 2024-03-04T09:09:32.837 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_session_close": 0, 2024-03-04T09:09:32.837 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_laggy": 0, 2024-03-04T09:09:32.837 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_wr": 0, 2024-03-04T09:09:32.837 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_rd": 2, 2024-03-04T09:09:32.838 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_del": 0 2024-03-04T09:09:32.838 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.838 INFO:teuthology.orchestra.run.smithi158.stdout: "oft": { 2024-03-04T09:09:32.838 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_total_objs": 0, 2024-03-04T09:09:32.838 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_total_kv_pairs": 0, 2024-03-04T09:09:32.838 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_total_updates": 0, 2024-03-04T09:09:32.838 INFO:teuthology.orchestra.run.smithi158.stdout: "omap_total_removes": 0 2024-03-04T09:09:32.838 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.839 INFO:teuthology.orchestra.run.smithi158.stdout: "osd-slow-ops": { 2024-03-04T09:09:32.839 INFO:teuthology.orchestra.run.smithi158.stdout: "slow_ops_count": 0 2024-03-04T09:09:32.839 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.839 INFO:teuthology.orchestra.run.smithi158.stdout: "purge_queue": { 2024-03-04T09:09:32.839 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_executing_ops": 0, 2024-03-04T09:09:32.839 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_executing_ops_high_water": 0, 2024-03-04T09:09:32.839 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_executing": 0, 2024-03-04T09:09:32.839 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_executing_high_water": 0, 2024-03-04T09:09:32.839 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_executed_ops": 0, 2024-03-04T09:09:32.840 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_executed": 0, 2024-03-04T09:09:32.840 INFO:teuthology.orchestra.run.smithi158.stdout: "pq_item_in_journal": 0 2024-03-04T09:09:32.840 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.840 INFO:teuthology.orchestra.run.smithi158.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-04T09:09:32.840 INFO:teuthology.orchestra.run.smithi158.stdout: "val": 0, 2024-03-04T09:09:32.840 INFO:teuthology.orchestra.run.smithi158.stdout: "max": 104857600, 2024-03-04T09:09:32.840 INFO:teuthology.orchestra.run.smithi158.stdout: "get_started": 0, 2024-03-04T09:09:32.840 INFO:teuthology.orchestra.run.smithi158.stdout: "get": 3459, 2024-03-04T09:09:32.841 INFO:teuthology.orchestra.run.smithi158.stdout: "get_sum": 547873577, 2024-03-04T09:09:32.841 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_fail": 0, 2024-03-04T09:09:32.841 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_success": 3459, 2024-03-04T09:09:32.841 INFO:teuthology.orchestra.run.smithi158.stdout: "take": 0, 2024-03-04T09:09:32.841 INFO:teuthology.orchestra.run.smithi158.stdout: "take_sum": 0, 2024-03-04T09:09:32.841 INFO:teuthology.orchestra.run.smithi158.stdout: "put": 3459, 2024-03-04T09:09:32.841 INFO:teuthology.orchestra.run.smithi158.stdout: "put_sum": 547873577, 2024-03-04T09:09:32.841 INFO:teuthology.orchestra.run.smithi158.stdout: "wait": { 2024-03-04T09:09:32.841 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.842 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.842 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.842 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-03-04T09:09:32.842 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.842 INFO:teuthology.orchestra.run.smithi158.stdout: "throttle-objecter_bytes": { 2024-03-04T09:09:32.842 INFO:teuthology.orchestra.run.smithi158.stdout: "val": 0, 2024-03-04T09:09:32.842 INFO:teuthology.orchestra.run.smithi158.stdout: "max": 104857600, 2024-03-04T09:09:32.842 INFO:teuthology.orchestra.run.smithi158.stdout: "get_started": 0, 2024-03-04T09:09:32.843 INFO:teuthology.orchestra.run.smithi158.stdout: "get": 0, 2024-03-04T09:09:32.843 INFO:teuthology.orchestra.run.smithi158.stdout: "get_sum": 0, 2024-03-04T09:09:32.843 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_fail": 0, 2024-03-04T09:09:32.843 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_success": 0, 2024-03-04T09:09:32.843 INFO:teuthology.orchestra.run.smithi158.stdout: "take": 3212, 2024-03-04T09:09:32.843 INFO:teuthology.orchestra.run.smithi158.stdout: "take_sum": 545859059, 2024-03-04T09:09:32.843 INFO:teuthology.orchestra.run.smithi158.stdout: "put": 3212, 2024-03-04T09:09:32.843 INFO:teuthology.orchestra.run.smithi158.stdout: "put_sum": 545859059, 2024-03-04T09:09:32.844 INFO:teuthology.orchestra.run.smithi158.stdout: "wait": { 2024-03-04T09:09:32.844 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.844 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.844 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.844 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-03-04T09:09:32.844 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.844 INFO:teuthology.orchestra.run.smithi158.stdout: "throttle-objecter_ops": { 2024-03-04T09:09:32.844 INFO:teuthology.orchestra.run.smithi158.stdout: "val": 0, 2024-03-04T09:09:32.845 INFO:teuthology.orchestra.run.smithi158.stdout: "max": 1024, 2024-03-04T09:09:32.845 INFO:teuthology.orchestra.run.smithi158.stdout: "get_started": 0, 2024-03-04T09:09:32.845 INFO:teuthology.orchestra.run.smithi158.stdout: "get": 0, 2024-03-04T09:09:32.845 INFO:teuthology.orchestra.run.smithi158.stdout: "get_sum": 0, 2024-03-04T09:09:32.845 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_fail": 0, 2024-03-04T09:09:32.845 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_success": 0, 2024-03-04T09:09:32.846 INFO:teuthology.orchestra.run.smithi158.stdout: "take": 3212, 2024-03-04T09:09:32.846 INFO:teuthology.orchestra.run.smithi158.stdout: "take_sum": 3212, 2024-03-04T09:09:32.846 INFO:teuthology.orchestra.run.smithi158.stdout: "put": 3212, 2024-03-04T09:09:32.846 INFO:teuthology.orchestra.run.smithi158.stdout: "put_sum": 3212, 2024-03-04T09:09:32.846 INFO:teuthology.orchestra.run.smithi158.stdout: "wait": { 2024-03-04T09:09:32.846 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.846 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.846 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.846 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-03-04T09:09:32.847 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.847 INFO:teuthology.orchestra.run.smithi158.stdout: "throttle-write_buf_throttle": { 2024-03-04T09:09:32.847 INFO:teuthology.orchestra.run.smithi158.stdout: "val": 0, 2024-03-04T09:09:32.847 INFO:teuthology.orchestra.run.smithi158.stdout: "max": 3758096384, 2024-03-04T09:09:32.847 INFO:teuthology.orchestra.run.smithi158.stdout: "get_started": 0, 2024-03-04T09:09:32.847 INFO:teuthology.orchestra.run.smithi158.stdout: "get": 0, 2024-03-04T09:09:32.847 INFO:teuthology.orchestra.run.smithi158.stdout: "get_sum": 0, 2024-03-04T09:09:32.848 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_fail": 0, 2024-03-04T09:09:32.848 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_success": 0, 2024-03-04T09:09:32.848 INFO:teuthology.orchestra.run.smithi158.stdout: "take": 0, 2024-03-04T09:09:32.848 INFO:teuthology.orchestra.run.smithi158.stdout: "take_sum": 0, 2024-03-04T09:09:32.848 INFO:teuthology.orchestra.run.smithi158.stdout: "put": 0, 2024-03-04T09:09:32.848 INFO:teuthology.orchestra.run.smithi158.stdout: "put_sum": 0, 2024-03-04T09:09:32.849 INFO:teuthology.orchestra.run.smithi158.stdout: "wait": { 2024-03-04T09:09:32.849 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.849 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.849 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.849 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-03-04T09:09:32.849 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:32.849 INFO:teuthology.orchestra.run.smithi158.stdout: "throttle-write_buf_throttle-0x555eb6a2e960": { 2024-03-04T09:09:32.850 INFO:teuthology.orchestra.run.smithi158.stdout: "val": 0, 2024-03-04T09:09:32.850 INFO:teuthology.orchestra.run.smithi158.stdout: "max": 3758096384, 2024-03-04T09:09:32.850 INFO:teuthology.orchestra.run.smithi158.stdout: "get_started": 0, 2024-03-04T09:09:32.850 INFO:teuthology.orchestra.run.smithi158.stdout: "get": 0, 2024-03-04T09:09:32.851 INFO:teuthology.orchestra.run.smithi158.stdout: "get_sum": 0, 2024-03-04T09:09:32.851 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_fail": 0, 2024-03-04T09:09:32.851 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_success": 0, 2024-03-04T09:09:32.851 INFO:teuthology.orchestra.run.smithi158.stdout: "take": 0, 2024-03-04T09:09:32.851 INFO:teuthology.orchestra.run.smithi158.stdout: "take_sum": 0, 2024-03-04T09:09:32.851 INFO:teuthology.orchestra.run.smithi158.stdout: "put": 0, 2024-03-04T09:09:32.852 INFO:teuthology.orchestra.run.smithi158.stdout: "put_sum": 0, 2024-03-04T09:09:32.852 INFO:teuthology.orchestra.run.smithi158.stdout: "wait": { 2024-03-04T09:09:32.852 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:32.852 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:32.852 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:32.853 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-03-04T09:09:32.853 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-03-04T09:09:32.853 INFO:teuthology.orchestra.run.smithi158.stdout:} 2024-03-04T09:09:33.560 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-03-04T09:09:33.560 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-03-04T09:09:33.560 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-03-04T09:09:33.560 DEBUG:tasks.check_counter:Getting stats from l 2024-03-04T09:09:33.560 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid b3c85c40-da03-11ee-95c6-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-04T09:09:33.798 INFO:teuthology.orchestra.run.smithi158.stderr:Inferring config /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T09:09:34.239 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:34 smithi158 ceph-mon[37241]: pgmap v330: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:09:34.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:34 smithi071 ceph-mon[32789]: pgmap v330: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:09:34.529 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:34 smithi150 ceph-mon[36807]: pgmap v330: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:09:35.524 INFO:teuthology.orchestra.run.smithi158.stdout:{ 2024-03-04T09:09:35.524 INFO:teuthology.orchestra.run.smithi158.stdout: "AsyncMessenger::Worker-0": { 2024-03-04T09:09:35.524 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_messages": 230, 2024-03-04T09:09:35.524 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages": 220, 2024-03-04T09:09:35.524 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_bytes": 1434432, 2024-03-04T09:09:35.525 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_bytes": 132272, 2024-03-04T09:09:35.525 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_created_connections": 4, 2024-03-04T09:09:35.525 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_active_connections": 0, 2024-03-04T09:09:35.525 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_total_time": 0.046272685, 2024-03-04T09:09:35.525 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_send_time": 0.013256948, 2024-03-04T09:09:35.525 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_recv_time": 0.035186413, 2024-03-04T09:09:35.525 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_fast_dispatch_time": 0.005335986, 2024-03-04T09:09:35.525 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:35.526 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 219, 2024-03-04T09:09:35.526 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.012068337, 2024-03-04T09:09:35.526 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000055106 2024-03-04T09:09:35.526 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:35.526 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:35.526 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:35.526 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:35.526 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:35.527 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:35.527 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_encrypted_bytes": 1434432, 2024-03-04T09:09:35.527 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_encrypted_bytes": 132272 2024-03-04T09:09:35.527 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:35.527 INFO:teuthology.orchestra.run.smithi158.stdout: "AsyncMessenger::Worker-1": { 2024-03-04T09:09:35.527 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_messages": 6, 2024-03-04T09:09:35.527 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages": 170, 2024-03-04T09:09:35.527 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_bytes": 3312, 2024-03-04T09:09:35.528 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_bytes": 184944, 2024-03-04T09:09:35.528 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_created_connections": 5, 2024-03-04T09:09:35.528 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_active_connections": 3, 2024-03-04T09:09:35.528 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_total_time": 0.011270492, 2024-03-04T09:09:35.528 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_send_time": 0.007374036, 2024-03-04T09:09:35.528 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_recv_time": 0.001752280, 2024-03-04T09:09:35.528 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:09:35.528 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:35.528 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 168, 2024-03-04T09:09:35.529 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.012865129, 2024-03-04T09:09:35.529 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000076578 2024-03-04T09:09:35.529 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:35.529 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:35.529 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:35.529 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:35.529 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:35.529 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:35.530 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-03-04T09:09:35.530 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_encrypted_bytes": 184944 2024-03-04T09:09:35.530 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:35.530 INFO:teuthology.orchestra.run.smithi158.stdout: "AsyncMessenger::Worker-2": { 2024-03-04T09:09:35.530 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_messages": 0, 2024-03-04T09:09:35.530 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages": 2, 2024-03-04T09:09:35.530 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_bytes": 0, 2024-03-04T09:09:35.530 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_bytes": 0, 2024-03-04T09:09:35.531 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_created_connections": 2, 2024-03-04T09:09:35.531 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_active_connections": 1, 2024-03-04T09:09:35.531 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_total_time": 0.000382201, 2024-03-04T09:09:35.531 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_send_time": 0.000000000, 2024-03-04T09:09:35.531 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_recv_time": 0.000310362, 2024-03-04T09:09:35.531 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:09:35.531 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:09:35.531 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:35.532 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:35.532 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:35.532 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:35.532 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:09:35.532 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:35.532 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:35.532 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:35.532 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:35.533 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-04T09:09:35.533 INFO:teuthology.orchestra.run.smithi158.stdout: "msgr_send_encrypted_bytes": 0 2024-03-04T09:09:35.533 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:35.533 INFO:teuthology.orchestra.run.smithi158.stdout: "cct": { 2024-03-04T09:09:35.533 INFO:teuthology.orchestra.run.smithi158.stdout: "total_workers": 0, 2024-03-04T09:09:35.533 INFO:teuthology.orchestra.run.smithi158.stdout: "unhealthy_workers": 0 2024-03-04T09:09:35.533 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:35.533 INFO:teuthology.orchestra.run.smithi158.stdout: "mempool": { 2024-03-04T09:09:35.533 INFO:teuthology.orchestra.run.smithi158.stdout: "bloom_filter_bytes": 0, 2024-03-04T09:09:35.534 INFO:teuthology.orchestra.run.smithi158.stdout: "bloom_filter_items": 0, 2024-03-04T09:09:35.534 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_alloc_bytes": 0, 2024-03-04T09:09:35.534 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_alloc_items": 0, 2024-03-04T09:09:35.534 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_data_bytes": 0, 2024-03-04T09:09:35.534 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_data_items": 0, 2024-03-04T09:09:35.534 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-04T09:09:35.534 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_onode_items": 0, 2024-03-04T09:09:35.534 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-04T09:09:35.535 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_meta_items": 0, 2024-03-04T09:09:35.535 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_other_bytes": 0, 2024-03-04T09:09:35.535 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_other_items": 0, 2024-03-04T09:09:35.535 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-04T09:09:35.535 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_cache_buffer_items": 0, 2024-03-04T09:09:35.535 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_extent_bytes": 0, 2024-03-04T09:09:35.535 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_extent_items": 0, 2024-03-04T09:09:35.535 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_blob_bytes": 0, 2024-03-04T09:09:35.536 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_blob_items": 0, 2024-03-04T09:09:35.536 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-04T09:09:35.536 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_shared_blob_items": 0, 2024-03-04T09:09:35.536 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-04T09:09:35.536 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_inline_bl_items": 0, 2024-03-04T09:09:35.536 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_fsck_bytes": 0, 2024-03-04T09:09:35.536 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_fsck_items": 0, 2024-03-04T09:09:35.536 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_txc_bytes": 0, 2024-03-04T09:09:35.537 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_txc_items": 0, 2024-03-04T09:09:35.537 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-04T09:09:35.537 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_deferred_items": 0, 2024-03-04T09:09:35.537 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_bytes": 0, 2024-03-04T09:09:35.537 INFO:teuthology.orchestra.run.smithi158.stdout: "bluestore_writing_items": 0, 2024-03-04T09:09:35.537 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_bytes": 0, 2024-03-04T09:09:35.537 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_items": 0, 2024-03-04T09:09:35.537 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_reader_bytes": 0, 2024-03-04T09:09:35.538 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_reader_items": 0, 2024-03-04T09:09:35.538 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_writer_bytes": 0, 2024-03-04T09:09:35.538 INFO:teuthology.orchestra.run.smithi158.stdout: "bluefs_file_writer_items": 0, 2024-03-04T09:09:35.538 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_anon_bytes": 148744, 2024-03-04T09:09:35.538 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_anon_items": 30, 2024-03-04T09:09:35.538 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_meta_bytes": 0, 2024-03-04T09:09:35.538 INFO:teuthology.orchestra.run.smithi158.stdout: "buffer_meta_items": 0, 2024-03-04T09:09:35.538 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_bytes": 0, 2024-03-04T09:09:35.539 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_items": 0, 2024-03-04T09:09:35.539 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_mapbl_bytes": 0, 2024-03-04T09:09:35.539 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_mapbl_items": 0, 2024-03-04T09:09:35.539 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_pglog_bytes": 0, 2024-03-04T09:09:35.539 INFO:teuthology.orchestra.run.smithi158.stdout: "osd_pglog_items": 0, 2024-03-04T09:09:35.539 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_bytes": 0, 2024-03-04T09:09:35.539 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_items": 0, 2024-03-04T09:09:35.539 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_mapping_bytes": 0, 2024-03-04T09:09:35.539 INFO:teuthology.orchestra.run.smithi158.stdout: "osdmap_mapping_items": 0, 2024-03-04T09:09:35.540 INFO:teuthology.orchestra.run.smithi158.stdout: "pgmap_bytes": 0, 2024-03-04T09:09:35.540 INFO:teuthology.orchestra.run.smithi158.stdout: "pgmap_items": 0, 2024-03-04T09:09:35.540 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_co_bytes": 928, 2024-03-04T09:09:35.540 INFO:teuthology.orchestra.run.smithi158.stdout: "mds_co_items": 2, 2024-03-04T09:09:35.540 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_1_bytes": 0, 2024-03-04T09:09:35.540 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_1_items": 0, 2024-03-04T09:09:35.540 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_2_bytes": 0, 2024-03-04T09:09:35.541 INFO:teuthology.orchestra.run.smithi158.stdout: "unittest_2_items": 0 2024-03-04T09:09:35.541 INFO:teuthology.orchestra.run.smithi158.stdout: }, 2024-03-04T09:09:35.541 INFO:teuthology.orchestra.run.smithi158.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-04T09:09:35.541 INFO:teuthology.orchestra.run.smithi158.stdout: "val": 0, 2024-03-04T09:09:35.541 INFO:teuthology.orchestra.run.smithi158.stdout: "max": 104857600, 2024-03-04T09:09:35.541 INFO:teuthology.orchestra.run.smithi158.stdout: "get_started": 0, 2024-03-04T09:09:35.542 INFO:teuthology.orchestra.run.smithi158.stdout: "get": 232, 2024-03-04T09:09:35.542 INFO:teuthology.orchestra.run.smithi158.stdout: "get_sum": 1401976, 2024-03-04T09:09:35.542 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_fail": 0, 2024-03-04T09:09:35.542 INFO:teuthology.orchestra.run.smithi158.stdout: "get_or_fail_success": 232, 2024-03-04T09:09:35.542 INFO:teuthology.orchestra.run.smithi158.stdout: "take": 0, 2024-03-04T09:09:35.542 INFO:teuthology.orchestra.run.smithi158.stdout: "take_sum": 0, 2024-03-04T09:09:35.542 INFO:teuthology.orchestra.run.smithi158.stdout: "put": 232, 2024-03-04T09:09:35.542 INFO:teuthology.orchestra.run.smithi158.stdout: "put_sum": 1401976, 2024-03-04T09:09:35.543 INFO:teuthology.orchestra.run.smithi158.stdout: "wait": { 2024-03-04T09:09:35.543 INFO:teuthology.orchestra.run.smithi158.stdout: "avgcount": 0, 2024-03-04T09:09:35.543 INFO:teuthology.orchestra.run.smithi158.stdout: "sum": 0.000000000, 2024-03-04T09:09:35.543 INFO:teuthology.orchestra.run.smithi158.stdout: "avgtime": 0.000000000 2024-03-04T09:09:35.543 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-03-04T09:09:35.543 INFO:teuthology.orchestra.run.smithi158.stdout: } 2024-03-04T09:09:35.543 INFO:teuthology.orchestra.run.smithi158.stdout:} 2024-03-04T09:09:36.182 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:36 smithi158 ceph-mon[37241]: pgmap v331: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-03-04T09:09:36.209 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-03-04T09:09:36.210 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-03-04T09:09:36.210 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-03-04T09:09:36.210 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-04T09:09:36.220 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-04T09:09:36.220 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-04T09:09:36.220 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T09:09:36.241 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-03-04T09:09:36.241 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T09:09:36.300 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T09:09:36.368 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:36 smithi071 ceph-mon[32789]: pgmap v331: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-03-04T09:09:36.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.457+0000 7ff11fb9a640 1 -- 172.21.15.71:0/920772836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1181001d0 msgr2=0x7ff1181005b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:36.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.457+0000 7ff11fb9a640 1 --2- 172.21.15.71:0/920772836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1181001d0 0x7ff1181005b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff114009900 tx=0x7ff11402bc30 comp rx=0 tx=0).stop 2024-03-04T09:09:36.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.457+0000 7ff11fb9a640 1 -- 172.21.15.71:0/920772836 shutdown_connections 2024-03-04T09:09:36.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.457+0000 7ff11fb9a640 1 --2- 172.21.15.71:0/920772836 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff118106990 0x7ff11810e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:36.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.457+0000 7ff11fb9a640 1 --2- 172.21.15.71:0/920772836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff118100b80 0x7ff1181062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:36.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.457+0000 7ff11fb9a640 1 --2- 172.21.15.71:0/920772836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1181001d0 0x7ff1181005b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:36.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.457+0000 7ff11fb9a640 1 -- 172.21.15.71:0/920772836 >> 172.21.15.71:0/920772836 conn(0x7ff1180757e0 msgr2=0x7ff118075bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:36.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.457+0000 7ff11fb9a640 1 -- 172.21.15.71:0/920772836 shutdown_connections 2024-03-04T09:09:36.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.458+0000 7ff11fb9a640 1 -- 172.21.15.71:0/920772836 wait complete. 2024-03-04T09:09:36.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.458+0000 7ff11fb9a640 1 Processor -- start 2024-03-04T09:09:36.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.458+0000 7ff11fb9a640 1 -- start start 2024-03-04T09:09:36.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.458+0000 7ff11fb9a640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff1181001d0 0x7ff11819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:36.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.458+0000 7ff11fb9a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff118106990 0x7ff11819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:36.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.458+0000 7ff11fb9a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff11819ea60 0x7ff11819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:36.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.458+0000 7ff11fb9a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff118111a80 con 0x7ff11819ea60 2024-03-04T09:09:36.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.458+0000 7ff11fb9a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff118111900 con 0x7ff118106990 2024-03-04T09:09:36.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.458+0000 7ff11fb9a640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff118111c00 con 0x7ff1181001d0 2024-03-04T09:09:36.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.458+0000 7ff11e137640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff11819ea60 0x7ff11819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:36.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.458+0000 7ff11e137640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff11819ea60 0x7ff11819ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41062/0 (socket says 172.21.15.71:41062) 2024-03-04T09:09:36.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.458+0000 7ff11e137640 1 -- 172.21.15.71:0/220237229 learned_addr learned my addr 172.21.15.71:0/220237229 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T09:09:36.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.459+0000 7ff11d135640 1 --2- 172.21.15.71:0/220237229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff118106990 0x7ff11819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:36.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.459+0000 7ff11d936640 1 --2- 172.21.15.71:0/220237229 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff1181001d0 0x7ff11819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:36.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.459+0000 7ff11e137640 1 -- 172.21.15.71:0/220237229 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff1181001d0 msgr2=0x7ff11819a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:36.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.459+0000 7ff11e137640 1 --2- 172.21.15.71:0/220237229 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff1181001d0 0x7ff11819a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:36.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.459+0000 7ff11e137640 1 -- 172.21.15.71:0/220237229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff118106990 msgr2=0x7ff11819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:36.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.459+0000 7ff11e137640 1 --2- 172.21.15.71:0/220237229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff118106990 0x7ff11819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:36.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.459+0000 7ff11e137640 1 -- 172.21.15.71:0/220237229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11819f550 con 0x7ff11819ea60 2024-03-04T09:09:36.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.459+0000 7ff11e137640 1 --2- 172.21.15.71:0/220237229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff11819ea60 0x7ff11819ef10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff11000eb10 tx=0x7ff11000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:36.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.459+0000 7ff10affd640 1 -- 172.21.15.71:0/220237229 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff110004b20 con 0x7ff11819ea60 2024-03-04T09:09:36.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.459+0000 7ff10affd640 1 -- 172.21.15.71:0/220237229 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff110004cc0 con 0x7ff11819ea60 2024-03-04T09:09:36.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.459+0000 7ff11fb9a640 1 -- 172.21.15.71:0/220237229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff11819f840 con 0x7ff11819ea60 2024-03-04T09:09:36.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.459+0000 7ff10affd640 1 -- 172.21.15.71:0/220237229 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff110010760 con 0x7ff11819ea60 2024-03-04T09:09:36.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.459+0000 7ff11fb9a640 1 -- 172.21.15.71:0/220237229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1181a7030 con 0x7ff11819ea60 2024-03-04T09:09:36.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.460+0000 7ff11fb9a640 1 -- 172.21.15.71:0/220237229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff118069700 con 0x7ff11819ea60 2024-03-04T09:09:36.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.461+0000 7ff10affd640 1 -- 172.21.15.71:0/220237229 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7ff110005000 con 0x7ff11819ea60 2024-03-04T09:09:36.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.462+0000 7ff10affd640 1 --2- 172.21.15.71:0/220237229 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7ff1040a1ef0 0x7ff1040a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:36.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.462+0000 7ff11d936640 1 --2- 172.21.15.71:0/220237229 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7ff1040a1ef0 0x7ff1040a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:36.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.462+0000 7ff10affd640 1 -- 172.21.15.71:0/220237229 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9467+0+0 (secure 0 0 0) 0x7ff1100cefe0 con 0x7ff11819ea60 2024-03-04T09:09:36.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.462+0000 7ff11d936640 1 --2- 172.21.15.71:0/220237229 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7ff1040a1ef0 0x7ff1040a43b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff114006010 tx=0x7ff114000bc0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:36.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.466+0000 7ff10affd640 1 -- 172.21.15.71:0/220237229 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff11009bd20 con 0x7ff11819ea60 2024-03-04T09:09:36.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:36 smithi150 ceph-mon[36807]: pgmap v331: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-03-04T09:09:36.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.639+0000 7ff11fb9a640 1 -- 172.21.15.71:0/220237229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff118068810 con 0x7ff11819ea60 2024-03-04T09:09:36.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.641+0000 7ff10affd640 1 -- 172.21.15.71:0/220237229 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+25291 (secure 0 0 0) 0x7ff11009b620 con 0x7ff11819ea60 2024-03-04T09:09:36.638 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T09:09:36.638 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":104,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","created":"2024-03-04T08:47:28.546102+0000","modified":"2024-03-04T09:09:14.702148+0000","last_up_change":"2024-03-04T08:55:30.744961+0000","last_in_change":"2024-03-04T08:55:11.570990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T08:51:14.933487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T08:56:28.253074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T08:56:29.543622+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":16,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T08:56:32.884220+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":16,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7e43b47d-8888-4631-bc96-8a96fcf8cd46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6803","nonce":1173641530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6805","nonce":1173641530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6809","nonce":1173641530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6807","nonce":1173641530}]},"public_addr":"172.21.15.71:6803/1173641530","cluster_addr":"172.21.15.71:6805/1173641530","heartbeat_back_addr":"172.21.15.71:6809/1173641530","heartbeat_front_addr":"172.21.15.71:6807/1173641530","state":["exists","up"]},{"osd":1,"uuid":"1209f214-f2e2-4209-bf69-4528a43a2123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6801","nonce":3383948110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6803","nonce":3383948110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6807","nonce":3383948110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6805","nonce":3383948110}]},"public_addr":"172.21.15.150:6801/3383948110","cluster_addr":"172.21.15.150:6803/3383948110","heartbeat_back_addr":"172.21.15.150:6807/3383948110","heartbeat_front_addr":"172.21.15.150:6805/3383948110","state":["exists","up"]},{"osd":2,"uuid":"b88c3082-93a9-4c9b-9221-a7105a44c91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6801","nonce":586944144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6803","nonce":586944144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6807","nonce":586944144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6805","nonce":586944144}]},"public_addr":"172.21.15.158:6801/586944144","cluster_addr":"172.21.15.158:6803/586944144","heartbeat_back_addr":"172.21.15.158:6807/586944144","heartbeat_front_addr":"172.21.15.158:6805/586944144","state":["exists","up"]},{"osd":3,"uuid":"e4fc7723-fd73-40b2-b9d0-7f9bda282ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6811","nonce":1311903262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6813","nonce":1311903262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6817","nonce":1311903262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6815","nonce":1311903262}]},"public_addr":"172.21.15.71:6811/1311903262","cluster_addr":"172.21.15.71:6813/1311903262","heartbeat_back_addr":"172.21.15.71:6817/1311903262","heartbeat_front_addr":"172.21.15.71:6815/1311903262","state":["exists","up"]},{"osd":4,"uuid":"e7eb72ac-f066-4ccd-b2bf-22a6a26daf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6809","nonce":2300837573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6811","nonce":2300837573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6815","nonce":2300837573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6813","nonce":2300837573}]},"public_addr":"172.21.15.150:6809/2300837573","cluster_addr":"172.21.15.150:6811/2300837573","heartbeat_back_addr":"172.21.15.150:6815/2300837573","heartbeat_front_addr":"172.21.15.150:6813/2300837573","state":["exists","up"]},{"osd":5,"uuid":"9b295840-e104-4a66-9cc4-7020627c1765","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6809","nonce":1922694045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6811","nonce":1922694045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6815","nonce":1922694045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6813","nonce":1922694045}]},"public_addr":"172.21.15.158:6809/1922694045","cluster_addr":"172.21.15.158:6811/1922694045","heartbeat_back_addr":"172.21.15.158:6815/1922694045","heartbeat_front_addr":"172.21.15.158:6813/1922694045","state":["exists","up"]},{"osd":6,"uuid":"6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6819","nonce":3848842707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6821","nonce":3848842707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6825","nonce":3848842707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6823","nonce":3848842707}]},"public_addr":"172.21.15.71:6819/3848842707","cluster_addr":"172.21.15.71:6821/3848842707","heartbeat_back_addr":"172.21.15.71:6825/3848842707","heartbeat_front_addr":"172.21.15.71:6823/3848842707","state":["exists","up"]},{"osd":7,"uuid":"6160a40a-d30f-4569-949c-0afb1bb3d133","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6817","nonce":1328330305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6819","nonce":1328330305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6823","nonce":1328330305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6821","nonce":1328330305}]},"public_addr":"172.21.15.150:6817/1328330305","cluster_addr":"172.21.15.150:6819/1328330305","heartbeat_back_addr":"172.21.15.150:6823/1328330305","heartbeat_front_addr":"172.21.15.150:6821/1328330305","state":["exists","up"]},{"osd":8,"uuid":"95ed2341-84a4-465c-9948-ac667680ce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6817","nonce":2195304577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6819","nonce":2195304577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6823","nonce":2195304577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6821","nonce":2195304577}]},"public_addr":"172.21.15.158:6817/2195304577","cluster_addr":"172.21.15.158:6819/2195304577","heartbeat_back_addr":"172.21.15.158:6823/2195304577","heartbeat_front_addr":"172.21.15.158:6821/2195304577","state":["exists","up"]},{"osd":9,"uuid":"088f6419-63a0-4768-afae-dbb8a5f5a565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6827","nonce":1913779347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6829","nonce":1913779347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6833","nonce":1913779347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6831","nonce":1913779347}]},"public_addr":"172.21.15.71:6827/1913779347","cluster_addr":"172.21.15.71:6829/1913779347","heartbeat_back_addr":"172.21.15.71:6833/1913779347","heartbeat_front_addr":"172.21.15.71:6831/1913779347","state":["exists","up"]},{"osd":10,"uuid":"18c85e24-068d-4f1e-a3b8-f7915b363234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6825","nonce":3295420996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6827","nonce":3295420996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6831","nonce":3295420996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6829","nonce":3295420996}]},"public_addr":"172.21.15.150:6825/3295420996","cluster_addr":"172.21.15.150:6827/3295420996","heartbeat_back_addr":"172.21.15.150:6831/3295420996","heartbeat_front_addr":"172.21.15.150:6829/3295420996","state":["exists","up"]},{"osd":11,"uuid":"4f6b3ddf-c482-4740-b1fd-de836e047c2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6825","nonce":3332083865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6827","nonce":3332083865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6831","nonce":3332083865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6829","nonce":3332083865}]},"public_addr":"172.21.15.158:6825/3332083865","cluster_addr":"172.21.15.158:6827/3332083865","heartbeat_back_addr":"172.21.15.158:6831/3332083865","heartbeat_front_addr":"172.21.15.158:6829/3332083865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:05.997832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:29.314235+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:08.819465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:41.194275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:06.032479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:38.814498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:13.067960+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:38.568479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:03.416713+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:38.349479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:04.096945+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:29.425250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/2792362717":"2024-03-05T08:58:54.820040+0000","172.21.15.71:0/1289352375":"2024-03-05T08:58:54.820040+0000","172.21.15.71:6801/2901868399":"2024-03-05T08:58:54.820040+0000","172.21.15.71:6800/2901868399":"2024-03-05T08:58:54.820040+0000","172.21.15.71:0/1087047482":"2024-03-05T08:58:54.820040+0000","172.21.15.71:0/4186644246":"2024-03-05T08:58:54.820040+0000","172.21.15.71:0/1287712358":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/3770452860":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/718743094":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/1904717532":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:6800/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6800/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/2862184313":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/3621754630":"2024-03-05T08:48:22.593931+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:09:36.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.643+0000 7ff11fb9a640 1 -- 172.21.15.71:0/220237229 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7ff1040a1ef0 msgr2=0x7ff1040a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:36.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.643+0000 7ff11fb9a640 1 --2- 172.21.15.71:0/220237229 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7ff1040a1ef0 0x7ff1040a43b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff114006010 tx=0x7ff114000bc0 comp rx=0 tx=0).stop 2024-03-04T09:09:36.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.643+0000 7ff11fb9a640 1 -- 172.21.15.71:0/220237229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff11819ea60 msgr2=0x7ff11819ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:36.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.643+0000 7ff11fb9a640 1 --2- 172.21.15.71:0/220237229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff11819ea60 0x7ff11819ef10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff11000eb10 tx=0x7ff11000c2d0 comp rx=0 tx=0).stop 2024-03-04T09:09:36.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.643+0000 7ff11fb9a640 1 -- 172.21.15.71:0/220237229 shutdown_connections 2024-03-04T09:09:36.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.643+0000 7ff11fb9a640 1 --2- 172.21.15.71:0/220237229 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7ff1040a1ef0 0x7ff1040a43b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:36.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.643+0000 7ff11fb9a640 1 --2- 172.21.15.71:0/220237229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff11819ea60 0x7ff11819ef10 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:36.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.643+0000 7ff11fb9a640 1 --2- 172.21.15.71:0/220237229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff118106990 0x7ff11819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:36.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.643+0000 7ff11fb9a640 1 --2- 172.21.15.71:0/220237229 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff1181001d0 0x7ff11819a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:36.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.643+0000 7ff11fb9a640 1 -- 172.21.15.71:0/220237229 >> 172.21.15.71:0/220237229 conn(0x7ff1180757e0 msgr2=0x7ff1180fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:36.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.643+0000 7ff11fb9a640 1 -- 172.21.15.71:0/220237229 shutdown_connections 2024-03-04T09:09:36.641 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.643+0000 7ff11fb9a640 1 -- 172.21.15.71:0/220237229 wait complete. 2024-03-04T09:09:36.650 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T08:51:14.933487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T08:56:28.253074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T08:56:29.543622+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 16, 'snap_epoch': 103, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T08:56:32.884220+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 16, 'snap_epoch': 103, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T09:09:36.651 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T09:09:36.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.799+0000 7f7925cc9640 1 -- 172.21.15.71:0/3255050465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7920101580 msgr2=0x7f7920101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:36.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.799+0000 7f7925cc9640 1 --2- 172.21.15.71:0/3255050465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7920101580 0x7f7920101a00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7908009900 tx=0x7f790802bc30 comp rx=0 tx=0).stop 2024-03-04T09:09:36.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.799+0000 7f7925cc9640 1 -- 172.21.15.71:0/3255050465 shutdown_connections 2024-03-04T09:09:36.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.799+0000 7f7925cc9640 1 --2- 172.21.15.71:0/3255050465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7920101f40 0x7f792010e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:36.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.799+0000 7f7925cc9640 1 --2- 172.21.15.71:0/3255050465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7920101580 0x7f7920101a00 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:36.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.799+0000 7f7925cc9640 1 --2- 172.21.15.71:0/3255050465 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7920107640 0x7f7920107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:36.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.799+0000 7f7925cc9640 1 -- 172.21.15.71:0/3255050465 >> 172.21.15.71:0/3255050465 conn(0x7f79200789d0 msgr2=0x7f7920078de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:36.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.799+0000 7f7925cc9640 1 -- 172.21.15.71:0/3255050465 shutdown_connections 2024-03-04T09:09:36.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.799+0000 7f7925cc9640 1 -- 172.21.15.71:0/3255050465 wait complete. 2024-03-04T09:09:36.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.799+0000 7f7925cc9640 1 Processor -- start 2024-03-04T09:09:36.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.799+0000 7f7925cc9640 1 -- start start 2024-03-04T09:09:36.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.800+0000 7f7925cc9640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7920101580 0x7f7920197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:36.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.800+0000 7f7925cc9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7920101f40 0x7f7920198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:36.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.800+0000 7f7925cc9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7920107640 0x7f792019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:36.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.800+0000 7f7925cc9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7920111980 con 0x7f7920101f40 2024-03-04T09:09:36.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.800+0000 7f7925cc9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7920111800 con 0x7f7920107640 2024-03-04T09:09:36.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.800+0000 7f7925cc9640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7920111b00 con 0x7f7920101580 2024-03-04T09:09:36.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.800+0000 7f791effd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7920101f40 0x7f7920198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:36.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.800+0000 7f791effd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7920101f40 0x7f7920198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41080/0 (socket says 172.21.15.71:41080) 2024-03-04T09:09:36.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.800+0000 7f791effd640 1 -- 172.21.15.71:0/2315374937 learned_addr learned my addr 172.21.15.71:0/2315374937 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T09:09:36.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.800+0000 7f791f7fe640 1 --2- 172.21.15.71:0/2315374937 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7920101580 0x7f7920197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:36.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.800+0000 7f791ffff640 1 --2- 172.21.15.71:0/2315374937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7920107640 0x7f792019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:36.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.800+0000 7f791effd640 1 -- 172.21.15.71:0/2315374937 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7920101580 msgr2=0x7f7920197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:36.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.800+0000 7f791effd640 1 --2- 172.21.15.71:0/2315374937 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7920101580 0x7f7920197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:36.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.800+0000 7f791effd640 1 -- 172.21.15.71:0/2315374937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7920107640 msgr2=0x7f792019c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:36.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.800+0000 7f791effd640 1 --2- 172.21.15.71:0/2315374937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7920107640 0x7f792019c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:36.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.800+0000 7f791effd640 1 -- 172.21.15.71:0/2315374937 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792019ce30 con 0x7f7920101f40 2024-03-04T09:09:36.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.800+0000 7f791effd640 1 --2- 172.21.15.71:0/2315374937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7920101f40 0x7f7920198380 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7908005940 tx=0x7f79080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:36.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.800+0000 7f791cff9640 1 -- 172.21.15.71:0/2315374937 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7908041070 con 0x7f7920101f40 2024-03-04T09:09:36.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.800+0000 7f791cff9640 1 -- 172.21.15.71:0/2315374937 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f790802ccc0 con 0x7f7920101f40 2024-03-04T09:09:36.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.800+0000 7f7925cc9640 1 -- 172.21.15.71:0/2315374937 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f792019d0c0 con 0x7f7920101f40 2024-03-04T09:09:36.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.800+0000 7f791cff9640 1 -- 172.21.15.71:0/2315374937 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7908003d80 con 0x7f7920101f40 2024-03-04T09:09:36.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.800+0000 7f7925cc9640 1 -- 172.21.15.71:0/2315374937 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79201a49b0 con 0x7f7920101f40 2024-03-04T09:09:36.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.801+0000 7f7925cc9640 1 -- 172.21.15.71:0/2315374937 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7920068810 con 0x7f7920101f40 2024-03-04T09:09:36.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.803+0000 7f791cff9640 1 -- 172.21.15.71:0/2315374937 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f7908044070 con 0x7f7920101f40 2024-03-04T09:09:36.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.803+0000 7f791cff9640 1 --2- 172.21.15.71:0/2315374937 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f790c0a1f70 0x7f790c0a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:36.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.804+0000 7f791f7fe640 1 --2- 172.21.15.71:0/2315374937 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f790c0a1f70 0x7f790c0a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:36.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.804+0000 7f791cff9640 1 -- 172.21.15.71:0/2315374937 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9467+0+0 (secure 0 0 0) 0x7f79080f01c0 con 0x7f7920101f40 2024-03-04T09:09:36.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.804+0000 7f791f7fe640 1 --2- 172.21.15.71:0/2315374937 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f790c0a1f70 0x7f790c0a4430 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f791000a400 tx=0x7f791000d040 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:36.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.807+0000 7f791cff9640 1 -- 172.21.15.71:0/2315374937 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f79080bc9c0 con 0x7f7920101f40 2024-03-04T09:09:36.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.981+0000 7f7925cc9640 1 -- 172.21.15.71:0/2315374937 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f79201094a0 con 0x7f7920101f40 2024-03-04T09:09:36.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.982+0000 7f791cff9640 1 -- 172.21.15.71:0/2315374937 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) v1 ==== 93+0+10 (secure 0 0 0) 0x7f79080c0a60 con 0x7f7920101f40 2024-03-04T09:09:36.978 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-03-04T09:09:36.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.984+0000 7f7925cc9640 1 -- 172.21.15.71:0/2315374937 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f790c0a1f70 msgr2=0x7f790c0a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:36.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.984+0000 7f7925cc9640 1 --2- 172.21.15.71:0/2315374937 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f790c0a1f70 0x7f790c0a4430 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f791000a400 tx=0x7f791000d040 comp rx=0 tx=0).stop 2024-03-04T09:09:36.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.984+0000 7f7925cc9640 1 -- 172.21.15.71:0/2315374937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7920101f40 msgr2=0x7f7920198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:36.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.984+0000 7f7925cc9640 1 --2- 172.21.15.71:0/2315374937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7920101f40 0x7f7920198380 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7908005940 tx=0x7f79080042f0 comp rx=0 tx=0).stop 2024-03-04T09:09:36.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.984+0000 7f7925cc9640 1 -- 172.21.15.71:0/2315374937 shutdown_connections 2024-03-04T09:09:36.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.984+0000 7f7925cc9640 1 --2- 172.21.15.71:0/2315374937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7920107640 0x7f792019c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:36.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.984+0000 7f7925cc9640 1 --2- 172.21.15.71:0/2315374937 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f790c0a1f70 0x7f790c0a4430 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:36.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.984+0000 7f7925cc9640 1 --2- 172.21.15.71:0/2315374937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7920101f40 0x7f7920198380 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:36.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.984+0000 7f7925cc9640 1 --2- 172.21.15.71:0/2315374937 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7920101580 0x7f7920197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:36.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.984+0000 7f7925cc9640 1 -- 172.21.15.71:0/2315374937 >> 172.21.15.71:0/2315374937 conn(0x7f79200789d0 msgr2=0x7f7920100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:36.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.984+0000 7f7925cc9640 1 -- 172.21.15.71:0/2315374937 shutdown_connections 2024-03-04T09:09:36.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:36.984+0000 7f7925cc9640 1 -- 172.21.15.71:0/2315374937 wait complete. 2024-03-04T09:09:36.991 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T09:09:37.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.100+0000 7f779d32b640 1 -- 172.21.15.71:0/3195237455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7798107640 msgr2=0x7f7798107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:37.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.100+0000 7f779d32b640 1 --2- 172.21.15.71:0/3195237455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7798107640 0x7f7798107a20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7788009900 tx=0x7f778802bc30 comp rx=0 tx=0).stop 2024-03-04T09:09:37.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.100+0000 7f779d32b640 1 -- 172.21.15.71:0/3195237455 shutdown_connections 2024-03-04T09:09:37.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.100+0000 7f779d32b640 1 --2- 172.21.15.71:0/3195237455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7798101f40 0x7f779810e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.100+0000 7f779d32b640 1 --2- 172.21.15.71:0/3195237455 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7798101580 0x7f7798101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.100+0000 7f779d32b640 1 --2- 172.21.15.71:0/3195237455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7798107640 0x7f7798107a20 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.100+0000 7f779d32b640 1 -- 172.21.15.71:0/3195237455 >> 172.21.15.71:0/3195237455 conn(0x7f77980fd010 msgr2=0x7f77980ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:37.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.100+0000 7f779d32b640 1 -- 172.21.15.71:0/3195237455 shutdown_connections 2024-03-04T09:09:37.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.100+0000 7f779d32b640 1 -- 172.21.15.71:0/3195237455 wait complete. 2024-03-04T09:09:37.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.100+0000 7f779d32b640 1 Processor -- start 2024-03-04T09:09:37.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.100+0000 7f779d32b640 1 -- start start 2024-03-04T09:09:37.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.101+0000 7f779d32b640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7798101580 0x7f7798197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:37.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.101+0000 7f779d32b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7798101f40 0x7f7798198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:37.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.101+0000 7f779d32b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7798107640 0x7f779819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:37.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.101+0000 7f779d32b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7798111900 con 0x7f7798101f40 2024-03-04T09:09:37.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.101+0000 7f779d32b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7798111780 con 0x7f7798107640 2024-03-04T09:09:37.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.101+0000 7f779d32b640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7798111a80 con 0x7f7798101580 2024-03-04T09:09:37.100 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.101+0000 7f77967fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7798101f40 0x7f7798198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:37.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.101+0000 7f77967fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7798101f40 0x7f7798198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41092/0 (socket says 172.21.15.71:41092) 2024-03-04T09:09:37.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.101+0000 7f77967fc640 1 -- 172.21.15.71:0/2879417265 learned_addr learned my addr 172.21.15.71:0/2879417265 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T09:09:37.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.101+0000 7f77977fe640 1 --2- 172.21.15.71:0/2879417265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7798107640 0x7f779819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:37.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.101+0000 7f7796ffd640 1 --2- 172.21.15.71:0/2879417265 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7798101580 0x7f7798197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:37.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.101+0000 7f77967fc640 1 -- 172.21.15.71:0/2879417265 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7798101580 msgr2=0x7f7798197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:37.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.101+0000 7f77967fc640 1 --2- 172.21.15.71:0/2879417265 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7798101580 0x7f7798197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.101+0000 7f77967fc640 1 -- 172.21.15.71:0/2879417265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7798107640 msgr2=0x7f779819c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:37.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.101+0000 7f77967fc640 1 --2- 172.21.15.71:0/2879417265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7798107640 0x7f779819c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.101+0000 7f77967fc640 1 -- 172.21.15.71:0/2879417265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f779819cdf0 con 0x7f7798101f40 2024-03-04T09:09:37.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.101+0000 7f7796ffd640 1 --2- 172.21.15.71:0/2879417265 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7798101580 0x7f7798197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:09:37.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.101+0000 7f77967fc640 1 --2- 172.21.15.71:0/2879417265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7798101f40 0x7f7798198340 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f778c00c920 tx=0x7f778c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:37.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.101+0000 7f777ffff640 1 -- 172.21.15.71:0/2879417265 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c007c70 con 0x7f7798101f40 2024-03-04T09:09:37.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.101+0000 7f779d32b640 1 -- 172.21.15.71:0/2879417265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f779819d0e0 con 0x7f7798101f40 2024-03-04T09:09:37.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.101+0000 7f777ffff640 1 -- 172.21.15.71:0/2879417265 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f778c007e10 con 0x7f7798101f40 2024-03-04T09:09:37.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.101+0000 7f777ffff640 1 -- 172.21.15.71:0/2879417265 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c018890 con 0x7f7798101f40 2024-03-04T09:09:37.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.101+0000 7f779d32b640 1 -- 172.21.15.71:0/2879417265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77981a4970 con 0x7f7798101f40 2024-03-04T09:09:37.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.102+0000 7f779d32b640 1 -- 172.21.15.71:0/2879417265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7798067b30 con 0x7f7798101f40 2024-03-04T09:09:37.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.103+0000 7f777ffff640 1 -- 172.21.15.71:0/2879417265 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f778c018a50 con 0x7f7798101f40 2024-03-04T09:09:37.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.104+0000 7f777ffff640 1 --2- 172.21.15.71:0/2879417265 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f77800a1ef0 0x7f77800a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:37.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.104+0000 7f7796ffd640 1 --2- 172.21.15.71:0/2879417265 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f77800a1ef0 0x7f77800a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:37.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.104+0000 7f777ffff640 1 -- 172.21.15.71:0/2879417265 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9467+0+0 (secure 0 0 0) 0x7f778c0ce920 con 0x7f7798101f40 2024-03-04T09:09:37.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.104+0000 7f7796ffd640 1 --2- 172.21.15.71:0/2879417265 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f77800a1ef0 0x7f77800a43b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7788005ab0 tx=0x7f7788005a20 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:37.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.107+0000 7f777ffff640 1 -- 172.21.15.71:0/2879417265 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f778c014030 con 0x7f7798101f40 2024-03-04T09:09:37.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.279+0000 7f779d32b640 1 -- 172.21.15.71:0/2879417265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f779802cff0 con 0x7f7798101f40 2024-03-04T09:09:37.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.280+0000 7f777ffff640 1 -- 172.21.15.71:0/2879417265 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v104) v1 ==== 104+0+11 (secure 0 0 0) 0x7f778c09b6f0 con 0x7f7798101f40 2024-03-04T09:09:37.276 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-03-04T09:09:37.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.282+0000 7f779d32b640 1 -- 172.21.15.71:0/2879417265 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f77800a1ef0 msgr2=0x7f77800a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:37.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.282+0000 7f779d32b640 1 --2- 172.21.15.71:0/2879417265 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f77800a1ef0 0x7f77800a43b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7788005ab0 tx=0x7f7788005a20 comp rx=0 tx=0).stop 2024-03-04T09:09:37.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.282+0000 7f779d32b640 1 -- 172.21.15.71:0/2879417265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7798101f40 msgr2=0x7f7798198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:37.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.282+0000 7f779d32b640 1 --2- 172.21.15.71:0/2879417265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7798101f40 0x7f7798198340 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f778c00c920 tx=0x7f778c00cde0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.282+0000 7f779d32b640 1 -- 172.21.15.71:0/2879417265 shutdown_connections 2024-03-04T09:09:37.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.282+0000 7f779d32b640 1 --2- 172.21.15.71:0/2879417265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7798107640 0x7f779819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.282+0000 7f779d32b640 1 --2- 172.21.15.71:0/2879417265 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f77800a1ef0 0x7f77800a43b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.282+0000 7f779d32b640 1 --2- 172.21.15.71:0/2879417265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7798101f40 0x7f7798198340 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.282+0000 7f779d32b640 1 --2- 172.21.15.71:0/2879417265 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7798101580 0x7f7798197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.282+0000 7f779d32b640 1 -- 172.21.15.71:0/2879417265 >> 172.21.15.71:0/2879417265 conn(0x7f77980fd010 msgr2=0x7f77980fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:37.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.282+0000 7f779d32b640 1 -- 172.21.15.71:0/2879417265 shutdown_connections 2024-03-04T09:09:37.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.282+0000 7f779d32b640 1 -- 172.21.15.71:0/2879417265 wait complete. 2024-03-04T09:09:37.288 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:37 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/220237229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:09:37.288 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:37 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2315374937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:09:37.289 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T09:09:37.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.397+0000 7f7f17f0c640 1 -- 172.21.15.71:0/1507922765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f10100440 msgr2=0x7f7f101008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:37.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.397+0000 7f7f17f0c640 1 --2- 172.21.15.71:0/1507922765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f10100440 0x7f7f101008c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7f00009930 tx=0x7f7f0002bc30 comp rx=0 tx=0).stop 2024-03-04T09:09:37.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.397+0000 7f7f17f0c640 1 -- 172.21.15.71:0/1507922765 shutdown_connections 2024-03-04T09:09:37.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.397+0000 7f7f17f0c640 1 --2- 172.21.15.71:0/1507922765 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f10100e00 0x7f7f1010d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.397+0000 7f7f17f0c640 1 --2- 172.21.15.71:0/1507922765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f10100440 0x7f7f101008c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.397+0000 7f7f17f0c640 1 --2- 172.21.15.71:0/1507922765 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f101064e0 0x7f7f101068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.397+0000 7f7f17f0c640 1 -- 172.21.15.71:0/1507922765 >> 172.21.15.71:0/1507922765 conn(0x7f7f100775d0 msgr2=0x7f7f100779e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:37.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.397+0000 7f7f17f0c640 1 -- 172.21.15.71:0/1507922765 shutdown_connections 2024-03-04T09:09:37.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.397+0000 7f7f17f0c640 1 -- 172.21.15.71:0/1507922765 wait complete. 2024-03-04T09:09:37.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.397+0000 7f7f17f0c640 1 Processor -- start 2024-03-04T09:09:37.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.397+0000 7f7f17f0c640 1 -- start start 2024-03-04T09:09:37.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.397+0000 7f7f17f0c640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f10100440 0x7f7f10196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:37.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.397+0000 7f7f17f0c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f10100e00 0x7f7f101972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:37.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.397+0000 7f7f17f0c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f101064e0 0x7f7f1019b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:37.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.397+0000 7f7f17f0c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f10110810 con 0x7f7f10100e00 2024-03-04T09:09:37.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.397+0000 7f7f17f0c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f10110690 con 0x7f7f101064e0 2024-03-04T09:09:37.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.397+0000 7f7f17f0c640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f10110990 con 0x7f7f10100440 2024-03-04T09:09:37.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.398+0000 7f7f154a7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f10100e00 0x7f7f101972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:37.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.398+0000 7f7f154a7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f10100e00 0x7f7f101972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41102/0 (socket says 172.21.15.71:41102) 2024-03-04T09:09:37.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.398+0000 7f7f154a7640 1 -- 172.21.15.71:0/4036260046 learned_addr learned my addr 172.21.15.71:0/4036260046 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T09:09:37.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.398+0000 7f7f164a9640 1 --2- 172.21.15.71:0/4036260046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f101064e0 0x7f7f1019b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:37.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.398+0000 7f7f15ca8640 1 --2- 172.21.15.71:0/4036260046 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f10100440 0x7f7f10196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:37.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.398+0000 7f7f154a7640 1 -- 172.21.15.71:0/4036260046 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f10100440 msgr2=0x7f7f10196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:37.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.398+0000 7f7f154a7640 1 --2- 172.21.15.71:0/4036260046 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f10100440 0x7f7f10196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.398+0000 7f7f154a7640 1 -- 172.21.15.71:0/4036260046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f101064e0 msgr2=0x7f7f1019b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:37.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.398+0000 7f7f154a7640 1 --2- 172.21.15.71:0/4036260046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f101064e0 0x7f7f1019b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.398+0000 7f7f154a7640 1 -- 172.21.15.71:0/4036260046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1019bda0 con 0x7f7f10100e00 2024-03-04T09:09:37.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.398+0000 7f7f154a7640 1 --2- 172.21.15.71:0/4036260046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f10100e00 0x7f7f101972f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7f00009230 tx=0x7f7f0002cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:37.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.398+0000 7f7f06ffd640 1 -- 172.21.15.71:0/4036260046 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f00041070 con 0x7f7f10100e00 2024-03-04T09:09:37.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.398+0000 7f7f06ffd640 1 -- 172.21.15.71:0/4036260046 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f00004450 con 0x7f7f10100e00 2024-03-04T09:09:37.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.398+0000 7f7f17f0c640 1 -- 172.21.15.71:0/4036260046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f1019c030 con 0x7f7f10100e00 2024-03-04T09:09:37.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.398+0000 7f7f06ffd640 1 -- 172.21.15.71:0/4036260046 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f00003e50 con 0x7f7f10100e00 2024-03-04T09:09:37.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.398+0000 7f7f17f0c640 1 -- 172.21.15.71:0/4036260046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f101a3920 con 0x7f7f10100e00 2024-03-04T09:09:37.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.399+0000 7f7f17f0c640 1 -- 172.21.15.71:0/4036260046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f100683c0 con 0x7f7f10100e00 2024-03-04T09:09:37.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.400+0000 7f7f06ffd640 1 -- 172.21.15.71:0/4036260046 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f7f00044070 con 0x7f7f10100e00 2024-03-04T09:09:37.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.401+0000 7f7f06ffd640 1 --2- 172.21.15.71:0/4036260046 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f7efc0a2090 0x7f7efc0a4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:37.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.401+0000 7f7f15ca8640 1 --2- 172.21.15.71:0/4036260046 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f7efc0a2090 0x7f7efc0a4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:37.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.401+0000 7f7f06ffd640 1 -- 172.21.15.71:0/4036260046 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9467+0+0 (secure 0 0 0) 0x7f7f000efd00 con 0x7f7f10100e00 2024-03-04T09:09:37.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.401+0000 7f7f15ca8640 1 --2- 172.21.15.71:0/4036260046 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f7efc0a2090 0x7f7efc0a4550 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0095f0 tx=0x7f7f0c005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:37.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.405+0000 7f7f06ffd640 1 -- 172.21.15.71:0/4036260046 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7f000bc500 con 0x7f7f10100e00 2024-03-04T09:09:37.431 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:37 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/220237229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:09:37.431 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:37 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2315374937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:09:37.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:37 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/220237229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:09:37.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:37 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2315374937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:09:37.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.578+0000 7f7f17f0c640 1 -- 172.21.15.71:0/4036260046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7f10108420 con 0x7f7f10100e00 2024-03-04T09:09:37.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.578+0000 7f7f06ffd640 1 -- 172.21.15.71:0/4036260046 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v104) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7f000c05a0 con 0x7f7f10100e00 2024-03-04T09:09:37.575 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2024-03-04T09:09:37.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.580+0000 7f7f17f0c640 1 -- 172.21.15.71:0/4036260046 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f7efc0a2090 msgr2=0x7f7efc0a4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:37.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.580+0000 7f7f17f0c640 1 --2- 172.21.15.71:0/4036260046 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f7efc0a2090 0x7f7efc0a4550 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0095f0 tx=0x7f7f0c005f90 comp rx=0 tx=0).stop 2024-03-04T09:09:37.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.580+0000 7f7f17f0c640 1 -- 172.21.15.71:0/4036260046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f10100e00 msgr2=0x7f7f101972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:37.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.580+0000 7f7f17f0c640 1 --2- 172.21.15.71:0/4036260046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f10100e00 0x7f7f101972f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7f00009230 tx=0x7f7f0002cb30 comp rx=0 tx=0).stop 2024-03-04T09:09:37.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.580+0000 7f7f17f0c640 1 -- 172.21.15.71:0/4036260046 shutdown_connections 2024-03-04T09:09:37.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.580+0000 7f7f17f0c640 1 --2- 172.21.15.71:0/4036260046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f101064e0 0x7f7f1019b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.580+0000 7f7f17f0c640 1 --2- 172.21.15.71:0/4036260046 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f7efc0a2090 0x7f7efc0a4550 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.580+0000 7f7f17f0c640 1 --2- 172.21.15.71:0/4036260046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f10100e00 0x7f7f101972f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.580+0000 7f7f17f0c640 1 --2- 172.21.15.71:0/4036260046 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f7f10100440 0x7f7f10196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.580+0000 7f7f17f0c640 1 -- 172.21.15.71:0/4036260046 >> 172.21.15.71:0/4036260046 conn(0x7f7f100775d0 msgr2=0x7f7f10102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:37.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.581+0000 7f7f17f0c640 1 -- 172.21.15.71:0/4036260046 shutdown_connections 2024-03-04T09:09:37.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.581+0000 7f7f17f0c640 1 -- 172.21.15.71:0/4036260046 wait complete. 2024-03-04T09:09:37.587 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T09:09:37.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.695+0000 7ff6a96af640 1 -- 172.21.15.71:0/3337105471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6a4106680 msgr2=0x7ff6a4106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:37.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.695+0000 7ff6a96af640 1 --2- 172.21.15.71:0/3337105471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6a4106680 0x7ff6a4106a60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff694009900 tx=0x7ff69402bcd0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.695+0000 7ff6a96af640 1 -- 172.21.15.71:0/3337105471 shutdown_connections 2024-03-04T09:09:37.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.695+0000 7ff6a96af640 1 --2- 172.21.15.71:0/3337105471 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6a4101170 0x7ff6a410d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.695+0000 7ff6a96af640 1 --2- 172.21.15.71:0/3337105471 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff6a41007b0 0x7ff6a4100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.695+0000 7ff6a96af640 1 --2- 172.21.15.71:0/3337105471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6a4106680 0x7ff6a4106a60 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.695+0000 7ff6a96af640 1 -- 172.21.15.71:0/3337105471 >> 172.21.15.71:0/3337105471 conn(0x7ff6a40774b0 msgr2=0x7ff6a40778c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:37.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.695+0000 7ff6a96af640 1 -- 172.21.15.71:0/3337105471 shutdown_connections 2024-03-04T09:09:37.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.695+0000 7ff6a96af640 1 -- 172.21.15.71:0/3337105471 wait complete. 2024-03-04T09:09:37.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.695+0000 7ff6a96af640 1 Processor -- start 2024-03-04T09:09:37.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.696+0000 7ff6a96af640 1 -- start start 2024-03-04T09:09:37.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.696+0000 7ff6a96af640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff6a41007b0 0x7ff6a4197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:37.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.696+0000 7ff6a96af640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6a4101170 0x7ff6a41983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:37.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.696+0000 7ff6a96af640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6a4106680 0x7ff6a419c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:37.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.696+0000 7ff6a96af640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a4111a10 con 0x7ff6a4101170 2024-03-04T09:09:37.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.696+0000 7ff6a96af640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a4111890 con 0x7ff6a4106680 2024-03-04T09:09:37.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.696+0000 7ff6a96af640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a4111b90 con 0x7ff6a41007b0 2024-03-04T09:09:37.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.696+0000 7ff69bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6a4101170 0x7ff6a41983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:37.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.696+0000 7ff69bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6a4101170 0x7ff6a41983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41114/0 (socket says 172.21.15.71:41114) 2024-03-04T09:09:37.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.696+0000 7ff69bfff640 1 -- 172.21.15.71:0/3369231201 learned_addr learned my addr 172.21.15.71:0/3369231201 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T09:09:37.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.696+0000 7ff6a37fe640 1 --2- 172.21.15.71:0/3369231201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6a4106680 0x7ff6a419c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:37.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.696+0000 7ff6a2ffd640 1 --2- 172.21.15.71:0/3369231201 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff6a41007b0 0x7ff6a4197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:37.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.696+0000 7ff69bfff640 1 -- 172.21.15.71:0/3369231201 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff6a41007b0 msgr2=0x7ff6a4197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:37.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.696+0000 7ff69bfff640 1 --2- 172.21.15.71:0/3369231201 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff6a41007b0 0x7ff6a4197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.696+0000 7ff69bfff640 1 -- 172.21.15.71:0/3369231201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6a4106680 msgr2=0x7ff6a419c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:37.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.696+0000 7ff69bfff640 1 --2- 172.21.15.71:0/3369231201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6a4106680 0x7ff6a419c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.696+0000 7ff69bfff640 1 -- 172.21.15.71:0/3369231201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a419ce50 con 0x7ff6a4101170 2024-03-04T09:09:37.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.696+0000 7ff69bfff640 1 --2- 172.21.15.71:0/3369231201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6a4101170 0x7ff6a41983f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff68c00d950 tx=0x7ff68c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:37.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.696+0000 7ff6a0ff9640 1 -- 172.21.15.71:0/3369231201 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c017030 con 0x7ff6a4101170 2024-03-04T09:09:37.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.696+0000 7ff6a0ff9640 1 -- 172.21.15.71:0/3369231201 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff68c004020 con 0x7ff6a4101170 2024-03-04T09:09:37.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.696+0000 7ff6a96af640 1 -- 172.21.15.71:0/3369231201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a419d140 con 0x7ff6a4101170 2024-03-04T09:09:37.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.696+0000 7ff6a0ff9640 1 -- 172.21.15.71:0/3369231201 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c0047a0 con 0x7ff6a4101170 2024-03-04T09:09:37.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.696+0000 7ff6a96af640 1 -- 172.21.15.71:0/3369231201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a41a49d0 con 0x7ff6a4101170 2024-03-04T09:09:37.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.698+0000 7ff6a96af640 1 -- 172.21.15.71:0/3369231201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a4101bf0 con 0x7ff6a4101170 2024-03-04T09:09:37.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.699+0000 7ff6a0ff9640 1 -- 172.21.15.71:0/3369231201 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7ff68c0041d0 con 0x7ff6a4101170 2024-03-04T09:09:37.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.699+0000 7ff6a0ff9640 1 --2- 172.21.15.71:0/3369231201 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7ff6900a1f70 0x7ff6900a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:37.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.699+0000 7ff6a2ffd640 1 --2- 172.21.15.71:0/3369231201 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7ff6900a1f70 0x7ff6900a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:37.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.699+0000 7ff6a0ff9640 1 -- 172.21.15.71:0/3369231201 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9467+0+0 (secure 0 0 0) 0x7ff68c0cdf70 con 0x7ff6a4101170 2024-03-04T09:09:37.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.700+0000 7ff6a2ffd640 1 --2- 172.21.15.71:0/3369231201 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7ff6900a1f70 0x7ff6900a4430 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff694036040 tx=0x7ff694005fd0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:37.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.704+0000 7ff6a0ff9640 1 -- 172.21.15.71:0/3369231201 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff68c09acb0 con 0x7ff6a4101170 2024-03-04T09:09:37.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.876+0000 7ff6a96af640 1 -- 172.21.15.71:0/3369231201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff6a41083f0 con 0x7ff6a4101170 2024-03-04T09:09:37.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.877+0000 7ff6a0ff9640 1 -- 172.21.15.71:0/3369231201 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v104) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff68c09a5b0 con 0x7ff6a4101170 2024-03-04T09:09:37.873 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-03-04T09:09:37.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.879+0000 7ff6a96af640 1 -- 172.21.15.71:0/3369231201 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7ff6900a1f70 msgr2=0x7ff6900a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:37.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.879+0000 7ff6a96af640 1 --2- 172.21.15.71:0/3369231201 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7ff6900a1f70 0x7ff6900a4430 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff694036040 tx=0x7ff694005fd0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.879+0000 7ff6a96af640 1 -- 172.21.15.71:0/3369231201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6a4101170 msgr2=0x7ff6a41983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:37.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.879+0000 7ff6a96af640 1 --2- 172.21.15.71:0/3369231201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6a4101170 0x7ff6a41983f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff68c00d950 tx=0x7ff68c00de10 comp rx=0 tx=0).stop 2024-03-04T09:09:37.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.879+0000 7ff6a96af640 1 -- 172.21.15.71:0/3369231201 shutdown_connections 2024-03-04T09:09:37.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.879+0000 7ff6a96af640 1 --2- 172.21.15.71:0/3369231201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff6a4106680 0x7ff6a419c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.879+0000 7ff6a96af640 1 --2- 172.21.15.71:0/3369231201 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7ff6900a1f70 0x7ff6900a4430 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.879+0000 7ff6a96af640 1 --2- 172.21.15.71:0/3369231201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6a4101170 0x7ff6a41983f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.879+0000 7ff6a96af640 1 --2- 172.21.15.71:0/3369231201 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7ff6a41007b0 0x7ff6a4197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.879+0000 7ff6a96af640 1 -- 172.21.15.71:0/3369231201 >> 172.21.15.71:0/3369231201 conn(0x7ff6a40774b0 msgr2=0x7ff6a40ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:37.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.879+0000 7ff6a96af640 1 -- 172.21.15.71:0/3369231201 shutdown_connections 2024-03-04T09:09:37.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.879+0000 7ff6a96af640 1 -- 172.21.15.71:0/3369231201 wait complete. 2024-03-04T09:09:37.886 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-04T09:09:37.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.992+0000 7f434d383640 1 -- 172.21.15.71:0/1561379621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4348101170 msgr2=0x7f434810d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:37.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.992+0000 7f434d383640 1 --2- 172.21.15.71:0/1561379621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4348101170 0x7f434810d2e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4340009ca0 tx=0x7f434002c180 comp rx=0 tx=0).stop 2024-03-04T09:09:37.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.992+0000 7f434d383640 1 -- 172.21.15.71:0/1561379621 shutdown_connections 2024-03-04T09:09:37.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.992+0000 7f434d383640 1 --2- 172.21.15.71:0/1561379621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4348101170 0x7f434810d2e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.992+0000 7f434d383640 1 --2- 172.21.15.71:0/1561379621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43481007b0 0x7f4348100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.992+0000 7f434d383640 1 --2- 172.21.15.71:0/1561379621 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4348106680 0x7f4348106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.992+0000 7f434d383640 1 -- 172.21.15.71:0/1561379621 >> 172.21.15.71:0/1561379621 conn(0x7f43480774b0 msgr2=0x7f43480778c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:37.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.992+0000 7f434d383640 1 -- 172.21.15.71:0/1561379621 shutdown_connections 2024-03-04T09:09:37.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.992+0000 7f434d383640 1 -- 172.21.15.71:0/1561379621 wait complete. 2024-03-04T09:09:37.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.993+0000 7f434d383640 1 Processor -- start 2024-03-04T09:09:37.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.993+0000 7f434d383640 1 -- start start 2024-03-04T09:09:37.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.993+0000 7f434d383640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f43481007b0 0x7f434819c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:37.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.993+0000 7f434d383640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4348101170 0x7f434819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:37.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.993+0000 7f434d383640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4348106680 0x7f4348196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:37.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.993+0000 7f434d383640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4348110890 con 0x7f4348101170 2024-03-04T09:09:37.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.993+0000 7f434d383640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4348110710 con 0x7f4348106680 2024-03-04T09:09:37.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.993+0000 7f434d383640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4348110a10 con 0x7f43481007b0 2024-03-04T09:09:37.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.993+0000 7f433ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4348101170 0x7f434819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:37.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.993+0000 7f433ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4348101170 0x7f434819ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41146/0 (socket says 172.21.15.71:41146) 2024-03-04T09:09:37.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.993+0000 7f433ffff640 1 -- 172.21.15.71:0/130635592 learned_addr learned my addr 172.21.15.71:0/130635592 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T09:09:37.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.993+0000 7f43477fe640 1 --2- 172.21.15.71:0/130635592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4348106680 0x7f4348196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:37.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.993+0000 7f4346ffd640 1 --2- 172.21.15.71:0/130635592 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f43481007b0 0x7f434819c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:37.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.993+0000 7f433ffff640 1 -- 172.21.15.71:0/130635592 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f43481007b0 msgr2=0x7f434819c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:37.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.993+0000 7f433ffff640 1 --2- 172.21.15.71:0/130635592 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f43481007b0 0x7f434819c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.993+0000 7f433ffff640 1 -- 172.21.15.71:0/130635592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4348106680 msgr2=0x7f4348196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:37.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.993+0000 7f433ffff640 1 --2- 172.21.15.71:0/130635592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4348106680 0x7f4348196af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:37.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.993+0000 7f433ffff640 1 -- 172.21.15.71:0/130635592 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4348197270 con 0x7f4348101170 2024-03-04T09:09:37.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.994+0000 7f433ffff640 1 --2- 172.21.15.71:0/130635592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4348101170 0x7f434819ce80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f433000e9f0 tx=0x7f433000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:37.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.994+0000 7f4344ff9640 1 -- 172.21.15.71:0/130635592 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433000cd80 con 0x7f4348101170 2024-03-04T09:09:37.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.994+0000 7f4344ff9640 1 -- 172.21.15.71:0/130635592 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4330010910 con 0x7f4348101170 2024-03-04T09:09:37.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.994+0000 7f434d383640 1 -- 172.21.15.71:0/130635592 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4348197560 con 0x7f4348101170 2024-03-04T09:09:37.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.994+0000 7f4344ff9640 1 -- 172.21.15.71:0/130635592 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4330018990 con 0x7f4348101170 2024-03-04T09:09:37.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.994+0000 7f434d383640 1 -- 172.21.15.71:0/130635592 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43481a3800 con 0x7f4348101170 2024-03-04T09:09:37.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.995+0000 7f434d383640 1 -- 172.21.15.71:0/130635592 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4348101bf0 con 0x7f4348101170 2024-03-04T09:09:37.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.996+0000 7f4344ff9640 1 -- 172.21.15.71:0/130635592 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f4330010ac0 con 0x7f4348101170 2024-03-04T09:09:37.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.997+0000 7f4344ff9640 1 --2- 172.21.15.71:0/130635592 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f43340a1ef0 0x7f43340a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:37.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.997+0000 7f4346ffd640 1 --2- 172.21.15.71:0/130635592 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f43340a1ef0 0x7f43340a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:37.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.997+0000 7f4344ff9640 1 -- 172.21.15.71:0/130635592 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9467+0+0 (secure 0 0 0) 0x7f43300ced40 con 0x7f4348101170 2024-03-04T09:09:37.996 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:37.997+0000 7f4346ffd640 1 --2- 172.21.15.71:0/130635592 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f43340a1ef0 0x7f43340a43b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4338005fd0 tx=0x7f4338005850 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:37.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.001+0000 7f4344ff9640 1 -- 172.21.15.71:0/130635592 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f433009ba80 con 0x7f4348101170 2024-03-04T09:09:38.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:38 smithi071 ceph-mon[32789]: pgmap v332: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T09:09:38.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:38 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2879417265' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:09:38.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:38 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/4036260046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:09:38.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:38 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3369231201' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:09:38.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.189+0000 7f434d383640 1 -- 172.21.15.71:0/130635592 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4348068810 con 0x7f4348101170 2024-03-04T09:09:38.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.189+0000 7f4344ff9640 1 -- 172.21.15.71:0/130635592 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v104) v1 ==== 81+0+945 (secure 0 0 0) 0x7f4348068810 con 0x7f4348101170 2024-03-04T09:09:38.185 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/2792362717 2024-03-05T08:58:54.820040+0000 2024-03-04T09:09:38.186 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/1289352375 2024-03-05T08:58:54.820040+0000 2024-03-04T09:09:38.186 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6801/2901868399 2024-03-05T08:58:54.820040+0000 2024-03-04T09:09:38.186 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6800/2901868399 2024-03-05T08:58:54.820040+0000 2024-03-04T09:09:38.186 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/1087047482 2024-03-05T08:58:54.820040+0000 2024-03-04T09:09:38.186 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/4186644246 2024-03-05T08:58:54.820040+0000 2024-03-04T09:09:38.186 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/1287712358 2024-03-05T08:48:22.593931+0000 2024-03-04T09:09:38.186 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/3770452860 2024-03-05T08:48:22.593931+0000 2024-03-04T09:09:38.186 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/718743094 2024-03-05T08:47:55.502914+0000 2024-03-04T09:09:38.187 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6801/1350134536 2024-03-05T08:47:55.502914+0000 2024-03-04T09:09:38.187 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/1904717532 2024-03-05T08:47:55.502914+0000 2024-03-04T09:09:38.187 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6801/2808156336 2024-03-05T08:48:22.593931+0000 2024-03-04T09:09:38.187 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6800/1350134536 2024-03-05T08:47:55.502914+0000 2024-03-04T09:09:38.187 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6800/2808156336 2024-03-05T08:48:22.593931+0000 2024-03-04T09:09:38.187 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/2862184313 2024-03-05T08:47:55.502914+0000 2024-03-04T09:09:38.187 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/3621754630 2024-03-05T08:48:22.593931+0000 2024-03-04T09:09:38.188 INFO:teuthology.orchestra.run.smithi071.stderr:listed 16 entries 2024-03-04T09:09:38.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.191+0000 7f434d383640 1 -- 172.21.15.71:0/130635592 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f43340a1ef0 msgr2=0x7f43340a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:38.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.191+0000 7f434d383640 1 --2- 172.21.15.71:0/130635592 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f43340a1ef0 0x7f43340a43b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4338005fd0 tx=0x7f4338005850 comp rx=0 tx=0).stop 2024-03-04T09:09:38.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.191+0000 7f434d383640 1 -- 172.21.15.71:0/130635592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4348101170 msgr2=0x7f434819ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:38.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.191+0000 7f434d383640 1 --2- 172.21.15.71:0/130635592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4348101170 0x7f434819ce80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f433000e9f0 tx=0x7f433000eeb0 comp rx=0 tx=0).stop 2024-03-04T09:09:38.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.191+0000 7f434d383640 1 -- 172.21.15.71:0/130635592 shutdown_connections 2024-03-04T09:09:38.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.191+0000 7f434d383640 1 --2- 172.21.15.71:0/130635592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4348106680 0x7f4348196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:38.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.191+0000 7f434d383640 1 --2- 172.21.15.71:0/130635592 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f43340a1ef0 0x7f43340a43b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:38.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.191+0000 7f434d383640 1 --2- 172.21.15.71:0/130635592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4348101170 0x7f434819ce80 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:38.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.191+0000 7f434d383640 1 --2- 172.21.15.71:0/130635592 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f43481007b0 0x7f434819c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:38.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.191+0000 7f434d383640 1 -- 172.21.15.71:0/130635592 >> 172.21.15.71:0/130635592 conn(0x7f43480774b0 msgr2=0x7f43480ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:38.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.191+0000 7f434d383640 1 -- 172.21.15.71:0/130635592 shutdown_connections 2024-03-04T09:09:38.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.191+0000 7f434d383640 1 -- 172.21.15.71:0/130635592 wait complete. 2024-03-04T09:09:38.198 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-04T09:09:38.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:09:38.198 DEBUG:teuthology.orchestra.run.smithi071:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-04T09:09:38.253 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2024-03-04T09:09:38.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:09:38.254 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-04T09:09:38.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:09:38.277 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-03-04T09:09:38.333 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-04T09:09:38.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:09:38.334 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-04T09:09:38.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:09:38.402 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link delete ceph-brx 2024-03-04T09:09:38.492 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-04T09:09:38.502 INFO:tasks.cephadm:Teardown begin 2024-03-04T09:09:38.502 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-04T09:09:38.502 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:09:38.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:38 smithi150 ceph-mon[36807]: pgmap v332: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T09:09:38.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:38 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2879417265' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:09:38.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:38 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/4036260046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:09:38.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:38 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3369231201' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:09:38.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:38 smithi158 ceph-mon[37241]: pgmap v332: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-04T09:09:38.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:38 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2879417265' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:09:38.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:38 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/4036260046' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:09:38.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:38 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3369231201' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:09:38.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.609+0000 7fc1826eb640 1 -- 172.21.15.71:0/2147880171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc17c1011e0 msgr2=0x7fc17c10d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:38.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.609+0000 7fc1826eb640 1 --2- 172.21.15.71:0/2147880171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc17c1011e0 0x7fc17c10d2b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc174009cd0 tx=0x7fc17402c0f0 comp rx=0 tx=0).stop 2024-03-04T09:09:38.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.609+0000 7fc1826eb640 1 -- 172.21.15.71:0/2147880171 shutdown_connections 2024-03-04T09:09:38.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.609+0000 7fc1826eb640 1 --2- 172.21.15.71:0/2147880171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc17c1011e0 0x7fc17c10d2b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:38.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.609+0000 7fc1826eb640 1 --2- 172.21.15.71:0/2147880171 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc17c100820 0x7fc17c100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:38.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.609+0000 7fc1826eb640 1 --2- 172.21.15.71:0/2147880171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc17c1054b0 0x7fc17c105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:38.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.609+0000 7fc1826eb640 1 -- 172.21.15.71:0/2147880171 >> 172.21.15.71:0/2147880171 conn(0x7fc17c0762d0 msgr2=0x7fc17c0766e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:38.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.609+0000 7fc1826eb640 1 -- 172.21.15.71:0/2147880171 shutdown_connections 2024-03-04T09:09:38.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.609+0000 7fc1826eb640 1 -- 172.21.15.71:0/2147880171 wait complete. 2024-03-04T09:09:38.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.609+0000 7fc1826eb640 1 Processor -- start 2024-03-04T09:09:38.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.610+0000 7fc1826eb640 1 -- start start 2024-03-04T09:09:38.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.610+0000 7fc1826eb640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc17c100820 0x7fc17c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:38.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.610+0000 7fc1826eb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc17c1011e0 0x7fc17c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:38.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.610+0000 7fc1826eb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc17c1054b0 0x7fc17c19c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:38.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.610+0000 7fc1826eb640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c111a00 con 0x7fc17c1011e0 2024-03-04T09:09:38.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.610+0000 7fc1826eb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c111880 con 0x7fc17c1054b0 2024-03-04T09:09:38.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.610+0000 7fc1826eb640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c111b80 con 0x7fc17c100820 2024-03-04T09:09:38.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.610+0000 7fc17b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc17c1011e0 0x7fc17c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:38.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.610+0000 7fc17b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc17c1011e0 0x7fc17c198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41164/0 (socket says 172.21.15.71:41164) 2024-03-04T09:09:38.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.610+0000 7fc17b7fe640 1 -- 172.21.15.71:0/2911001720 learned_addr learned my addr 172.21.15.71:0/2911001720 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T09:09:38.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.610+0000 7fc180c88640 1 --2- 172.21.15.71:0/2911001720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc17c1054b0 0x7fc17c19c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:38.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.610+0000 7fc17bfff640 1 --2- 172.21.15.71:0/2911001720 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc17c100820 0x7fc17c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:38.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.610+0000 7fc17b7fe640 1 -- 172.21.15.71:0/2911001720 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc17c100820 msgr2=0x7fc17c197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:38.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.610+0000 7fc17b7fe640 1 --2- 172.21.15.71:0/2911001720 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc17c100820 0x7fc17c197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:38.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.610+0000 7fc17b7fe640 1 -- 172.21.15.71:0/2911001720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc17c1054b0 msgr2=0x7fc17c19c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:38.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.610+0000 7fc17b7fe640 1 --2- 172.21.15.71:0/2911001720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc17c1054b0 0x7fc17c19c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:38.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.610+0000 7fc17b7fe640 1 -- 172.21.15.71:0/2911001720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c19ce60 con 0x7fc17c1011e0 2024-03-04T09:09:38.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.610+0000 7fc17b7fe640 1 --2- 172.21.15.71:0/2911001720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc17c1011e0 0x7fc17c198400 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc16c00c970 tx=0x7fc16c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:38.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.611+0000 7fc1797fa640 1 -- 172.21.15.71:0/2911001720 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c007bb0 con 0x7fc17c1011e0 2024-03-04T09:09:38.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.611+0000 7fc1826eb640 1 -- 172.21.15.71:0/2911001720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc17c19d150 con 0x7fc17c1011e0 2024-03-04T09:09:38.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.611+0000 7fc1797fa640 1 -- 172.21.15.71:0/2911001720 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc16c007d50 con 0x7fc17c1011e0 2024-03-04T09:09:38.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.611+0000 7fc1797fa640 1 -- 172.21.15.71:0/2911001720 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c00f450 con 0x7fc17c1011e0 2024-03-04T09:09:38.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.611+0000 7fc1826eb640 1 -- 172.21.15.71:0/2911001720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc17c1a49e0 con 0x7fc17c1011e0 2024-03-04T09:09:38.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.612+0000 7fc1826eb640 1 -- 172.21.15.71:0/2911001720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc17c101bf0 con 0x7fc17c1011e0 2024-03-04T09:09:38.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.613+0000 7fc1797fa640 1 -- 172.21.15.71:0/2911001720 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7fc16c005020 con 0x7fc17c1011e0 2024-03-04T09:09:38.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.613+0000 7fc1797fa640 1 --2- 172.21.15.71:0/2911001720 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fc1640a1ef0 0x7fc1640a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:38.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.613+0000 7fc17bfff640 1 --2- 172.21.15.71:0/2911001720 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fc1640a1ef0 0x7fc1640a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:38.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.613+0000 7fc1797fa640 1 -- 172.21.15.71:0/2911001720 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9467+0+0 (secure 0 0 0) 0x7fc16c0ce3f0 con 0x7fc17c1011e0 2024-03-04T09:09:38.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.614+0000 7fc17bfff640 1 --2- 172.21.15.71:0/2911001720 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fc1640a1ef0 0x7fc1640a43b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc170009aa0 tx=0x7fc170005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:38.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.617+0000 7fc1797fa640 1 -- 172.21.15.71:0/2911001720 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc16c014030 con 0x7fc17c1011e0 2024-03-04T09:09:38.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.832+0000 7fc1826eb640 1 -- 172.21.15.71:0/2911001720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc17c1083c0 con 0x7fc17c1011e0 2024-03-04T09:09:38.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.833+0000 7fc1797fa640 1 -- 172.21.15.71:0/2911001720 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11339 (secure 0 0 0) 0x7fc16c09b110 con 0x7fc17c1011e0 2024-03-04T09:09:38.830 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T09:09:38.830 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3975379807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3975379807},{"type":"v1","addr":"172.21.15.71:6835","nonce":3975379807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1962929347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1962929347},{"type":"v1","addr":"172.21.15.71:6837","nonce":1962929347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/1459252176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1459252176},{"type":"v1","addr":"172.21.15.71:6839","nonce":1459252176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6841/1083546600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":1083546600},{"type":"v1","addr":"172.21.15.71:6841","nonce":1083546600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/178179528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":178179528},{"type":"v1","addr":"172.21.15.158:6833","nonce":178179528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2468877675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2468877675},{"type":"v1","addr":"172.21.15.150:6833","nonce":2468877675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1652094309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1652094309},{"type":"v1","addr":"172.21.15.158:6835","nonce":1652094309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1802416462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1802416462},{"type":"v1","addr":"172.21.15.150:6835","nonce":1802416462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/617058380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":617058380},{"type":"v1","addr":"172.21.15.158:6837","nonce":617058380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/913465600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":913465600},{"type":"v1","addr":"172.21.15.150:6837","nonce":913465600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T08:56:30.343357+0000","modified":"2024-03-04T08:56:54.901214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.158:6839/2921704804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6838","nonce":2921704804},{"type":"v1","addr":"172.21.15.158:6839","nonce":2921704804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/2639278083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2639278083},{"type":"v1","addr":"172.21.15.150:6839","nonce":2639278083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491]},"id":1}]} 2024-03-04T09:09:38.830 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 20 2024-03-04T09:09:38.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.835+0000 7fc1826eb640 1 -- 172.21.15.71:0/2911001720 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fc1640a1ef0 msgr2=0x7fc1640a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:38.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.835+0000 7fc1826eb640 1 --2- 172.21.15.71:0/2911001720 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fc1640a1ef0 0x7fc1640a43b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc170009aa0 tx=0x7fc170005f90 comp rx=0 tx=0).stop 2024-03-04T09:09:38.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.835+0000 7fc1826eb640 1 -- 172.21.15.71:0/2911001720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc17c1011e0 msgr2=0x7fc17c198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:38.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.835+0000 7fc1826eb640 1 --2- 172.21.15.71:0/2911001720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc17c1011e0 0x7fc17c198400 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc16c00c970 tx=0x7fc16c00ce30 comp rx=0 tx=0).stop 2024-03-04T09:09:38.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.835+0000 7fc1826eb640 1 -- 172.21.15.71:0/2911001720 shutdown_connections 2024-03-04T09:09:38.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.835+0000 7fc1826eb640 1 --2- 172.21.15.71:0/2911001720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc17c1054b0 0x7fc17c19c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:38.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.835+0000 7fc1826eb640 1 --2- 172.21.15.71:0/2911001720 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fc1640a1ef0 0x7fc1640a43b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:38.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.835+0000 7fc1826eb640 1 --2- 172.21.15.71:0/2911001720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc17c1011e0 0x7fc17c198400 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:38.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.835+0000 7fc1826eb640 1 --2- 172.21.15.71:0/2911001720 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc17c100820 0x7fc17c197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:38.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.835+0000 7fc1826eb640 1 -- 172.21.15.71:0/2911001720 >> 172.21.15.71:0/2911001720 conn(0x7fc17c0762d0 msgr2=0x7fc17c0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:38.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.835+0000 7fc1826eb640 1 -- 172.21.15.71:0/2911001720 shutdown_connections 2024-03-04T09:09:38.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.835+0000 7fc1826eb640 1 -- 172.21.15.71:0/2911001720 wait complete. 2024-03-04T09:09:38.843 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:09:38.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.948+0000 7fda96b3e640 1 -- 172.21.15.71:0/2510864018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda901001d0 msgr2=0x7fda901005b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:38.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.948+0000 7fda96b3e640 1 --2- 172.21.15.71:0/2510864018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda901001d0 0x7fda901005b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fda84009900 tx=0x7fda8402bc30 comp rx=0 tx=0).stop 2024-03-04T09:09:38.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.948+0000 7fda96b3e640 1 -- 172.21.15.71:0/2510864018 shutdown_connections 2024-03-04T09:09:38.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.948+0000 7fda96b3e640 1 --2- 172.21.15.71:0/2510864018 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fda90106990 0x7fda9010e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:38.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.948+0000 7fda96b3e640 1 --2- 172.21.15.71:0/2510864018 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda90100b80 0x7fda901062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:38.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.948+0000 7fda96b3e640 1 --2- 172.21.15.71:0/2510864018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda901001d0 0x7fda901005b0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:38.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.948+0000 7fda96b3e640 1 -- 172.21.15.71:0/2510864018 >> 172.21.15.71:0/2510864018 conn(0x7fda900757e0 msgr2=0x7fda90075bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:38.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.948+0000 7fda96b3e640 1 -- 172.21.15.71:0/2510864018 shutdown_connections 2024-03-04T09:09:38.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.948+0000 7fda96b3e640 1 -- 172.21.15.71:0/2510864018 wait complete. 2024-03-04T09:09:38.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.949+0000 7fda96b3e640 1 Processor -- start 2024-03-04T09:09:38.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.949+0000 7fda96b3e640 1 -- start start 2024-03-04T09:09:38.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.949+0000 7fda96b3e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda901001d0 0x7fda90197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:38.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.949+0000 7fda96b3e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda90106990 0x7fda901984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:38.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.949+0000 7fda96b3e640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fda9019c840 0x7fda9019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:38.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.949+0000 7fda96b3e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda90111a80 con 0x7fda901001d0 2024-03-04T09:09:38.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.949+0000 7fda96b3e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda90111900 con 0x7fda90106990 2024-03-04T09:09:38.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.949+0000 7fda96b3e640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda90111c00 con 0x7fda9019c840 2024-03-04T09:09:38.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.949+0000 7fda948da640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda901001d0 0x7fda90197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:38.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.949+0000 7fda948da640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda901001d0 0x7fda90197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41172/0 (socket says 172.21.15.71:41172) 2024-03-04T09:09:38.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.949+0000 7fda948da640 1 -- 172.21.15.71:0/2093207893 learned_addr learned my addr 172.21.15.71:0/2093207893 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T09:09:38.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.949+0000 7fda8ffff640 1 --2- 172.21.15.71:0/2093207893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda90106990 0x7fda901984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:38.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.949+0000 7fda948da640 1 -- 172.21.15.71:0/2093207893 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fda9019c840 msgr2=0x7fda9019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:38.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.949+0000 7fda948da640 1 --2- 172.21.15.71:0/2093207893 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fda9019c840 0x7fda9019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:38.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.949+0000 7fda948da640 1 -- 172.21.15.71:0/2093207893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda90106990 msgr2=0x7fda901984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:38.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.949+0000 7fda948da640 1 --2- 172.21.15.71:0/2093207893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda90106990 0x7fda901984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:38.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.949+0000 7fda948da640 1 -- 172.21.15.71:0/2093207893 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9019d330 con 0x7fda901001d0 2024-03-04T09:09:38.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.949+0000 7fda8ffff640 1 --2- 172.21.15.71:0/2093207893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda90106990 0x7fda901984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:09:38.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.949+0000 7fda948da640 1 --2- 172.21.15.71:0/2093207893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda901001d0 0x7fda90197f80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fda84006010 tx=0x7fda8402cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:38.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.950+0000 7fda8dffb640 1 -- 172.21.15.71:0/2093207893 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda84041070 con 0x7fda901001d0 2024-03-04T09:09:38.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.950+0000 7fda8dffb640 1 -- 172.21.15.71:0/2093207893 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda84004450 con 0x7fda901001d0 2024-03-04T09:09:38.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.950+0000 7fda96b3e640 1 -- 172.21.15.71:0/2093207893 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda9019d5c0 con 0x7fda901001d0 2024-03-04T09:09:38.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.950+0000 7fda8dffb640 1 -- 172.21.15.71:0/2093207893 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda84003e00 con 0x7fda901001d0 2024-03-04T09:09:38.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.950+0000 7fda96b3e640 1 -- 172.21.15.71:0/2093207893 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda901a4d80 con 0x7fda901001d0 2024-03-04T09:09:38.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.951+0000 7fda96b3e640 1 -- 172.21.15.71:0/2093207893 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda90069700 con 0x7fda901001d0 2024-03-04T09:09:38.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.952+0000 7fda8dffb640 1 -- 172.21.15.71:0/2093207893 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7fda84044070 con 0x7fda901001d0 2024-03-04T09:09:38.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.952+0000 7fda8dffb640 1 --2- 172.21.15.71:0/2093207893 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fda800a1ef0 0x7fda800a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:38.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.953+0000 7fda8ffff640 1 --2- 172.21.15.71:0/2093207893 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fda800a1ef0 0x7fda800a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:38.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.953+0000 7fda8dffb640 1 -- 172.21.15.71:0/2093207893 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9467+0+0 (secure 0 0 0) 0x7fda840f00b0 con 0x7fda901001d0 2024-03-04T09:09:38.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.953+0000 7fda8ffff640 1 --2- 172.21.15.71:0/2093207893 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fda800a1ef0 0x7fda800a43b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fda900ffb60 tx=0x7fda7c005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:38.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:38.957+0000 7fda8dffb640 1 -- 172.21.15.71:0/2093207893 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fda840bcf00 con 0x7fda901001d0 2024-03-04T09:09:39.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.170+0000 7fda96b3e640 1 -- 172.21.15.71:0/2093207893 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda90109be0 con 0x7fda901001d0 2024-03-04T09:09:39.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.171+0000 7fda8dffb640 1 -- 172.21.15.71:0/2093207893 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11339 (secure 0 0 0) 0x7fda840bc800 con 0x7fda901001d0 2024-03-04T09:09:39.167 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T09:09:39.167 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3975379807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3975379807},{"type":"v1","addr":"172.21.15.71:6835","nonce":3975379807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1962929347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1962929347},{"type":"v1","addr":"172.21.15.71:6837","nonce":1962929347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/1459252176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1459252176},{"type":"v1","addr":"172.21.15.71:6839","nonce":1459252176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6841/1083546600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":1083546600},{"type":"v1","addr":"172.21.15.71:6841","nonce":1083546600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6833/178179528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6832","nonce":178179528},{"type":"v1","addr":"172.21.15.158:6833","nonce":178179528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2468877675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2468877675},{"type":"v1","addr":"172.21.15.150:6833","nonce":2468877675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6835/1652094309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6834","nonce":1652094309},{"type":"v1","addr":"172.21.15.158:6835","nonce":1652094309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1802416462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1802416462},{"type":"v1","addr":"172.21.15.150:6835","nonce":1802416462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.158:6837/617058380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6836","nonce":617058380},{"type":"v1","addr":"172.21.15.158:6837","nonce":617058380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/913465600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":913465600},{"type":"v1","addr":"172.21.15.150:6837","nonce":913465600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T08:56:30.343357+0000","modified":"2024-03-04T08:56:54.901214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.158:6839/2921704804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6838","nonce":2921704804},{"type":"v1","addr":"172.21.15.158:6839","nonce":2921704804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/2639278083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":2639278083},{"type":"v1","addr":"172.21.15.150:6839","nonce":2639278083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491]},"id":1}]} 2024-03-04T09:09:39.168 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 20 2024-03-04T09:09:39.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.173+0000 7fda96b3e640 1 -- 172.21.15.71:0/2093207893 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fda800a1ef0 msgr2=0x7fda800a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:39.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.173+0000 7fda96b3e640 1 --2- 172.21.15.71:0/2093207893 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fda800a1ef0 0x7fda800a43b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fda900ffb60 tx=0x7fda7c005f90 comp rx=0 tx=0).stop 2024-03-04T09:09:39.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.173+0000 7fda96b3e640 1 -- 172.21.15.71:0/2093207893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda901001d0 msgr2=0x7fda90197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:39.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.173+0000 7fda96b3e640 1 --2- 172.21.15.71:0/2093207893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda901001d0 0x7fda90197f80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fda84006010 tx=0x7fda8402cb30 comp rx=0 tx=0).stop 2024-03-04T09:09:39.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.173+0000 7fda96b3e640 1 -- 172.21.15.71:0/2093207893 shutdown_connections 2024-03-04T09:09:39.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.173+0000 7fda96b3e640 1 --2- 172.21.15.71:0/2093207893 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fda9019c840 0x7fda9019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.173+0000 7fda96b3e640 1 --2- 172.21.15.71:0/2093207893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda90106990 0x7fda901984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.173+0000 7fda96b3e640 1 --2- 172.21.15.71:0/2093207893 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fda800a1ef0 0x7fda800a43b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.173+0000 7fda96b3e640 1 --2- 172.21.15.71:0/2093207893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda901001d0 0x7fda90197f80 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.173+0000 7fda96b3e640 1 -- 172.21.15.71:0/2093207893 >> 172.21.15.71:0/2093207893 conn(0x7fda900757e0 msgr2=0x7fda900fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:39.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.173+0000 7fda96b3e640 1 -- 172.21.15.71:0/2093207893 shutdown_connections 2024-03-04T09:09:39.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.173+0000 7fda96b3e640 1 -- 172.21.15.71:0/2093207893 wait complete. 2024-03-04T09:09:39.180 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T09:09:39.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.284+0000 7fc2b140e640 1 -- 172.21.15.71:0/2468224699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2ac106610 msgr2=0x7fc2ac1069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:39.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.284+0000 7fc2b140e640 1 --2- 172.21.15.71:0/2468224699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2ac106610 0x7fc2ac1069f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009900 tx=0x7fc29c02bc30 comp rx=0 tx=0).stop 2024-03-04T09:09:39.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.284+0000 7fc2b140e640 1 -- 172.21.15.71:0/2468224699 shutdown_connections 2024-03-04T09:09:39.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.284+0000 7fc2b140e640 1 --2- 172.21.15.71:0/2468224699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2ac101170 0x7fc2ac10d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.284+0000 7fc2b140e640 1 --2- 172.21.15.71:0/2468224699 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2ac1007b0 0x7fc2ac100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.284+0000 7fc2b140e640 1 --2- 172.21.15.71:0/2468224699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2ac106610 0x7fc2ac1069f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.284+0000 7fc2b140e640 1 -- 172.21.15.71:0/2468224699 >> 172.21.15.71:0/2468224699 conn(0x7fc2ac0774b0 msgr2=0x7fc2ac0778c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:39.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.284+0000 7fc2b140e640 1 -- 172.21.15.71:0/2468224699 shutdown_connections 2024-03-04T09:09:39.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.284+0000 7fc2b140e640 1 -- 172.21.15.71:0/2468224699 wait complete. 2024-03-04T09:09:39.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.285+0000 7fc2b140e640 1 Processor -- start 2024-03-04T09:09:39.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.285+0000 7fc2b140e640 1 -- start start 2024-03-04T09:09:39.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.285+0000 7fc2b140e640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2ac1007b0 0x7fc2ac198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:39.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.285+0000 7fc2b140e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2ac101170 0x7fc2ac1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:39.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.285+0000 7fc2b140e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2ac106610 0x7fc2ac1a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:39.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.285+0000 7fc2b140e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac110810 con 0x7fc2ac106610 2024-03-04T09:09:39.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.285+0000 7fc2b140e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac110690 con 0x7fc2ac101170 2024-03-04T09:09:39.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.285+0000 7fc2b140e640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac110990 con 0x7fc2ac1007b0 2024-03-04T09:09:39.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.285+0000 7fc2ab7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2ac106610 0x7fc2ac1a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:39.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.285+0000 7fc2ab7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2ac106610 0x7fc2ac1a3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41192/0 (socket says 172.21.15.71:41192) 2024-03-04T09:09:39.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.285+0000 7fc2ab7fe640 1 -- 172.21.15.71:0/3588054210 learned_addr learned my addr 172.21.15.71:0/3588054210 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T09:09:39.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.285+0000 7fc2a3fff640 1 --2- 172.21.15.71:0/3588054210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2ac101170 0x7fc2ac1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:39.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.285+0000 7fc2aaffd640 1 --2- 172.21.15.71:0/3588054210 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2ac1007b0 0x7fc2ac198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:39.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.285+0000 7fc2ab7fe640 1 -- 172.21.15.71:0/3588054210 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2ac1007b0 msgr2=0x7fc2ac198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:39.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.285+0000 7fc2ab7fe640 1 --2- 172.21.15.71:0/3588054210 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2ac1007b0 0x7fc2ac198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.285+0000 7fc2ab7fe640 1 -- 172.21.15.71:0/3588054210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2ac101170 msgr2=0x7fc2ac1991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:39.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.285+0000 7fc2ab7fe640 1 --2- 172.21.15.71:0/3588054210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2ac101170 0x7fc2ac1991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.285+0000 7fc2ab7fe640 1 -- 172.21.15.71:0/3588054210 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ac1a4270 con 0x7fc2ac106610 2024-03-04T09:09:39.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.286+0000 7fc2ab7fe640 1 --2- 172.21.15.71:0/3588054210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2ac106610 0x7fc2ac1a3d30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400efd0 tx=0x7fc2a400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:39.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.286+0000 7fc2a8ff9640 1 -- 172.21.15.71:0/3588054210 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4003b70 con 0x7fc2ac106610 2024-03-04T09:09:39.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.286+0000 7fc2a8ff9640 1 -- 172.21.15.71:0/3588054210 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2a4003d10 con 0x7fc2ac106610 2024-03-04T09:09:39.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.286+0000 7fc2b140e640 1 -- 172.21.15.71:0/3588054210 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2ac1a4440 con 0x7fc2ac106610 2024-03-04T09:09:39.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.286+0000 7fc2a8ff9640 1 -- 172.21.15.71:0/3588054210 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a40109f0 con 0x7fc2ac106610 2024-03-04T09:09:39.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.286+0000 7fc2b140e640 1 -- 172.21.15.71:0/3588054210 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2ac1a48b0 con 0x7fc2ac106610 2024-03-04T09:09:39.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.287+0000 7fc2b140e640 1 -- 172.21.15.71:0/3588054210 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2ac101bd0 con 0x7fc2ac106610 2024-03-04T09:09:39.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.288+0000 7fc2a8ff9640 1 -- 172.21.15.71:0/3588054210 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7fc2a4004020 con 0x7fc2ac106610 2024-03-04T09:09:39.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.288+0000 7fc2a8ff9640 1 --2- 172.21.15.71:0/3588054210 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fc2980a1ef0 0x7fc2980a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:39.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.288+0000 7fc2aaffd640 1 --2- 172.21.15.71:0/3588054210 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fc2980a1ef0 0x7fc2980a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:39.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.288+0000 7fc2a8ff9640 1 -- 172.21.15.71:0/3588054210 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9467+0+0 (secure 0 0 0) 0x7fc2a40ce910 con 0x7fc2ac106610 2024-03-04T09:09:39.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.289+0000 7fc2aaffd640 1 --2- 172.21.15.71:0/3588054210 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fc2980a1ef0 0x7fc2980a43b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc29c006010 tx=0x7fc29c005dd0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:39.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.293+0000 7fc2a8ff9640 1 -- 172.21.15.71:0/3588054210 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc2a409b650 con 0x7fc2ac106610 2024-03-04T09:09:39.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:39 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/130635592' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T09:09:39.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:39 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2911001720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:09:39.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:39 smithi071 ceph-mon[32789]: pgmap v333: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:09:39.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.469+0000 7fc2b140e640 1 -- 172.21.15.71:0/3588054210 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2ac108410 con 0x7fc2ac106610 2024-03-04T09:09:39.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.471+0000 7fc2a8ff9640 1 -- 172.21.15.71:0/3588054210 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+25291 (secure 0 0 0) 0x7fc2a409af50 con 0x7fc2ac106610 2024-03-04T09:09:39.468 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-03-04T09:09:39.468 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":104,"fsid":"b3c85c40-da03-11ee-95c6-87774f69a715","created":"2024-03-04T08:47:28.546102+0000","modified":"2024-03-04T09:09:14.702148+0000","last_up_change":"2024-03-04T08:55:30.744961+0000","last_in_change":"2024-03-04T08:55:11.570990+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T08:51:14.933487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T08:56:28.253074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T08:56:29.543622+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":16,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T08:56:32.884220+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":16,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7e43b47d-8888-4631-bc96-8a96fcf8cd46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6803","nonce":1173641530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6805","nonce":1173641530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6809","nonce":1173641530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":1173641530},{"type":"v1","addr":"172.21.15.71:6807","nonce":1173641530}]},"public_addr":"172.21.15.71:6803/1173641530","cluster_addr":"172.21.15.71:6805/1173641530","heartbeat_back_addr":"172.21.15.71:6809/1173641530","heartbeat_front_addr":"172.21.15.71:6807/1173641530","state":["exists","up"]},{"osd":1,"uuid":"1209f214-f2e2-4209-bf69-4528a43a2123","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6801","nonce":3383948110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6803","nonce":3383948110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6807","nonce":3383948110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":3383948110},{"type":"v1","addr":"172.21.15.150:6805","nonce":3383948110}]},"public_addr":"172.21.15.150:6801/3383948110","cluster_addr":"172.21.15.150:6803/3383948110","heartbeat_back_addr":"172.21.15.150:6807/3383948110","heartbeat_front_addr":"172.21.15.150:6805/3383948110","state":["exists","up"]},{"osd":2,"uuid":"b88c3082-93a9-4c9b-9221-a7105a44c91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6800","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6801","nonce":586944144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6802","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6803","nonce":586944144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6806","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6807","nonce":586944144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6804","nonce":586944144},{"type":"v1","addr":"172.21.15.158:6805","nonce":586944144}]},"public_addr":"172.21.15.158:6801/586944144","cluster_addr":"172.21.15.158:6803/586944144","heartbeat_back_addr":"172.21.15.158:6807/586944144","heartbeat_front_addr":"172.21.15.158:6805/586944144","state":["exists","up"]},{"osd":3,"uuid":"e4fc7723-fd73-40b2-b9d0-7f9bda282ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6811","nonce":1311903262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6813","nonce":1311903262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6817","nonce":1311903262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1311903262},{"type":"v1","addr":"172.21.15.71:6815","nonce":1311903262}]},"public_addr":"172.21.15.71:6811/1311903262","cluster_addr":"172.21.15.71:6813/1311903262","heartbeat_back_addr":"172.21.15.71:6817/1311903262","heartbeat_front_addr":"172.21.15.71:6815/1311903262","state":["exists","up"]},{"osd":4,"uuid":"e7eb72ac-f066-4ccd-b2bf-22a6a26daf41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6809","nonce":2300837573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6811","nonce":2300837573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6815","nonce":2300837573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":2300837573},{"type":"v1","addr":"172.21.15.150:6813","nonce":2300837573}]},"public_addr":"172.21.15.150:6809/2300837573","cluster_addr":"172.21.15.150:6811/2300837573","heartbeat_back_addr":"172.21.15.150:6815/2300837573","heartbeat_front_addr":"172.21.15.150:6813/2300837573","state":["exists","up"]},{"osd":5,"uuid":"9b295840-e104-4a66-9cc4-7020627c1765","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6808","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6809","nonce":1922694045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6810","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6811","nonce":1922694045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6814","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6815","nonce":1922694045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6812","nonce":1922694045},{"type":"v1","addr":"172.21.15.158:6813","nonce":1922694045}]},"public_addr":"172.21.15.158:6809/1922694045","cluster_addr":"172.21.15.158:6811/1922694045","heartbeat_back_addr":"172.21.15.158:6815/1922694045","heartbeat_front_addr":"172.21.15.158:6813/1922694045","state":["exists","up"]},{"osd":6,"uuid":"6cb65bd9-f8a9-43fa-a29e-a5b2e7f55ab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6819","nonce":3848842707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6821","nonce":3848842707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6825","nonce":3848842707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":3848842707},{"type":"v1","addr":"172.21.15.71:6823","nonce":3848842707}]},"public_addr":"172.21.15.71:6819/3848842707","cluster_addr":"172.21.15.71:6821/3848842707","heartbeat_back_addr":"172.21.15.71:6825/3848842707","heartbeat_front_addr":"172.21.15.71:6823/3848842707","state":["exists","up"]},{"osd":7,"uuid":"6160a40a-d30f-4569-949c-0afb1bb3d133","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6817","nonce":1328330305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6819","nonce":1328330305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6823","nonce":1328330305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":1328330305},{"type":"v1","addr":"172.21.15.150:6821","nonce":1328330305}]},"public_addr":"172.21.15.150:6817/1328330305","cluster_addr":"172.21.15.150:6819/1328330305","heartbeat_back_addr":"172.21.15.150:6823/1328330305","heartbeat_front_addr":"172.21.15.150:6821/1328330305","state":["exists","up"]},{"osd":8,"uuid":"95ed2341-84a4-465c-9948-ac667680ce34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6816","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6817","nonce":2195304577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6818","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6819","nonce":2195304577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6822","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6823","nonce":2195304577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6820","nonce":2195304577},{"type":"v1","addr":"172.21.15.158:6821","nonce":2195304577}]},"public_addr":"172.21.15.158:6817/2195304577","cluster_addr":"172.21.15.158:6819/2195304577","heartbeat_back_addr":"172.21.15.158:6823/2195304577","heartbeat_front_addr":"172.21.15.158:6821/2195304577","state":["exists","up"]},{"osd":9,"uuid":"088f6419-63a0-4768-afae-dbb8a5f5a565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6827","nonce":1913779347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6829","nonce":1913779347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6833","nonce":1913779347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":1913779347},{"type":"v1","addr":"172.21.15.71:6831","nonce":1913779347}]},"public_addr":"172.21.15.71:6827/1913779347","cluster_addr":"172.21.15.71:6829/1913779347","heartbeat_back_addr":"172.21.15.71:6833/1913779347","heartbeat_front_addr":"172.21.15.71:6831/1913779347","state":["exists","up"]},{"osd":10,"uuid":"18c85e24-068d-4f1e-a3b8-f7915b363234","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6825","nonce":3295420996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6827","nonce":3295420996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6831","nonce":3295420996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":3295420996},{"type":"v1","addr":"172.21.15.150:6829","nonce":3295420996}]},"public_addr":"172.21.15.150:6825/3295420996","cluster_addr":"172.21.15.150:6827/3295420996","heartbeat_back_addr":"172.21.15.150:6831/3295420996","heartbeat_front_addr":"172.21.15.150:6829/3295420996","state":["exists","up"]},{"osd":11,"uuid":"4f6b3ddf-c482-4740-b1fd-de836e047c2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6824","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6825","nonce":3332083865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6826","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6827","nonce":3332083865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6830","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6831","nonce":3332083865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.158:6828","nonce":3332083865},{"type":"v1","addr":"172.21.15.158:6829","nonce":3332083865}]},"public_addr":"172.21.15.158:6825/3332083865","cluster_addr":"172.21.15.158:6827/3332083865","heartbeat_back_addr":"172.21.15.158:6831/3332083865","heartbeat_front_addr":"172.21.15.158:6829/3332083865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:05.997832+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:50:29.314235+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:08.819465+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:51:41.194275+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:06.032479+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:52:38.814498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:13.067960+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:53:38.568479+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:03.416713+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:54:38.349479+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:04.096945+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T08:55:29.425250+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/2792362717":"2024-03-05T08:58:54.820040+0000","172.21.15.71:0/1289352375":"2024-03-05T08:58:54.820040+0000","172.21.15.71:6801/2901868399":"2024-03-05T08:58:54.820040+0000","172.21.15.71:6800/2901868399":"2024-03-05T08:58:54.820040+0000","172.21.15.71:0/1087047482":"2024-03-05T08:58:54.820040+0000","172.21.15.71:0/4186644246":"2024-03-05T08:58:54.820040+0000","172.21.15.71:0/1287712358":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/3770452860":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/718743094":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/1904717532":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6801/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:6800/1350134536":"2024-03-05T08:47:55.502914+0000","172.21.15.71:6800/2808156336":"2024-03-05T08:48:22.593931+0000","172.21.15.71:0/2862184313":"2024-03-05T08:47:55.502914+0000","172.21.15.71:0/3621754630":"2024-03-05T08:48:22.593931+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:09:39.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.473+0000 7fc2b140e640 1 -- 172.21.15.71:0/3588054210 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fc2980a1ef0 msgr2=0x7fc2980a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:39.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.473+0000 7fc2b140e640 1 --2- 172.21.15.71:0/3588054210 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fc2980a1ef0 0x7fc2980a43b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc29c006010 tx=0x7fc29c005dd0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.473+0000 7fc2b140e640 1 -- 172.21.15.71:0/3588054210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2ac106610 msgr2=0x7fc2ac1a3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:39.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.473+0000 7fc2b140e640 1 --2- 172.21.15.71:0/3588054210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2ac106610 0x7fc2ac1a3d30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400efd0 tx=0x7fc2a400c6a0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.473+0000 7fc2b140e640 1 -- 172.21.15.71:0/3588054210 shutdown_connections 2024-03-04T09:09:39.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.473+0000 7fc2b140e640 1 --2- 172.21.15.71:0/3588054210 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fc2980a1ef0 0x7fc2980a43b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.473+0000 7fc2b140e640 1 --2- 172.21.15.71:0/3588054210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2ac106610 0x7fc2ac1a3d30 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.473+0000 7fc2b140e640 1 --2- 172.21.15.71:0/3588054210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2ac101170 0x7fc2ac1991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.473+0000 7fc2b140e640 1 --2- 172.21.15.71:0/3588054210 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fc2ac1007b0 0x7fc2ac198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.473+0000 7fc2b140e640 1 -- 172.21.15.71:0/3588054210 >> 172.21.15.71:0/3588054210 conn(0x7fc2ac0774b0 msgr2=0x7fc2ac0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:39.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.473+0000 7fc2b140e640 1 -- 172.21.15.71:0/3588054210 shutdown_connections 2024-03-04T09:09:39.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.473+0000 7fc2b140e640 1 -- 172.21.15.71:0/3588054210 wait complete. 2024-03-04T09:09:39.480 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-04T09:09:39.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:39 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/130635592' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T09:09:39.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:39 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2911001720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:09:39.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:39 smithi150 ceph-mon[36807]: pgmap v333: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:09:39.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:39 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/130635592' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T09:09:39.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:39 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2911001720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:09:39.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:39 smithi158 ceph-mon[37241]: pgmap v333: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-03-04T09:09:39.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.587+0000 7f4a139d7640 1 -- 172.21.15.71:0/1495123141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c103130 msgr2=0x7f4a0c10b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:39.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.587+0000 7f4a139d7640 1 --2- 172.21.15.71:0/1495123141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c103130 0x7f4a0c10b610 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009900 tx=0x7f49fc02bcd0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.587+0000 7f4a139d7640 1 -- 172.21.15.71:0/1495123141 shutdown_connections 2024-03-04T09:09:39.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.587+0000 7f4a139d7640 1 --2- 172.21.15.71:0/1495123141 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4a0c10bbe0 0x7f4a0c10e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.587+0000 7f4a139d7640 1 --2- 172.21.15.71:0/1495123141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c103130 0x7f4a0c10b610 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.587+0000 7f4a139d7640 1 --2- 172.21.15.71:0/1495123141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a0c102810 0x7f4a0c102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.587+0000 7f4a139d7640 1 -- 172.21.15.71:0/1495123141 >> 172.21.15.71:0/1495123141 conn(0x7f4a0c076850 msgr2=0x7f4a0c076c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:39.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.587+0000 7f4a139d7640 1 -- 172.21.15.71:0/1495123141 shutdown_connections 2024-03-04T09:09:39.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.587+0000 7f4a139d7640 1 -- 172.21.15.71:0/1495123141 wait complete. 2024-03-04T09:09:39.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.588+0000 7f4a139d7640 1 Processor -- start 2024-03-04T09:09:39.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.588+0000 7f4a139d7640 1 -- start start 2024-03-04T09:09:39.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.588+0000 7f4a139d7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c102810 0x7f4a0c197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:39.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.588+0000 7f4a139d7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a0c103130 0x7f4a0c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:39.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.588+0000 7f4a139d7640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4a0c10bbe0 0x7f4a0c19c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:39.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.588+0000 7f4a139d7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c111970 con 0x7f4a0c102810 2024-03-04T09:09:39.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.588+0000 7f4a139d7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c1117f0 con 0x7f4a0c103130 2024-03-04T09:09:39.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.588+0000 7f4a139d7640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c111af0 con 0x7f4a0c10bbe0 2024-03-04T09:09:39.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.588+0000 7f4a11773640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c102810 0x7f4a0c197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:39.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.588+0000 7f4a11773640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c102810 0x7f4a0c197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41212/0 (socket says 172.21.15.71:41212) 2024-03-04T09:09:39.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.588+0000 7f4a11773640 1 -- 172.21.15.71:0/201654691 learned_addr learned my addr 172.21.15.71:0/201654691 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T09:09:39.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.588+0000 7f4a10f72640 1 --2- 172.21.15.71:0/201654691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a0c103130 0x7f4a0c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:39.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.588+0000 7f4a11773640 1 -- 172.21.15.71:0/201654691 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4a0c10bbe0 msgr2=0x7f4a0c19c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:39.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.588+0000 7f4a11773640 1 --2- 172.21.15.71:0/201654691 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4a0c10bbe0 0x7f4a0c19c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.588+0000 7f4a11773640 1 -- 172.21.15.71:0/201654691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a0c103130 msgr2=0x7f4a0c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:39.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.588+0000 7f4a11773640 1 --2- 172.21.15.71:0/201654691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a0c103130 0x7f4a0c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.588+0000 7f4a11773640 1 -- 172.21.15.71:0/201654691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0c19ce60 con 0x7f4a0c102810 2024-03-04T09:09:39.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.588+0000 7f4a10f72640 1 --2- 172.21.15.71:0/201654691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a0c103130 0x7f4a0c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:09:39.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.588+0000 7f4a11773640 1 --2- 172.21.15.71:0/201654691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c102810 0x7f4a0c197e70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800e9f0 tx=0x7f4a0800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:39.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.588+0000 7f4a027fc640 1 -- 172.21.15.71:0/201654691 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0800cd80 con 0x7f4a0c102810 2024-03-04T09:09:39.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.588+0000 7f4a027fc640 1 -- 172.21.15.71:0/201654691 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a08010910 con 0x7f4a0c102810 2024-03-04T09:09:39.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.588+0000 7f4a139d7640 1 -- 172.21.15.71:0/201654691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0c19d150 con 0x7f4a0c102810 2024-03-04T09:09:39.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.589+0000 7f4a027fc640 1 -- 172.21.15.71:0/201654691 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a08018990 con 0x7f4a0c102810 2024-03-04T09:09:39.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.589+0000 7f4a139d7640 1 -- 172.21.15.71:0/201654691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0c1a49e0 con 0x7f4a0c102810 2024-03-04T09:09:39.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.590+0000 7f4a139d7640 1 -- 172.21.15.71:0/201654691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a0c06a3e0 con 0x7f4a0c102810 2024-03-04T09:09:39.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.591+0000 7f4a027fc640 1 -- 172.21.15.71:0/201654691 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f4a08010ac0 con 0x7f4a0c102810 2024-03-04T09:09:39.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.591+0000 7f4a027fc640 1 --2- 172.21.15.71:0/201654691 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f49f80a1ef0 0x7f49f80a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:39.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.591+0000 7f4a10f72640 1 --2- 172.21.15.71:0/201654691 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f49f80a1ef0 0x7f49f80a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:39.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.591+0000 7f4a027fc640 1 -- 172.21.15.71:0/201654691 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9467+0+0 (secure 0 0 0) 0x7f4a080cee10 con 0x7f4a0c102810 2024-03-04T09:09:39.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.592+0000 7f4a10f72640 1 --2- 172.21.15.71:0/201654691 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f49f80a1ef0 0x7f49f80a43b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0ffcd0 tx=0x7f49fc005a20 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:39.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.595+0000 7f4a027fc640 1 -- 172.21.15.71:0/201654691 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4a0809bb50 con 0x7f4a0c102810 2024-03-04T09:09:39.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.805+0000 7f4a139d7640 1 -- 172.21.15.71:0/201654691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4a0c068810 con 0x7f4a0c102810 2024-03-04T09:09:39.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.808+0000 7f4a027fc640 1 -- 172.21.15.71:0/201654691 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4a0809b450 con 0x7f4a0c102810 2024-03-04T09:09:39.804 INFO:teuthology.orchestra.run.smithi071.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-04T09:09:39.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.811+0000 7f4a139d7640 1 -- 172.21.15.71:0/201654691 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f49f80a1ef0 msgr2=0x7f49f80a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:39.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.811+0000 7f4a139d7640 1 --2- 172.21.15.71:0/201654691 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f49f80a1ef0 0x7f49f80a43b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0ffcd0 tx=0x7f49fc005a20 comp rx=0 tx=0).stop 2024-03-04T09:09:39.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.811+0000 7f4a139d7640 1 -- 172.21.15.71:0/201654691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c102810 msgr2=0x7f4a0c197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:39.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.811+0000 7f4a139d7640 1 --2- 172.21.15.71:0/201654691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c102810 0x7f4a0c197e70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800e9f0 tx=0x7f4a0800eeb0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.811+0000 7f4a139d7640 1 -- 172.21.15.71:0/201654691 shutdown_connections 2024-03-04T09:09:39.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.811+0000 7f4a139d7640 1 --2- 172.21.15.71:0/201654691 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f4a0c10bbe0 0x7f4a0c19c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.811+0000 7f4a139d7640 1 --2- 172.21.15.71:0/201654691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a0c103130 0x7f4a0c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.811+0000 7f4a139d7640 1 --2- 172.21.15.71:0/201654691 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f49f80a1ef0 0x7f49f80a43b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.811+0000 7f4a139d7640 1 --2- 172.21.15.71:0/201654691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c102810 0x7f4a0c197e70 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.811+0000 7f4a139d7640 1 -- 172.21.15.71:0/201654691 >> 172.21.15.71:0/201654691 conn(0x7f4a0c076850 msgr2=0x7f4a0c0fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:39.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.811+0000 7f4a139d7640 1 -- 172.21.15.71:0/201654691 shutdown_connections 2024-03-04T09:09:39.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.811+0000 7f4a139d7640 1 -- 172.21.15.71:0/201654691 wait complete. 2024-03-04T09:09:39.818 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-04T09:09:39.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.925+0000 7f94e7cf4640 1 -- 172.21.15.71:0/4202865143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94e01075d0 msgr2=0x7f94e01079b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:39.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.925+0000 7f94e7cf4640 1 --2- 172.21.15.71:0/4202865143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94e01075d0 0x7f94e01079b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009930 tx=0x7f94dc02bc30 comp rx=0 tx=0).stop 2024-03-04T09:09:39.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.926+0000 7f94e7cf4640 1 -- 172.21.15.71:0/4202865143 shutdown_connections 2024-03-04T09:09:39.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.926+0000 7f94e7cf4640 1 --2- 172.21.15.71:0/4202865143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94e0069190 0x7f94e010e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.926+0000 7f94e7cf4640 1 --2- 172.21.15.71:0/4202865143 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f94e00687d0 0x7f94e0068c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.926+0000 7f94e7cf4640 1 --2- 172.21.15.71:0/4202865143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94e01075d0 0x7f94e01079b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.926+0000 7f94e7cf4640 1 -- 172.21.15.71:0/4202865143 >> 172.21.15.71:0/4202865143 conn(0x7f94e00775d0 msgr2=0x7f94e00779e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:39.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.926+0000 7f94e7cf4640 1 -- 172.21.15.71:0/4202865143 shutdown_connections 2024-03-04T09:09:39.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.926+0000 7f94e7cf4640 1 -- 172.21.15.71:0/4202865143 wait complete. 2024-03-04T09:09:39.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.926+0000 7f94e7cf4640 1 Processor -- start 2024-03-04T09:09:39.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.926+0000 7f94e7cf4640 1 -- start start 2024-03-04T09:09:39.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.926+0000 7f94e7cf4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94e00687d0 0x7f94e0197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:39.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.926+0000 7f94e7cf4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94e0069190 0x7f94e0198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:39.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.926+0000 7f94e7cf4640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f94e01075d0 0x7f94e019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:39.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.926+0000 7f94e7cf4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e0111900 con 0x7f94e00687d0 2024-03-04T09:09:39.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.926+0000 7f94e7cf4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e0111780 con 0x7f94e0069190 2024-03-04T09:09:39.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.926+0000 7f94e7cf4640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e0111a80 con 0x7f94e01075d0 2024-03-04T09:09:39.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.926+0000 7f94e5a90640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94e00687d0 0x7f94e0197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:39.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.926+0000 7f94e5a90640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94e00687d0 0x7f94e0197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41242/0 (socket says 172.21.15.71:41242) 2024-03-04T09:09:39.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.926+0000 7f94e5a90640 1 -- 172.21.15.71:0/492959486 learned_addr learned my addr 172.21.15.71:0/492959486 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T09:09:39.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.927+0000 7f94e528f640 1 --2- 172.21.15.71:0/492959486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94e0069190 0x7f94e0198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:39.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.927+0000 7f94e6291640 1 --2- 172.21.15.71:0/492959486 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f94e01075d0 0x7f94e019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:39.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.927+0000 7f94e5a90640 1 -- 172.21.15.71:0/492959486 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f94e01075d0 msgr2=0x7f94e019c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:39.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.927+0000 7f94e5a90640 1 --2- 172.21.15.71:0/492959486 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f94e01075d0 0x7f94e019c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.927+0000 7f94e5a90640 1 -- 172.21.15.71:0/492959486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94e0069190 msgr2=0x7f94e0198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:39.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.927+0000 7f94e5a90640 1 --2- 172.21.15.71:0/492959486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94e0069190 0x7f94e0198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:39.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.927+0000 7f94e5a90640 1 -- 172.21.15.71:0/492959486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e019ce30 con 0x7f94e00687d0 2024-03-04T09:09:39.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.927+0000 7f94e5a90640 1 --2- 172.21.15.71:0/492959486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94e00687d0 0x7f94e0197e40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009230 tx=0x7f94dc02cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:39.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.927+0000 7f94d6ffd640 1 -- 172.21.15.71:0/492959486 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc041070 con 0x7f94e00687d0 2024-03-04T09:09:39.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.927+0000 7f94d6ffd640 1 -- 172.21.15.71:0/492959486 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94dc004450 con 0x7f94e00687d0 2024-03-04T09:09:39.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.927+0000 7f94e7cf4640 1 -- 172.21.15.71:0/492959486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e019d0c0 con 0x7f94e00687d0 2024-03-04T09:09:39.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.927+0000 7f94d6ffd640 1 -- 172.21.15.71:0/492959486 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc003e50 con 0x7f94e00687d0 2024-03-04T09:09:39.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.927+0000 7f94e7cf4640 1 -- 172.21.15.71:0/492959486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e01a49b0 con 0x7f94e00687d0 2024-03-04T09:09:39.928 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.928+0000 7f94e7cf4640 1 -- 172.21.15.71:0/492959486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e00683c0 con 0x7f94e00687d0 2024-03-04T09:09:39.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.929+0000 7f94d6ffd640 1 -- 172.21.15.71:0/492959486 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f94dc044070 con 0x7f94e00687d0 2024-03-04T09:09:39.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.929+0000 7f94d6ffd640 1 --2- 172.21.15.71:0/492959486 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f94cc0a2110 0x7f94cc0a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:39.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.930+0000 7f94e528f640 1 --2- 172.21.15.71:0/492959486 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f94cc0a2110 0x7f94cc0a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:39.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.930+0000 7f94d6ffd640 1 -- 172.21.15.71:0/492959486 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9465+0+0 (secure 0 0 0) 0x7f94dc0efc70 con 0x7f94e00687d0 2024-03-04T09:09:39.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.930+0000 7f94e528f640 1 --2- 172.21.15.71:0/492959486 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f94cc0a2110 0x7f94cc0a45d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f94e0106ca0 tx=0x7f94d0005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:39.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:39.934+0000 7f94d6ffd640 1 -- 172.21.15.71:0/492959486 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f94dc0bc470 con 0x7f94e00687d0 2024-03-04T09:09:40.094 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:39 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-a[32785]: 2024-03-04T09:09:39.806+0000 7fcac2e29700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-04T09:09:40.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.150+0000 7f94e7cf4640 1 -- 172.21.15.71:0/492959486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f94e0109a50 con 0x7f94e00687d0 2024-03-04T09:09:40.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:40 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2093207893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:09:40.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:40 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3588054210' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:09:40.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:40 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/201654691' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-04T09:09:40.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:40 smithi150 ceph-mon[36807]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-04T09:09:40.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:40 smithi150 ceph-mon[36807]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-04T09:09:40.280 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:40 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/201654691' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-04T09:09:40.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:40 smithi150 ceph-mon[36807]: osdmap e105: 12 total, 12 up, 12 in 2024-03-04T09:09:40.281 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:40 smithi150 ceph-mon[36807]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-03-04T09:09:40.281 INFO:journalctl@ceph.mds.b.smithi150.stdout:Mar 04 09:09:39 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-b[66660]: ignoring --setuser ceph since I am not root 2024-03-04T09:09:40.281 INFO:journalctl@ceph.mds.b.smithi150.stdout:Mar 04 09:09:39 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-b[66660]: ignoring --setgroup ceph since I am not root 2024-03-04T09:09:40.281 INFO:journalctl@ceph.mds.b.smithi150.stdout:Mar 04 09:09:39 smithi150 ceph-mds[66664]: -- 172.21.15.150:0/165367504 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b386311a40 con 0x55b3862e4800 2024-03-04T09:09:40.281 INFO:journalctl@ceph.mds.b.smithi150.stdout:Mar 04 09:09:39 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-b[66660]: starting mds.b at 2024-03-04T09:09:40.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:40 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2093207893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:09:40.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:40 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3588054210' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:09:40.302 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:40 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/201654691' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-04T09:09:40.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:40 smithi158 ceph-mon[37241]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-04T09:09:40.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:40 smithi158 ceph-mon[37241]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-04T09:09:40.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:40 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/201654691' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-04T09:09:40.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:40 smithi158 ceph-mon[37241]: osdmap e105: 12 total, 12 up, 12 in 2024-03-04T09:09:40.303 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:40 smithi158 ceph-mon[37241]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-03-04T09:09:40.303 INFO:journalctl@ceph.mds.i.smithi158.stdout:Mar 04 09:09:39 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-i[68666]: ignoring --setuser ceph since I am not root 2024-03-04T09:09:40.303 INFO:journalctl@ceph.mds.i.smithi158.stdout:Mar 04 09:09:39 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-i[68666]: ignoring --setgroup ceph since I am not root 2024-03-04T09:09:40.304 INFO:journalctl@ceph.mds.i.smithi158.stdout:Mar 04 09:09:39 smithi158 ceph-mds[68670]: -- 172.21.15.158:0/4017483268 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5608c48eda40 con 0x5608c48c0800 2024-03-04T09:09:40.304 INFO:journalctl@ceph.mds.i.smithi158.stdout:Mar 04 09:09:39 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-i[68666]: starting mds.i at 2024-03-04T09:09:40.357 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:40 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2093207893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:09:40.358 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:40 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3588054210' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:09:40.358 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:40 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/201654691' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-04T09:09:40.358 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:40 smithi071 ceph-mon[32789]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-04T09:09:40.358 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:40 smithi071 ceph-mon[32789]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-04T09:09:40.358 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:40 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/201654691' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-04T09:09:40.358 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:40 smithi071 ceph-mon[32789]: osdmap e105: 12 total, 12 up, 12 in 2024-03-04T09:09:40.358 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:40 smithi071 ceph-mon[32789]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-03-04T09:09:40.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.809+0000 7f94d6ffd640 1 -- 172.21.15.71:0/492959486 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) v1 ==== 104+0+0 (secure 0 0 0) 0x7f94dc0c0510 con 0x7f94e00687d0 2024-03-04T09:09:40.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.811+0000 7f94e7cf4640 1 -- 172.21.15.71:0/492959486 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f94cc0a2110 msgr2=0x7f94cc0a45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:40.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.811+0000 7f94e7cf4640 1 --2- 172.21.15.71:0/492959486 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f94cc0a2110 0x7f94cc0a45d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f94e0106ca0 tx=0x7f94d0005f90 comp rx=0 tx=0).stop 2024-03-04T09:09:40.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.811+0000 7f94e7cf4640 1 -- 172.21.15.71:0/492959486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94e00687d0 msgr2=0x7f94e0197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:40.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.811+0000 7f94e7cf4640 1 --2- 172.21.15.71:0/492959486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94e00687d0 0x7f94e0197e40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009230 tx=0x7f94dc02cb30 comp rx=0 tx=0).stop 2024-03-04T09:09:40.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.811+0000 7f94e7cf4640 1 -- 172.21.15.71:0/492959486 shutdown_connections 2024-03-04T09:09:40.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.811+0000 7f94e7cf4640 1 --2- 172.21.15.71:0/492959486 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7f94e01075d0 0x7f94e019c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:40.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.811+0000 7f94e7cf4640 1 --2- 172.21.15.71:0/492959486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94e0069190 0x7f94e0198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:40.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.811+0000 7f94e7cf4640 1 --2- 172.21.15.71:0/492959486 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7f94cc0a2110 0x7f94cc0a45d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:40.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.811+0000 7f94e7cf4640 1 --2- 172.21.15.71:0/492959486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94e00687d0 0x7f94e0197e40 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:40.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.811+0000 7f94e7cf4640 1 -- 172.21.15.71:0/492959486 >> 172.21.15.71:0/492959486 conn(0x7f94e00775d0 msgr2=0x7f94e00ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:40.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.812+0000 7f94e7cf4640 1 -- 172.21.15.71:0/492959486 shutdown_connections 2024-03-04T09:09:40.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.812+0000 7f94e7cf4640 1 -- 172.21.15.71:0/492959486 wait complete. 2024-03-04T09:09:40.820 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-04T09:09:40.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.932+0000 7fb816b1e640 1 -- 172.21.15.71:0/2361622932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb810100440 msgr2=0x7fb8101008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:40.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.932+0000 7fb816b1e640 1 --2- 172.21.15.71:0/2361622932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb810100440 0x7fb8101008c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009930 tx=0x7fb7fc02bc30 comp rx=0 tx=0).stop 2024-03-04T09:09:40.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.932+0000 7fb816b1e640 1 -- 172.21.15.71:0/2361622932 shutdown_connections 2024-03-04T09:09:40.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.932+0000 7fb816b1e640 1 --2- 172.21.15.71:0/2361622932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb810100e00 0x7fb81010d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:40.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.932+0000 7fb816b1e640 1 --2- 172.21.15.71:0/2361622932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb810100440 0x7fb8101008c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:40.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.932+0000 7fb816b1e640 1 --2- 172.21.15.71:0/2361622932 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8101064e0 0x7fb8101068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:40.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.932+0000 7fb816b1e640 1 -- 172.21.15.71:0/2361622932 >> 172.21.15.71:0/2361622932 conn(0x7fb8100775d0 msgr2=0x7fb8100779e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:40.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.932+0000 7fb816b1e640 1 -- 172.21.15.71:0/2361622932 shutdown_connections 2024-03-04T09:09:40.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.932+0000 7fb816b1e640 1 -- 172.21.15.71:0/2361622932 wait complete. 2024-03-04T09:09:40.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.933+0000 7fb816b1e640 1 Processor -- start 2024-03-04T09:09:40.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.933+0000 7fb816b1e640 1 -- start start 2024-03-04T09:09:40.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.933+0000 7fb816b1e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb810100440 0x7fb81019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:40.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.933+0000 7fb816b1e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb810100e00 0x7fb81019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:40.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.933+0000 7fb816b1e640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8101064e0 0x7fb81019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:40.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.933+0000 7fb816b1e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb810110800 con 0x7fb810100440 2024-03-04T09:09:40.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.933+0000 7fb816b1e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb810110680 con 0x7fb810100e00 2024-03-04T09:09:40.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.933+0000 7fb816b1e640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb810110980 con 0x7fb8101064e0 2024-03-04T09:09:40.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.933+0000 7fb8148ba640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb810100440 0x7fb81019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:40.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.933+0000 7fb8148ba640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb810100440 0x7fb81019a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41260/0 (socket says 172.21.15.71:41260) 2024-03-04T09:09:40.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.933+0000 7fb8148ba640 1 -- 172.21.15.71:0/3634756651 learned_addr learned my addr 172.21.15.71:0/3634756651 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T09:09:40.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.933+0000 7fb8150bb640 1 --2- 172.21.15.71:0/3634756651 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8101064e0 0x7fb81019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:40.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.933+0000 7fb8148ba640 1 -- 172.21.15.71:0/3634756651 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8101064e0 msgr2=0x7fb81019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:40.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.933+0000 7fb8148ba640 1 --2- 172.21.15.71:0/3634756651 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8101064e0 0x7fb81019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:40.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.933+0000 7fb80ffff640 1 --2- 172.21.15.71:0/3634756651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb810100e00 0x7fb81019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:40.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.933+0000 7fb8148ba640 1 -- 172.21.15.71:0/3634756651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb810100e00 msgr2=0x7fb81019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:40.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.934+0000 7fb8148ba640 1 --2- 172.21.15.71:0/3634756651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb810100e00 0x7fb81019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:40.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.934+0000 7fb8148ba640 1 -- 172.21.15.71:0/3634756651 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81019f050 con 0x7fb810100440 2024-03-04T09:09:40.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.934+0000 7fb8150bb640 1 --2- 172.21.15.71:0/3634756651 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8101064e0 0x7fb81019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T09:09:40.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.934+0000 7fb80ffff640 1 --2- 172.21.15.71:0/3634756651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb810100e00 0x7fb81019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T09:09:40.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.934+0000 7fb8148ba640 1 --2- 172.21.15.71:0/3634756651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb810100440 0x7fb81019a060 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb80400d770 tx=0x7fb80400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:40.934 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.934+0000 7fb80dffb640 1 -- 172.21.15.71:0/3634756651 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb804017070 con 0x7fb810100440 2024-03-04T09:09:40.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.934+0000 7fb80dffb640 1 -- 172.21.15.71:0/3634756651 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb804005500 con 0x7fb810100440 2024-03-04T09:09:40.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.934+0000 7fb816b1e640 1 -- 172.21.15.71:0/3634756651 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb81019f340 con 0x7fb810100440 2024-03-04T09:09:40.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.934+0000 7fb80dffb640 1 -- 172.21.15.71:0/3634756651 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb804004e00 con 0x7fb810100440 2024-03-04T09:09:40.935 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.934+0000 7fb816b1e640 1 -- 172.21.15.71:0/3634756651 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8101a6bd0 con 0x7fb810100440 2024-03-04T09:09:40.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.935+0000 7fb816b1e640 1 -- 172.21.15.71:0/3634756651 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8100683c0 con 0x7fb810100440 2024-03-04T09:09:40.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.936+0000 7fb80dffb640 1 -- 172.21.15.71:0/3634756651 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7fb804004fa0 con 0x7fb810100440 2024-03-04T09:09:40.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.936+0000 7fb80dffb640 1 --2- 172.21.15.71:0/3634756651 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fb8000a1ef0 0x7fb8000a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:40.936 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.937+0000 7fb80ffff640 1 --2- 172.21.15.71:0/3634756651 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fb8000a1ef0 0x7fb8000a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:40.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.937+0000 7fb80dffb640 1 -- 172.21.15.71:0/3634756651 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9465+0+0 (secure 0 0 0) 0x7fb8040cdfc0 con 0x7fb810100440 2024-03-04T09:09:40.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.937+0000 7fb80ffff640 1 --2- 172.21.15.71:0/3634756651 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fb8000a1ef0 0x7fb8000a43b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc036040 tx=0x7fb7fc000bc0 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:40.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:40.941+0000 7fb80dffb640 1 -- 172.21.15.71:0/3634756651 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb80409ad00 con 0x7fb810100440 2024-03-04T09:09:41.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:41.124+0000 7fb816b1e640 1 -- 172.21.15.71:0/3634756651 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb8101a6e00 con 0x7fb810100440 2024-03-04T09:09:41.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:41 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/492959486' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-04T09:09:41.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:41 smithi071 ceph-mon[32789]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-04T09:09:41.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:41 smithi071 ceph-mon[32789]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-04T09:09:41.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:41 smithi071 ceph-mon[32789]: Cluster is now healthy 2024-03-04T09:09:41.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:41 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/492959486' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-04T09:09:41.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:41 smithi071 ceph-mon[32789]: mds.? [v2:172.21.15.150:6838/682125122,v1:172.21.15.150:6839/682125122] up:boot 2024-03-04T09:09:41.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:41 smithi071 ceph-mon[32789]: mds.? [v2:172.21.15.158:6838/3953641452,v1:172.21.15.158:6839/3953641452] up:boot 2024-03-04T09:09:41.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:41 smithi071 ceph-mon[32789]: fsmap 12 up:standby 2024-03-04T09:09:41.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:41 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-04T09:09:41.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:41 smithi071 ceph-mon[32789]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-04T09:09:41.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:41 smithi071 ceph-mon[32789]: pgmap v335: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 307 B/s wr, 2 op/s 2024-03-04T09:09:41.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:41 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/492959486' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-04T09:09:41.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:41 smithi150 ceph-mon[36807]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-04T09:09:41.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:41 smithi150 ceph-mon[36807]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-04T09:09:41.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:41 smithi150 ceph-mon[36807]: Cluster is now healthy 2024-03-04T09:09:41.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:41 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/492959486' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-04T09:09:41.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:41 smithi150 ceph-mon[36807]: mds.? [v2:172.21.15.150:6838/682125122,v1:172.21.15.150:6839/682125122] up:boot 2024-03-04T09:09:41.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:41 smithi150 ceph-mon[36807]: mds.? [v2:172.21.15.158:6838/3953641452,v1:172.21.15.158:6839/3953641452] up:boot 2024-03-04T09:09:41.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:41 smithi150 ceph-mon[36807]: fsmap 12 up:standby 2024-03-04T09:09:41.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:41 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-04T09:09:41.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:41 smithi150 ceph-mon[36807]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-04T09:09:41.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:41 smithi150 ceph-mon[36807]: pgmap v335: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 307 B/s wr, 2 op/s 2024-03-04T09:09:41.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:41 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/492959486' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-04T09:09:41.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:41 smithi158 ceph-mon[37241]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-04T09:09:41.552 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:41 smithi158 ceph-mon[37241]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-04T09:09:41.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:41 smithi158 ceph-mon[37241]: Cluster is now healthy 2024-03-04T09:09:41.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:41 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/492959486' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-04T09:09:41.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:41 smithi158 ceph-mon[37241]: mds.? [v2:172.21.15.150:6838/682125122,v1:172.21.15.150:6839/682125122] up:boot 2024-03-04T09:09:41.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:41 smithi158 ceph-mon[37241]: mds.? [v2:172.21.15.158:6838/3953641452,v1:172.21.15.158:6839/3953641452] up:boot 2024-03-04T09:09:41.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:41 smithi158 ceph-mon[37241]: fsmap 12 up:standby 2024-03-04T09:09:41.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:41 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-04T09:09:41.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:41 smithi158 ceph-mon[37241]: from='mgr.15399 172.21.15.71:0/2344241147' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-04T09:09:41.553 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:41 smithi158 ceph-mon[37241]: pgmap v335: 129 pgs: 129 active+clean; 418 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 307 B/s wr, 2 op/s 2024-03-04T09:09:42.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.112+0000 7fb80dffb640 1 -- 172.21.15.71:0/3634756651 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v106) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb80409a600 con 0x7fb810100440 2024-03-04T09:09:42.109 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' removed 2024-03-04T09:09:42.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.115+0000 7fb816b1e640 1 -- 172.21.15.71:0/3634756651 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fb8000a1ef0 msgr2=0x7fb8000a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:42.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.115+0000 7fb816b1e640 1 --2- 172.21.15.71:0/3634756651 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fb8000a1ef0 0x7fb8000a43b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc036040 tx=0x7fb7fc000bc0 comp rx=0 tx=0).stop 2024-03-04T09:09:42.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.115+0000 7fb816b1e640 1 -- 172.21.15.71:0/3634756651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb810100440 msgr2=0x7fb81019a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:42.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.115+0000 7fb816b1e640 1 --2- 172.21.15.71:0/3634756651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb810100440 0x7fb81019a060 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb80400d770 tx=0x7fb80400dc30 comp rx=0 tx=0).stop 2024-03-04T09:09:42.111 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.115+0000 7fb816b1e640 1 -- 172.21.15.71:0/3634756651 shutdown_connections 2024-03-04T09:09:42.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.115+0000 7fb816b1e640 1 --2- 172.21.15.71:0/3634756651 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb8101064e0 0x7fb81019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:42.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.115+0000 7fb816b1e640 1 --2- 172.21.15.71:0/3634756651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb810100e00 0x7fb81019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:42.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.115+0000 7fb816b1e640 1 --2- 172.21.15.71:0/3634756651 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fb8000a1ef0 0x7fb8000a43b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:42.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.115+0000 7fb816b1e640 1 --2- 172.21.15.71:0/3634756651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb810100440 0x7fb81019a060 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:42.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.115+0000 7fb816b1e640 1 -- 172.21.15.71:0/3634756651 >> 172.21.15.71:0/3634756651 conn(0x7fb8100775d0 msgr2=0x7fb810068730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:42.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.115+0000 7fb816b1e640 1 -- 172.21.15.71:0/3634756651 shutdown_connections 2024-03-04T09:09:42.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.115+0000 7fb816b1e640 1 -- 172.21.15.71:0/3634756651 wait complete. 2024-03-04T09:09:42.126 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-04T09:09:42.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.239+0000 7fb1033e9640 1 -- 172.21.15.71:0/588768831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0fc100440 msgr2=0x7fb0fc1008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:42.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.239+0000 7fb1033e9640 1 --2- 172.21.15.71:0/588768831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0fc100440 0x7fb0fc1008c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009930 tx=0x7fb0f002bc30 comp rx=0 tx=0).stop 2024-03-04T09:09:42.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.239+0000 7fb1033e9640 1 -- 172.21.15.71:0/588768831 shutdown_connections 2024-03-04T09:09:42.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.239+0000 7fb1033e9640 1 --2- 172.21.15.71:0/588768831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0fc100e00 0x7fb0fc10d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:42.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.239+0000 7fb1033e9640 1 --2- 172.21.15.71:0/588768831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0fc100440 0x7fb0fc1008c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:42.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.239+0000 7fb1033e9640 1 --2- 172.21.15.71:0/588768831 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb0fc1064e0 0x7fb0fc1068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:42.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.239+0000 7fb1033e9640 1 -- 172.21.15.71:0/588768831 >> 172.21.15.71:0/588768831 conn(0x7fb0fc0775d0 msgr2=0x7fb0fc0779e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:42.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.239+0000 7fb1033e9640 1 -- 172.21.15.71:0/588768831 shutdown_connections 2024-03-04T09:09:42.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.239+0000 7fb1033e9640 1 -- 172.21.15.71:0/588768831 wait complete. 2024-03-04T09:09:42.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.239+0000 7fb1033e9640 1 Processor -- start 2024-03-04T09:09:42.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.239+0000 7fb1033e9640 1 -- start start 2024-03-04T09:09:42.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.239+0000 7fb1033e9640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb0fc100440 0x7fb0fc198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:42.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.239+0000 7fb1033e9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0fc100e00 0x7fb0fc1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:42.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.239+0000 7fb1033e9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0fc1064e0 0x7fb0fc1a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:42.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.239+0000 7fb1033e9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc110800 con 0x7fb0fc100e00 2024-03-04T09:09:42.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.239+0000 7fb1033e9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc110680 con 0x7fb0fc1064e0 2024-03-04T09:09:42.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.239+0000 7fb1033e9640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc110980 con 0x7fb0fc100440 2024-03-04T09:09:42.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.240+0000 7fb100984640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0fc100e00 0x7fb0fc1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:42.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.240+0000 7fb100984640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0fc100e00 0x7fb0fc1991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41272/0 (socket says 172.21.15.71:41272) 2024-03-04T09:09:42.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.240+0000 7fb100984640 1 -- 172.21.15.71:0/1053432533 learned_addr learned my addr 172.21.15.71:0/1053432533 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T09:09:42.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.240+0000 7fb101185640 1 --2- 172.21.15.71:0/1053432533 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb0fc100440 0x7fb0fc198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:42.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.240+0000 7fb100984640 1 -- 172.21.15.71:0/1053432533 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb0fc100440 msgr2=0x7fb0fc198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:42.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.240+0000 7fb100984640 1 --2- 172.21.15.71:0/1053432533 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb0fc100440 0x7fb0fc198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:42.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.240+0000 7fb100984640 1 -- 172.21.15.71:0/1053432533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0fc1064e0 msgr2=0x7fb0fc1a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:42.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.240+0000 7fb101986640 1 --2- 172.21.15.71:0/1053432533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0fc1064e0 0x7fb0fc1a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:42.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.240+0000 7fb100984640 1 --2- 172.21.15.71:0/1053432533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0fc1064e0 0x7fb0fc1a3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:42.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.240+0000 7fb100984640 1 -- 172.21.15.71:0/1053432533 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0fc1a4240 con 0x7fb0fc100e00 2024-03-04T09:09:42.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.240+0000 7fb100984640 1 --2- 172.21.15.71:0/1053432533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0fc100e00 0x7fb0fc1991c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009230 tx=0x7fb0f002cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:42.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.240+0000 7fb0fa7fc640 1 -- 172.21.15.71:0/1053432533 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0041070 con 0x7fb0fc100e00 2024-03-04T09:09:42.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.240+0000 7fb0fa7fc640 1 -- 172.21.15.71:0/1053432533 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0f0004450 con 0x7fb0fc100e00 2024-03-04T09:09:42.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.240+0000 7fb1033e9640 1 -- 172.21.15.71:0/1053432533 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0fc1a4410 con 0x7fb0fc100e00 2024-03-04T09:09:42.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.240+0000 7fb0fa7fc640 1 -- 172.21.15.71:0/1053432533 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0034440 con 0x7fb0fc100e00 2024-03-04T09:09:42.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.240+0000 7fb1033e9640 1 -- 172.21.15.71:0/1053432533 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0fc1a4870 con 0x7fb0fc100e00 2024-03-04T09:09:42.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.241+0000 7fb1033e9640 1 -- 172.21.15.71:0/1053432533 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0fc0683c0 con 0x7fb0fc100e00 2024-03-04T09:09:42.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.242+0000 7fb0fa7fc640 1 -- 172.21.15.71:0/1053432533 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7fb0f0044070 con 0x7fb0fc100e00 2024-03-04T09:09:42.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.243+0000 7fb0fa7fc640 1 --2- 172.21.15.71:0/1053432533 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fb0e40a1ef0 0x7fb0e40a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:42.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.243+0000 7fb0fa7fc640 1 -- 172.21.15.71:0/1053432533 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8911+0+0 (secure 0 0 0) 0x7fb0f00efd10 con 0x7fb0fc100e00 2024-03-04T09:09:42.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.243+0000 7fb101185640 1 --2- 172.21.15.71:0/1053432533 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fb0e40a1ef0 0x7fb0e40a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:42.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.243+0000 7fb101185640 1 --2- 172.21.15.71:0/1053432533 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fb0e40a1ef0 0x7fb0e40a43b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4005fd0 tx=0x7fb0f400b390 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:42.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.247+0000 7fb0fa7fc640 1 -- 172.21.15.71:0/1053432533 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb0f00bcd90 con 0x7fb0fc100e00 2024-03-04T09:09:42.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:42.422+0000 7fb1033e9640 1 -- 172.21.15.71:0/1053432533 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb0fc068d20 con 0x7fb0fc100e00 2024-03-04T09:09:42.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:42 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3634756651' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T09:09:42.499 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:42 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3634756651' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T09:09:42.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:42 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3634756651' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T09:09:43.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.117+0000 7fb0fa7fc640 1 -- 172.21.15.71:0/1053432533 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v107) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb0f00bc690 con 0x7fb0fc100e00 2024-03-04T09:09:43.113 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' removed 2024-03-04T09:09:43.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.120+0000 7fb1033e9640 1 -- 172.21.15.71:0/1053432533 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fb0e40a1ef0 msgr2=0x7fb0e40a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:43.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.120+0000 7fb1033e9640 1 --2- 172.21.15.71:0/1053432533 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fb0e40a1ef0 0x7fb0e40a43b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4005fd0 tx=0x7fb0f400b390 comp rx=0 tx=0).stop 2024-03-04T09:09:43.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.120+0000 7fb1033e9640 1 -- 172.21.15.71:0/1053432533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0fc100e00 msgr2=0x7fb0fc1991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:43.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.120+0000 7fb1033e9640 1 --2- 172.21.15.71:0/1053432533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0fc100e00 0x7fb0fc1991c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009230 tx=0x7fb0f002cc70 comp rx=0 tx=0).stop 2024-03-04T09:09:43.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.120+0000 7fb1033e9640 1 -- 172.21.15.71:0/1053432533 shutdown_connections 2024-03-04T09:09:43.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.120+0000 7fb1033e9640 1 --2- 172.21.15.71:0/1053432533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0fc1064e0 0x7fb0fc1a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:43.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.120+0000 7fb1033e9640 1 --2- 172.21.15.71:0/1053432533 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fb0e40a1ef0 0x7fb0e40a43b0 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4005fd0 tx=0x7fb0f400b390 comp rx=0 tx=0).stop 2024-03-04T09:09:43.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.120+0000 7fb1033e9640 1 --2- 172.21.15.71:0/1053432533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0fc100e00 0x7fb0fc1991c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:43.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.120+0000 7fb1033e9640 1 --2- 172.21.15.71:0/1053432533 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fb0fc100440 0x7fb0fc198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:43.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.120+0000 7fb1033e9640 1 -- 172.21.15.71:0/1053432533 >> 172.21.15.71:0/1053432533 conn(0x7fb0fc0775d0 msgr2=0x7fb0fc102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:43.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.121+0000 7fb1033e9640 1 -- 172.21.15.71:0/1053432533 shutdown_connections 2024-03-04T09:09:43.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.121+0000 7fb1033e9640 1 -- 172.21.15.71:0/1053432533 wait complete. 2024-03-04T09:09:43.131 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-04T09:09:43.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.242+0000 7fdce17e2640 1 -- 172.21.15.71:0/3673911227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcdc101580 msgr2=0x7fdcdc101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:43.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.242+0000 7fdce17e2640 1 --2- 172.21.15.71:0/3673911227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcdc101580 0x7fdcdc101a00 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009900 tx=0x7fdcd002bc30 comp rx=0 tx=0).stop 2024-03-04T09:09:43.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.242+0000 7fdce17e2640 1 -- 172.21.15.71:0/3673911227 shutdown_connections 2024-03-04T09:09:43.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.242+0000 7fdce17e2640 1 --2- 172.21.15.71:0/3673911227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcdc101f40 0x7fdcdc10e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:43.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.242+0000 7fdce17e2640 1 --2- 172.21.15.71:0/3673911227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcdc101580 0x7fdcdc101a00 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:43.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.242+0000 7fdce17e2640 1 --2- 172.21.15.71:0/3673911227 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdcdc107640 0x7fdcdc107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:43.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.242+0000 7fdce17e2640 1 -- 172.21.15.71:0/3673911227 >> 172.21.15.71:0/3673911227 conn(0x7fdcdc0789d0 msgr2=0x7fdcdc078de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:43.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.242+0000 7fdce17e2640 1 -- 172.21.15.71:0/3673911227 shutdown_connections 2024-03-04T09:09:43.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.242+0000 7fdce17e2640 1 -- 172.21.15.71:0/3673911227 wait complete. 2024-03-04T09:09:43.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.243+0000 7fdce17e2640 1 Processor -- start 2024-03-04T09:09:43.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.243+0000 7fdce17e2640 1 -- start start 2024-03-04T09:09:43.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.243+0000 7fdce17e2640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdcdc101580 0x7fdcdc197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:43.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.243+0000 7fdce17e2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcdc101f40 0x7fdcdc198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:43.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.243+0000 7fdce17e2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcdc107640 0x7fdcdc19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:43.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.243+0000 7fdce17e2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc111900 con 0x7fdcdc107640 2024-03-04T09:09:43.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.243+0000 7fdce17e2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc111780 con 0x7fdcdc101f40 2024-03-04T09:09:43.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.243+0000 7fdce17e2640 1 -- --> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc111a80 con 0x7fdcdc101580 2024-03-04T09:09:43.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.243+0000 7fdcdb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcdc107640 0x7fdcdc19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:43.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.243+0000 7fdcdb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcdc107640 0x7fdcdc19c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41290/0 (socket says 172.21.15.71:41290) 2024-03-04T09:09:43.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.243+0000 7fdcdb7fe640 1 -- 172.21.15.71:0/2895757755 learned_addr learned my addr 172.21.15.71:0/2895757755 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-03-04T09:09:43.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.243+0000 7fdcdaffd640 1 --2- >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdcdc101580 0x7fdcdc197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:43.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.244+0000 7fdcda7fc640 1 --2- 172.21.15.71:0/2895757755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcdc101f40 0x7fdcdc198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:43.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.244+0000 7fdcdb7fe640 1 -- 172.21.15.71:0/2895757755 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdcdc101580 msgr2=0x7fdcdc197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:43.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.244+0000 7fdcdb7fe640 1 --2- 172.21.15.71:0/2895757755 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdcdc101580 0x7fdcdc197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:43.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.244+0000 7fdcdb7fe640 1 -- 172.21.15.71:0/2895757755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcdc101f40 msgr2=0x7fdcdc198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:43.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.244+0000 7fdcdb7fe640 1 --2- 172.21.15.71:0/2895757755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcdc101f40 0x7fdcdc198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:43.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.244+0000 7fdcdb7fe640 1 -- 172.21.15.71:0/2895757755 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc19cdb0 con 0x7fdcdc107640 2024-03-04T09:09:43.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.244+0000 7fdcdaffd640 1 --2- 172.21.15.71:0/2895757755 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdcdc101580 0x7fdcdc197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:09:43.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.244+0000 7fdcdb7fe640 1 --2- 172.21.15.71:0/2895757755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcdc107640 0x7fdcdc19c680 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400eaa0 tx=0x7fdcd400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:43.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.244+0000 7fdcc3fff640 1 -- 172.21.15.71:0/2895757755 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd400cdb0 con 0x7fdcdc107640 2024-03-04T09:09:43.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.244+0000 7fdcc3fff640 1 -- 172.21.15.71:0/2895757755 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdcd4005020 con 0x7fdcdc107640 2024-03-04T09:09:43.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.244+0000 7fdce17e2640 1 -- 172.21.15.71:0/2895757755 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcdc19d0a0 con 0x7fdcdc107640 2024-03-04T09:09:43.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.244+0000 7fdcc3fff640 1 -- 172.21.15.71:0/2895757755 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd40116b0 con 0x7fdcdc107640 2024-03-04T09:09:43.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.244+0000 7fdce17e2640 1 -- 172.21.15.71:0/2895757755 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcdc1a4930 con 0x7fdcdc107640 2024-03-04T09:09:43.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.245+0000 7fdce17e2640 1 -- 172.21.15.71:0/2895757755 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcdc068810 con 0x7fdcdc107640 2024-03-04T09:09:43.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.246+0000 7fdcc3fff640 1 -- 172.21.15.71:0/2895757755 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7fdcd4005560 con 0x7fdcdc107640 2024-03-04T09:09:43.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.246+0000 7fdcc3fff640 1 --2- 172.21.15.71:0/2895757755 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fdcc40a1ef0 0x7fdcc40a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:09:43.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.247+0000 7fdcdaffd640 1 --2- 172.21.15.71:0/2895757755 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fdcc40a1ef0 0x7fdcc40a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:09:43.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.247+0000 7fdcc3fff640 1 -- 172.21.15.71:0/2895757755 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8410+0+0 (secure 0 0 0) 0x7fdcd40ce2e0 con 0x7fdcdc107640 2024-03-04T09:09:43.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.247+0000 7fdcdaffd640 1 --2- 172.21.15.71:0/2895757755 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fdcc40a1ef0 0x7fdcc40a43b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdccc009fc0 tx=0x7fdccc005f90 comp rx=0 tx=0).ready entity=mgr.15399 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:09:43.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.251+0000 7fdcc3fff640 1 -- 172.21.15.71:0/2895757755 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdcd4016030 con 0x7fdcdc107640 2024-03-04T09:09:43.424 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:43 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/3634756651' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-04T09:09:43.424 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:43 smithi071 ceph-mon[32789]: osdmap e106: 12 total, 12 up, 12 in 2024-03-04T09:09:43.424 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:43 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1053432533' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T09:09:43.424 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:43 smithi071 ceph-mon[32789]: pgmap v337: 97 pgs: 97 active+clean; 25 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:09:43.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:43.429+0000 7fdce17e2640 1 -- 172.21.15.71:0/2895757755 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdcdc02cff0 con 0x7fdcdc107640 2024-03-04T09:09:43.447 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:43 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/3634756651' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-04T09:09:43.448 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:43 smithi158 ceph-mon[37241]: osdmap e106: 12 total, 12 up, 12 in 2024-03-04T09:09:43.448 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:43 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1053432533' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T09:09:43.448 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:43 smithi158 ceph-mon[37241]: pgmap v337: 97 pgs: 97 active+clean; 25 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:09:43.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:43 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/3634756651' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-04T09:09:43.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:43 smithi150 ceph-mon[36807]: osdmap e106: 12 total, 12 up, 12 in 2024-03-04T09:09:43.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:43 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1053432533' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T09:09:43.531 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:43 smithi150 ceph-mon[36807]: pgmap v337: 97 pgs: 97 active+clean; 25 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:09:44.115 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:44.119+0000 7fdcc3fff640 1 -- 172.21.15.71:0/2895757755 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v108) v1 ==== 178+0+0 (secure 0 0 0) 0x7fdcd409adf0 con 0x7fdcdc107640 2024-03-04T09:09:44.115 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data_ec' removed 2024-03-04T09:09:44.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:44.122+0000 7fdce17e2640 1 -- 172.21.15.71:0/2895757755 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fdcc40a1ef0 msgr2=0x7fdcc40a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:44.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:44.122+0000 7fdce17e2640 1 --2- 172.21.15.71:0/2895757755 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fdcc40a1ef0 0x7fdcc40a43b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdccc009fc0 tx=0x7fdccc005f90 comp rx=0 tx=0).stop 2024-03-04T09:09:44.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:44.122+0000 7fdce17e2640 1 -- 172.21.15.71:0/2895757755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcdc107640 msgr2=0x7fdcdc19c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:09:44.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:44.122+0000 7fdce17e2640 1 --2- 172.21.15.71:0/2895757755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcdc107640 0x7fdcdc19c680 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400eaa0 tx=0x7fdcd400ef60 comp rx=0 tx=0).stop 2024-03-04T09:09:44.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:44.122+0000 7fdce17e2640 1 -- 172.21.15.71:0/2895757755 shutdown_connections 2024-03-04T09:09:44.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:44.122+0000 7fdce17e2640 1 --2- 172.21.15.71:0/2895757755 >> [v2:172.21.15.71:6800/217968316,v1:172.21.15.71:6801/217968316] conn(0x7fdcc40a1ef0 0x7fdcc40a43b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:44.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:44.122+0000 7fdce17e2640 1 --2- 172.21.15.71:0/2895757755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcdc107640 0x7fdcdc19c680 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:44.119 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:44.122+0000 7fdce17e2640 1 --2- 172.21.15.71:0/2895757755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcdc101f40 0x7fdcdc198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:44.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:44.122+0000 7fdce17e2640 1 --2- 172.21.15.71:0/2895757755 >> [v2:172.21.15.158:3300/0,v1:172.21.15.158:6789/0] conn(0x7fdcdc101580 0x7fdcdc197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:09:44.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:44.122+0000 7fdce17e2640 1 -- 172.21.15.71:0/2895757755 >> 172.21.15.71:0/2895757755 conn(0x7fdcdc0789d0 msgr2=0x7fdcdc100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:09:44.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:44.123+0000 7fdce17e2640 1 -- 172.21.15.71:0/2895757755 shutdown_connections 2024-03-04T09:09:44.120 INFO:teuthology.orchestra.run.smithi071.stderr:2024-03-04T09:09:44.123+0000 7fdce17e2640 1 -- 172.21.15.71:0/2895757755 wait complete. 2024-03-04T09:09:44.135 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-04T09:09:44.162 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-04T09:09:44.198 DEBUG:teuthology.orchestra.run.smithi158:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-04T09:09:44.227 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:44 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/1053432533' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-04T09:09:44.227 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:44 smithi158 ceph-mon[37241]: osdmap e107: 12 total, 12 up, 12 in 2024-03-04T09:09:44.227 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:44 smithi158 ceph-mon[37241]: from='client.? 172.21.15.71:0/2895757755' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T09:09:44.230 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-04T09:09:44.231 DEBUG:teuthology.orchestra.run.smithi071:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-04T09:09:44.246 DEBUG:teuthology.orchestra.run.smithi150:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-04T09:09:44.262 DEBUG:teuthology.orchestra.run.smithi158:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-04T09:09:44.286 INFO:tasks.cephadm:Stopping all daemons... 2024-03-04T09:09:44.286 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-04T09:09:44.286 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mon.a 2024-03-04T09:09:44.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:44 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/1053432533' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-04T09:09:44.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:44 smithi071 ceph-mon[32789]: osdmap e107: 12 total, 12 up, 12 in 2024-03-04T09:09:44.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:44 smithi071 ceph-mon[32789]: from='client.? 172.21.15.71:0/2895757755' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T09:09:44.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:44 smithi071 systemd[1]: Stopping Ceph mon.a for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:09:44.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:44 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/1053432533' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-04T09:09:44.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:44 smithi150 ceph-mon[36807]: osdmap e107: 12 total, 12 up, 12 in 2024-03-04T09:09:44.530 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:44 smithi150 ceph-mon[36807]: from='client.? 172.21.15.71:0/2895757755' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T09:09:44.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:44 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-a[32785]: 2024-03-04T09:09:44.589+0000 7fcac8634700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:09:44.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:44 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-a[32785]: 2024-03-04T09:09:44.589+0000 7fcac8634700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-04T09:09:44.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:44 smithi071 podman[92259]: 2024-03-04 09:09:44.784434843 +0000 UTC m=+0.310271523 container died c1582a68dbdaef2eeb47857e48d2011d0c4578c625f6ad27d898f5c5c8f3d9ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-a, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-04T09:09:45.421 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:45 smithi071 podman[92259]: 2024-03-04 09:09:45.241358887 +0000 UTC m=+0.767195566 container cleanup c1582a68dbdaef2eeb47857e48d2011d0c4578c625f6ad27d898f5c5c8f3d9ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-04T09:09:45.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Mar 04 09:09:45 smithi071 bash[92259]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-a 2024-03-04T09:09:45.707 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mon.a.service' 2024-03-04T09:09:45.757 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:09:45.758 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-04T09:09:45.758 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-04T09:09:45.758 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mon.b 2024-03-04T09:09:46.086 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:45 smithi150 systemd[1]: Stopping Ceph mon.b for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:09:46.445 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:46 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-b[36803]: 2024-03-04T09:09:46.089+0000 7f3c8b06e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:09:46.445 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:46 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-b[36803]: 2024-03-04T09:09:46.089+0000 7f3c8b06e700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-04T09:09:46.445 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:46 smithi150 podman[72114]: 2024-03-04 09:09:46.110763591 +0000 UTC m=+0.145224961 container died ad4f4fe1bfd01bc8bc5b70f8a5a287e5bd52aef1dc7185e3f4a9950e16522c29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-b, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:09:46.763 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:46 smithi150 podman[72114]: 2024-03-04 09:09:46.532224812 +0000 UTC m=+0.566686173 container cleanup ad4f4fe1bfd01bc8bc5b70f8a5a287e5bd52aef1dc7185e3f4a9950e16522c29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-b, ceph=True, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-04T09:09:46.763 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:46 smithi150 bash[72114]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-b 2024-03-04T09:09:46.987 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mon.b.service' 2024-03-04T09:09:47.013 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:46 smithi150 podman[72128]: 2024-03-04 09:09:46.767348603 +0000 UTC m=+0.659481840 container remove ad4f4fe1bfd01bc8bc5b70f8a5a287e5bd52aef1dc7185e3f4a9950e16522c29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-b, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:09:47.013 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:46 smithi150 systemd[1]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mon.b.service: Deactivated successfully. 2024-03-04T09:09:47.013 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:46 smithi150 systemd[1]: Stopped Ceph mon.b for b3c85c40-da03-11ee-95c6-87774f69a715. 2024-03-04T09:09:47.013 INFO:journalctl@ceph.mon.b.smithi150.stdout:Mar 04 09:09:46 smithi150 systemd[1]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mon.b.service: Consumed 14.165s CPU time. 2024-03-04T09:09:47.033 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:09:47.034 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-04T09:09:47.034 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-04T09:09:47.034 DEBUG:teuthology.orchestra.run.smithi158:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mon.c 2024-03-04T09:09:47.353 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:47 smithi158 systemd[1]: Stopping Ceph mon.c for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:09:47.688 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:47 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-c[37224]: 2024-03-04T09:09:47.356+0000 7f99bc588700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:09:47.688 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:47 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-c[37224]: 2024-03-04T09:09:47.356+0000 7f99bc588700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-04T09:09:47.688 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:47 smithi158 podman[74440]: 2024-03-04 09:09:47.408094199 +0000 UTC m=+0.176721066 container died 78dd30fa6ca723f20dd601f927fd348f9d18406e2ebb15468bcba74089501d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-03-04T09:09:48.047 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:47 smithi158 podman[74440]: 2024-03-04 09:09:47.783561417 +0000 UTC m=+0.552188264 container cleanup 78dd30fa6ca723f20dd601f927fd348f9d18406e2ebb15468bcba74089501d59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-04T09:09:48.048 INFO:journalctl@ceph.mon.c.smithi158.stdout:Mar 04 09:09:47 smithi158 bash[74440]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mon-c 2024-03-04T09:09:48.151 DEBUG:teuthology.orchestra.run.smithi158:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mon.c.service' 2024-03-04T09:09:48.203 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:09:48.203 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-04T09:09:48.204 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-04T09:09:48.204 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mgr.x 2024-03-04T09:09:48.524 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 09:09:48 smithi071 systemd[1]: Stopping Ceph mgr.x for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:09:48.825 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 09:09:48 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T09:09:48.529+0000 7fa73c17b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:09:48.825 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 09:09:48 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x[33018]: 2024-03-04T09:09:48.529+0000 7fa73c17b700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-04T09:09:48.825 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 09:09:48 smithi071 podman[92379]: 2024-03-04 09:09:48.56109543 +0000 UTC m=+0.155487440 container died ba61387fd7a74c3136ce81a40d45ecb0a69bf18a1bd9a8d36b64da68c6ba3452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:09:49.171 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 09:09:48 smithi071 podman[92379]: 2024-03-04 09:09:48.937073623 +0000 UTC m=+0.531465631 container cleanup ba61387fd7a74c3136ce81a40d45ecb0a69bf18a1bd9a8d36b64da68c6ba3452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-04T09:09:49.172 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Mar 04 09:09:48 smithi071 bash[92379]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-x 2024-03-04T09:09:49.359 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mgr.x.service' 2024-03-04T09:09:49.396 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:09:49.396 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-04T09:09:49.396 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-04T09:09:49.396 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mgr.y 2024-03-04T09:09:49.754 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 09:09:49 smithi150 systemd[1]: Stopping Ceph mgr.y for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:09:50.030 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 09:09:49 smithi150 podman[72239]: 2024-03-04 09:09:49.758402528 +0000 UTC m=+0.171440428 container died 39baab92d66cee74f7b95e4b036bc8bdeb7b996f314addd358c01b65fbad5a67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, RELEASE=wip-lusov-qdb-1bc250c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-04T09:09:50.502 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 09:09:50 smithi150 podman[72239]: 2024-03-04 09:09:50.163079763 +0000 UTC m=+0.576117657 container cleanup 39baab92d66cee74f7b95e4b036bc8bdeb7b996f314addd358c01b65fbad5a67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-04T09:09:50.502 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Mar 04 09:09:50 smithi150 bash[72239]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-y 2024-03-04T09:09:50.716 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mgr.y.service' 2024-03-04T09:09:50.754 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:09:50.754 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-04T09:09:50.754 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-04T09:09:50.754 DEBUG:teuthology.orchestra.run.smithi158:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mgr.z 2024-03-04T09:09:51.052 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 09:09:50 smithi158 systemd[1]: Stopping Ceph mgr.z for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:09:51.331 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 09:09:51 smithi158 podman[74564]: 2024-03-04 09:09:51.068566961 +0000 UTC m=+0.120763266 container died 4f4b699bb6120cc86b0423567dbe9d5fac28ceb3d1b2c2a982564f4fc041ce7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:09:51.698 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 09:09:51 smithi158 podman[74564]: 2024-03-04 09:09:51.43514644 +0000 UTC m=+0.487342770 container cleanup 4f4b699bb6120cc86b0423567dbe9d5fac28ceb3d1b2c2a982564f4fc041ce7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-03-04T09:09:51.699 INFO:journalctl@ceph.mgr.z.smithi158.stdout:Mar 04 09:09:51 smithi158 bash[74564]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mgr-z 2024-03-04T09:09:51.841 DEBUG:teuthology.orchestra.run.smithi158:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mgr.z.service' 2024-03-04T09:09:51.879 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:09:51.879 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-04T09:09:51.879 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-04T09:09:51.879 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.0 2024-03-04T09:09:52.171 INFO:journalctl@ceph.osd.0.smithi071.stdout:Mar 04 09:09:51 smithi071 systemd[1]: Stopping Ceph osd.0 for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:09:52.671 INFO:journalctl@ceph.osd.0.smithi071.stdout:Mar 04 09:09:52 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-0[47571]: 2024-03-04T09:09:52.191+0000 7f1ecfefe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:09:52.672 INFO:journalctl@ceph.osd.0.smithi071.stdout:Mar 04 09:09:52 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-0[47571]: 2024-03-04T09:09:52.191+0000 7f1ecfefe700 -1 osd.0 108 *** Got signal Terminated *** 2024-03-04T09:09:52.672 INFO:journalctl@ceph.osd.0.smithi071.stdout:Mar 04 09:09:52 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-0[47571]: 2024-03-04T09:09:52.191+0000 7f1ecfefe700 -1 osd.0 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T09:09:57.922 INFO:journalctl@ceph.osd.0.smithi071.stdout:Mar 04 09:09:57 smithi071 podman[92501]: 2024-03-04 09:09:57.621702743 +0000 UTC m=+5.554469866 container died c7c221fecf17261d2b91870b54a734d6f5ad47e94b4db65ff26cec9f84c0682c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-03-04T09:09:58.273 INFO:journalctl@ceph.osd.0.smithi071.stdout:Mar 04 09:09:58 smithi071 podman[92501]: 2024-03-04 09:09:58.025803827 +0000 UTC m=+5.958570960 container cleanup c7c221fecf17261d2b91870b54a734d6f5ad47e94b4db65ff26cec9f84c0682c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-03-04T09:09:58.273 INFO:journalctl@ceph.osd.0.smithi071.stdout:Mar 04 09:09:58 smithi071 bash[92501]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-0 2024-03-04T09:09:58.671 INFO:journalctl@ceph.osd.0.smithi071.stdout:Mar 04 09:09:58 smithi071 podman[92519]: 2024-03-04 09:09:58.277549797 +0000 UTC m=+0.657692693 container remove c7c221fecf17261d2b91870b54a734d6f5ad47e94b4db65ff26cec9f84c0682c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-0, RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:09:58.969 INFO:journalctl@ceph.osd.0.smithi071.stdout:Mar 04 09:09:58 smithi071 podman[92609]: 2024-03-04 09:09:58.599228078 +0000 UTC m=+0.018205096 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:09:58.969 INFO:journalctl@ceph.osd.0.smithi071.stdout:Mar 04 09:09:58 smithi071 podman[92609]: 2024-03-04 09:09:58.720164405 +0000 UTC m=+0.139141424 container create 04226c7cceb66da071a5918f8b3b6e1c1f7d6f52cadff4df063c3dac546dfea0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:09:58.969 INFO:journalctl@ceph.osd.0.smithi071.stdout:Mar 04 09:09:58 smithi071 podman[92609]: 2024-03-04 09:09:58.973471578 +0000 UTC m=+0.392448606 container init 04226c7cceb66da071a5918f8b3b6e1c1f7d6f52cadff4df063c3dac546dfea0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:09:59.260 INFO:journalctl@ceph.osd.0.smithi071.stdout:Mar 04 09:09:58 smithi071 podman[92609]: 2024-03-04 09:09:58.976839876 +0000 UTC m=+0.395816903 container start 04226c7cceb66da071a5918f8b3b6e1c1f7d6f52cadff4df063c3dac546dfea0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-0-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-1bc250c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-04T09:09:59.260 INFO:journalctl@ceph.osd.0.smithi071.stdout:Mar 04 09:09:59 smithi071 podman[92609]: 2024-03-04 09:09:59.098700634 +0000 UTC m=+0.517677667 container attach 04226c7cceb66da071a5918f8b3b6e1c1f7d6f52cadff4df063c3dac546dfea0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-0-deactivate, RELEASE=wip-lusov-qdb-1bc250c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-04T09:09:59.564 INFO:journalctl@ceph.osd.0.smithi071.stdout:Mar 04 09:09:59 smithi071 podman[92609]: 2024-03-04 09:09:59.264669714 +0000 UTC m=+0.683646737 container died 04226c7cceb66da071a5918f8b3b6e1c1f7d6f52cadff4df063c3dac546dfea0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T09:09:59.872 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.0.service' 2024-03-04T09:09:59.914 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:09:59.915 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-04T09:09:59.915 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-04T09:09:59.916 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.1 2024-03-04T09:10:00.239 INFO:journalctl@ceph.osd.1.smithi150.stdout:Mar 04 09:09:59 smithi150 systemd[1]: Stopping Ceph osd.1 for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:10:00.531 INFO:journalctl@ceph.osd.1.smithi150.stdout:Mar 04 09:10:00 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-1[41405]: 2024-03-04T09:10:00.242+0000 7f070cac0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:10:00.532 INFO:journalctl@ceph.osd.1.smithi150.stdout:Mar 04 09:10:00 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-1[41405]: 2024-03-04T09:10:00.242+0000 7f070cac0700 -1 osd.1 108 *** Got signal Terminated *** 2024-03-04T09:10:00.532 INFO:journalctl@ceph.osd.1.smithi150.stdout:Mar 04 09:10:00 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-1[41405]: 2024-03-04T09:10:00.242+0000 7f070cac0700 -1 osd.1 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T09:10:06.030 INFO:journalctl@ceph.osd.1.smithi150.stdout:Mar 04 09:10:05 smithi150 podman[72366]: 2024-03-04 09:10:05.736555865 +0000 UTC m=+5.618965286 container died 1b3cc123c3f0ec1aa727cea2d6a292cc92a205f6b0b42727f395fe24d5c1abc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-1, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-04T09:10:06.382 INFO:journalctl@ceph.osd.1.smithi150.stdout:Mar 04 09:10:06 smithi150 podman[72366]: 2024-03-04 09:10:06.160066157 +0000 UTC m=+6.042475571 container cleanup 1b3cc123c3f0ec1aa727cea2d6a292cc92a205f6b0b42727f395fe24d5c1abc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-1, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True) 2024-03-04T09:10:06.382 INFO:journalctl@ceph.osd.1.smithi150.stdout:Mar 04 09:10:06 smithi150 bash[72366]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-1 2024-03-04T09:10:06.780 INFO:journalctl@ceph.osd.1.smithi150.stdout:Mar 04 09:10:06 smithi150 podman[72382]: 2024-03-04 09:10:06.386783177 +0000 UTC m=+0.647430515 container remove 1b3cc123c3f0ec1aa727cea2d6a292cc92a205f6b0b42727f395fe24d5c1abc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-1, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-04T09:10:07.102 INFO:journalctl@ceph.osd.1.smithi150.stdout:Mar 04 09:10:06 smithi150 podman[72474]: 2024-03-04 09:10:06.725977496 +0000 UTC m=+0.017704480 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:10:07.103 INFO:journalctl@ceph.osd.1.smithi150.stdout:Mar 04 09:10:06 smithi150 podman[72474]: 2024-03-04 09:10:06.854593409 +0000 UTC m=+0.146320382 container create 0b487f03d7bd9eeace6edc4f7f6f7a6541e3b239f8bfc29569efb67fb4815014 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:10:07.103 INFO:journalctl@ceph.osd.1.smithi150.stdout:Mar 04 09:10:07 smithi150 podman[72474]: 2024-03-04 09:10:07.107202946 +0000 UTC m=+0.398929938 container init 0b487f03d7bd9eeace6edc4f7f6f7a6541e3b239f8bfc29569efb67fb4815014 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:10:07.375 INFO:journalctl@ceph.osd.1.smithi150.stdout:Mar 04 09:10:07 smithi150 podman[72474]: 2024-03-04 09:10:07.110594321 +0000 UTC m=+0.402321297 container start 0b487f03d7bd9eeace6edc4f7f6f7a6541e3b239f8bfc29569efb67fb4815014 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T09:10:07.375 INFO:journalctl@ceph.osd.1.smithi150.stdout:Mar 04 09:10:07 smithi150 podman[72474]: 2024-03-04 09:10:07.207687124 +0000 UTC m=+0.499414103 container attach 0b487f03d7bd9eeace6edc4f7f6f7a6541e3b239f8bfc29569efb67fb4815014 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222) 2024-03-04T09:10:07.698 INFO:journalctl@ceph.osd.1.smithi150.stdout:Mar 04 09:10:07 smithi150 podman[72474]: 2024-03-04 09:10:07.41418102 +0000 UTC m=+0.705908003 container died 0b487f03d7bd9eeace6edc4f7f6f7a6541e3b239f8bfc29569efb67fb4815014 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-04T09:10:08.036 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.1.service' 2024-03-04T09:10:08.077 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:10:08.078 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-04T09:10:08.078 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-04T09:10:08.078 DEBUG:teuthology.orchestra.run.smithi158:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.2 2024-03-04T09:10:08.385 INFO:journalctl@ceph.osd.2.smithi158.stdout:Mar 04 09:10:08 smithi158 systemd[1]: Stopping Ceph osd.2 for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:10:08.802 INFO:journalctl@ceph.osd.2.smithi158.stdout:Mar 04 09:10:08 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-2[42413]: 2024-03-04T09:10:08.388+0000 7f2aaae8e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:10:08.803 INFO:journalctl@ceph.osd.2.smithi158.stdout:Mar 04 09:10:08 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-2[42413]: 2024-03-04T09:10:08.388+0000 7f2aaae8e700 -1 osd.2 108 *** Got signal Terminated *** 2024-03-04T09:10:08.803 INFO:journalctl@ceph.osd.2.smithi158.stdout:Mar 04 09:10:08 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-2[42413]: 2024-03-04T09:10:08.388+0000 7f2aaae8e700 -1 osd.2 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T09:10:14.053 INFO:journalctl@ceph.osd.2.smithi158.stdout:Mar 04 09:10:13 smithi158 podman[74688]: 2024-03-04 09:10:13.775584587 +0000 UTC m=+5.503762596 container died 3f5227346c44fb5aca34c20352f1fe1ac292a03d8fcf956d226354847f376b4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-04T09:10:14.413 INFO:journalctl@ceph.osd.2.smithi158.stdout:Mar 04 09:10:14 smithi158 podman[74688]: 2024-03-04 09:10:14.16561791 +0000 UTC m=+5.893795920 container cleanup 3f5227346c44fb5aca34c20352f1fe1ac292a03d8fcf956d226354847f376b4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, ceph=True, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-04T09:10:14.413 INFO:journalctl@ceph.osd.2.smithi158.stdout:Mar 04 09:10:14 smithi158 bash[74688]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-2 2024-03-04T09:10:14.726 INFO:journalctl@ceph.osd.2.smithi158.stdout:Mar 04 09:10:14 smithi158 podman[74705]: 2024-03-04 09:10:14.417207288 +0000 UTC m=+0.642648428 container remove 3f5227346c44fb5aca34c20352f1fe1ac292a03d8fcf956d226354847f376b4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-2, RELEASE=wip-lusov-qdb-1bc250c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-04T09:10:15.041 INFO:journalctl@ceph.osd.2.smithi158.stdout:Mar 04 09:10:14 smithi158 podman[74797]: 2024-03-04 09:10:14.633382227 +0000 UTC m=+0.017727557 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:10:15.042 INFO:journalctl@ceph.osd.2.smithi158.stdout:Mar 04 09:10:14 smithi158 podman[74797]: 2024-03-04 09:10:14.734857643 +0000 UTC m=+0.119202968 container create f54557cfb3c0939878eec37737dac5447fd8f7de406515349b9c16bdc9f7d5e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-2-deactivate, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-04T09:10:15.042 INFO:journalctl@ceph.osd.2.smithi158.stdout:Mar 04 09:10:14 smithi158 podman[74797]: 2024-03-04 09:10:14.971072878 +0000 UTC m=+0.355418208 container init f54557cfb3c0939878eec37737dac5447fd8f7de406515349b9c16bdc9f7d5e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-04T09:10:15.042 INFO:journalctl@ceph.osd.2.smithi158.stdout:Mar 04 09:10:14 smithi158 podman[74797]: 2024-03-04 09:10:14.974350807 +0000 UTC m=+0.358696137 container start f54557cfb3c0939878eec37737dac5447fd8f7de406515349b9c16bdc9f7d5e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, ceph=True) 2024-03-04T09:10:15.042 INFO:journalctl@ceph.osd.2.smithi158.stdout:Mar 04 09:10:15 smithi158 podman[74797]: 2024-03-04 09:10:15.046087245 +0000 UTC m=+0.430432593 container attach f54557cfb3c0939878eec37737dac5447fd8f7de406515349b9c16bdc9f7d5e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, org.label-schema.schema-version=1.0) 2024-03-04T09:10:15.302 INFO:journalctl@ceph.osd.2.smithi158.stdout:Mar 04 09:10:15 smithi158 podman[74797]: 2024-03-04 09:10:15.271577138 +0000 UTC m=+0.655922477 container died f54557cfb3c0939878eec37737dac5447fd8f7de406515349b9c16bdc9f7d5e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:10:15.889 DEBUG:teuthology.orchestra.run.smithi158:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.2.service' 2024-03-04T09:10:15.931 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:10:15.932 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-04T09:10:15.932 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-04T09:10:15.932 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.3 2024-03-04T09:10:16.242 INFO:journalctl@ceph.osd.3.smithi071.stdout:Mar 04 09:10:15 smithi071 systemd[1]: Stopping Ceph osd.3 for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:10:16.671 INFO:journalctl@ceph.osd.3.smithi071.stdout:Mar 04 09:10:16 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-3[54095]: 2024-03-04T09:10:16.246+0000 7fac9789a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:10:16.672 INFO:journalctl@ceph.osd.3.smithi071.stdout:Mar 04 09:10:16 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-3[54095]: 2024-03-04T09:10:16.246+0000 7fac9789a700 -1 osd.3 108 *** Got signal Terminated *** 2024-03-04T09:10:16.672 INFO:journalctl@ceph.osd.3.smithi071.stdout:Mar 04 09:10:16 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-3[54095]: 2024-03-04T09:10:16.246+0000 7fac9789a700 -1 osd.3 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T09:10:20.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:20 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:20.593+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:21.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:20 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:20.686+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:21.884 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:21 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:21.611+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:21.885 INFO:journalctl@ceph.osd.3.smithi071.stdout:Mar 04 09:10:21 smithi071 podman[92721]: 2024-03-04 09:10:21.610484337 +0000 UTC m=+5.480527797 container died 8b0a4ae2c24fef7957c09817d078e5e78de642ff3f9076154173eea14960005c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD) 2024-03-04T09:10:22.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:21 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:21.704+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:22.172 INFO:journalctl@ceph.osd.3.smithi071.stdout:Mar 04 09:10:21 smithi071 podman[92721]: 2024-03-04 09:10:21.988116499 +0000 UTC m=+5.858159958 container cleanup 8b0a4ae2c24fef7957c09817d078e5e78de642ff3f9076154173eea14960005c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-3, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-04T09:10:22.172 INFO:journalctl@ceph.osd.3.smithi071.stdout:Mar 04 09:10:21 smithi071 bash[92721]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-3 2024-03-04T09:10:22.571 INFO:journalctl@ceph.osd.3.smithi071.stdout:Mar 04 09:10:22 smithi071 podman[92750]: 2024-03-04 09:10:22.256558829 +0000 UTC m=+0.641972365 container remove 8b0a4ae2c24fef7957c09817d078e5e78de642ff3f9076154173eea14960005c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-3, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T09:10:22.803 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:22 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:22.383+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:52.168431+0000 front 2024-03-04T09:09:52.168503+0000 (oldest deadline 2024-03-04T09:10:21.568939+0000) 2024-03-04T09:10:22.804 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:22 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:22.399+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:22.839 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:22 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:22.575+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:22.840 INFO:journalctl@ceph.osd.3.smithi071.stdout:Mar 04 09:10:22 smithi071 podman[92841]: 2024-03-04 09:10:22.499045925 +0000 UTC m=+0.018920741 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:10:22.840 INFO:journalctl@ceph.osd.3.smithi071.stdout:Mar 04 09:10:22 smithi071 podman[92841]: 2024-03-04 09:10:22.641007262 +0000 UTC m=+0.160882108 container create 39886d9486d8d9c190a3ae7e3a0de868c1d05964d1feb9ff64ff943cf795501e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:10:22.840 INFO:journalctl@ceph.osd.3.smithi071.stdout:Mar 04 09:10:22 smithi071 podman[92841]: 2024-03-04 09:10:22.84401438 +0000 UTC m=+0.363889194 container init 39886d9486d8d9c190a3ae7e3a0de868c1d05964d1feb9ff64ff943cf795501e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:10:23.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:22 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:22.722+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:23.128 INFO:journalctl@ceph.osd.3.smithi071.stdout:Mar 04 09:10:22 smithi071 podman[92841]: 2024-03-04 09:10:22.847128122 +0000 UTC m=+0.367002925 container start 39886d9486d8d9c190a3ae7e3a0de868c1d05964d1feb9ff64ff943cf795501e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-3-deactivate, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-04T09:10:23.128 INFO:journalctl@ceph.osd.3.smithi071.stdout:Mar 04 09:10:22 smithi071 podman[92841]: 2024-03-04 09:10:22.935690532 +0000 UTC m=+0.455565359 container attach 39886d9486d8d9c190a3ae7e3a0de868c1d05964d1feb9ff64ff943cf795501e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-1bc250c, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux ) 2024-03-04T09:10:23.422 INFO:journalctl@ceph.osd.3.smithi071.stdout:Mar 04 09:10:23 smithi071 podman[92841]: 2024-03-04 09:10:23.13259267 +0000 UTC m=+0.652467474 container died 39886d9486d8d9c190a3ae7e3a0de868c1d05964d1feb9ff64ff943cf795501e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-04T09:10:23.803 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:23 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:23.379+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:52.168431+0000 front 2024-03-04T09:09:52.168503+0000 (oldest deadline 2024-03-04T09:10:21.568939+0000) 2024-03-04T09:10:23.803 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:23 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:23.578+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:23.803 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:23 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:23.424+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:23.819 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.3.service' 2024-03-04T09:10:23.862 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:10:23.863 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-04T09:10:23.863 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-04T09:10:23.863 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.4 2024-03-04T09:10:24.030 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 04 09:10:23 smithi150 systemd[1]: Stopping Ceph osd.4 for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:10:24.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:23 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:23.772+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:24.530 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 04 09:10:24 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-4[48124]: 2024-03-04T09:10:24.172+0000 7f5097f4f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:10:24.530 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 04 09:10:24 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-4[48124]: 2024-03-04T09:10:24.172+0000 7f5097f4f700 -1 osd.4 108 *** Got signal Terminated *** 2024-03-04T09:10:24.530 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 04 09:10:24 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-4[48124]: 2024-03-04T09:10:24.172+0000 7f5097f4f700 -1 osd.4 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T09:10:24.802 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:24 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:24.386+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:24.803 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:24 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:24.366+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:52.168431+0000 front 2024-03-04T09:09:52.168503+0000 (oldest deadline 2024-03-04T09:10:21.568939+0000) 2024-03-04T09:10:24.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:24 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:24.592+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:25.280 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:24 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:24.808+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:25.802 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:25 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:25.400+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:25.803 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:25 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:25.368+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:52.168431+0000 front 2024-03-04T09:09:52.168503+0000 (oldest deadline 2024-03-04T09:10:21.568939+0000) 2024-03-04T09:10:25.921 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:25 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:25.592+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:26.280 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:25 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:25.791+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:26.280 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:26 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:26.016+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:26.779 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 04 09:10:26 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-4[48124]: 2024-03-04T09:10:26.343+0000 7f5094543700 -1 osd.4 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:00.527854+0000 front 2024-03-04T09:10:00.527876+0000 (oldest deadline 2024-03-04T09:10:25.828063+0000) 2024-03-04T09:10:26.802 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:26 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:26.429+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:26.803 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:26 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:26.336+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:52.168431+0000 front 2024-03-04T09:09:52.168503+0000 (oldest deadline 2024-03-04T09:10:21.568939+0000) 2024-03-04T09:10:26.921 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:26 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:26.571+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:26.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:26 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:26.571+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:27.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:26 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:26.780+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:27.346 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:27 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:27.054+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:27.668 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:27 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:27.461+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:27.669 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:27 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:27.374+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:52.168431+0000 front 2024-03-04T09:09:52.168503+0000 (oldest deadline 2024-03-04T09:10:21.568939+0000) 2024-03-04T09:10:27.669 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:27 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:27.374+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.970384+0000 front 2024-03-04T09:10:04.970510+0000 (oldest deadline 2024-03-04T09:10:27.270452+0000) 2024-03-04T09:10:27.780 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 04 09:10:27 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-4[48124]: 2024-03-04T09:10:27.350+0000 7f5094543700 -1 osd.4 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:00.527854+0000 front 2024-03-04T09:10:00.527876+0000 (oldest deadline 2024-03-04T09:10:25.828063+0000) 2024-03-04T09:10:27.921 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:27 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:27.558+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:27.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:27 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:27.558+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:28.099 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:27 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:27.807+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:28.375 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:28 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:28.103+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:28.428 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:28 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:28.341+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:52.168431+0000 front 2024-03-04T09:09:52.168503+0000 (oldest deadline 2024-03-04T09:10:21.568939+0000) 2024-03-04T09:10:28.428 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:28 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:28.341+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.970384+0000 front 2024-03-04T09:10:04.970510+0000 (oldest deadline 2024-03-04T09:10:27.270452+0000) 2024-03-04T09:10:28.428 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:28 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:28.176+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:28.777 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 04 09:10:28 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-4[48124]: 2024-03-04T09:10:28.379+0000 7f5094543700 -1 osd.4 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:00.527854+0000 front 2024-03-04T09:10:00.527876+0000 (oldest deadline 2024-03-04T09:10:25.828063+0000) 2024-03-04T09:10:28.802 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:28 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:28.431+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:28.921 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:28 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:28.599+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:28.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:28 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:28.599+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:29.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:28 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:28.781+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:29.438 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:29 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:29.175+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:29.438 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:29 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:29.346+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:52.168431+0000 front 2024-03-04T09:09:52.168503+0000 (oldest deadline 2024-03-04T09:10:21.568939+0000) 2024-03-04T09:10:29.439 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:29 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:29.346+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.970384+0000 front 2024-03-04T09:10:04.970510+0000 (oldest deadline 2024-03-04T09:10:27.270452+0000) 2024-03-04T09:10:29.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:29 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:29.142+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:29.798 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 04 09:10:29 smithi150 podman[72592]: 2024-03-04 09:10:29.534262354 +0000 UTC m=+5.477610675 container died a0aeed32e054c7faa6b1527c65997cfe1938c6d70c0105eb08c1c19e6733727b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-4, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:10:29.802 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:29 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:29.441+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:29.921 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:29 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:29.558+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.893223+0000 front 2024-03-04T09:10:03.892986+0000 (oldest deadline 2024-03-04T09:10:28.593135+0000) 2024-03-04T09:10:29.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:29 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:29.601+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:29.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:29 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:29.601+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:30.171 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:29 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:29.801+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:30.171 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:29 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:29.801+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:30.171 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 04 09:10:29 smithi150 podman[72592]: 2024-03-04 09:10:29.948278849 +0000 UTC m=+5.891627158 container cleanup a0aeed32e054c7faa6b1527c65997cfe1938c6d70c0105eb08c1c19e6733727b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-1bc250c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-03-04T09:10:30.172 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 04 09:10:29 smithi150 bash[72592]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-4 2024-03-04T09:10:30.458 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:30 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:30.310+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:52.168431+0000 front 2024-03-04T09:09:52.168503+0000 (oldest deadline 2024-03-04T09:10:21.568939+0000) 2024-03-04T09:10:30.458 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:30 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:30.310+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.970384+0000 front 2024-03-04T09:10:04.970510+0000 (oldest deadline 2024-03-04T09:10:27.270452+0000) 2024-03-04T09:10:30.459 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:30 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:30.173+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:30.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:30 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:30.174+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:30.530 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 04 09:10:30 smithi150 podman[72609]: 2024-03-04 09:10:30.20361503 +0000 UTC m=+0.671474732 container remove a0aeed32e054c7faa6b1527c65997cfe1938c6d70c0105eb08c1c19e6733727b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-1bc250c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0) 2024-03-04T09:10:30.802 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:30 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:30.462+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:30.876 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:30 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:30.832+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:30.876 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:30 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:30.832+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:30.876 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 04 09:10:30 smithi150 podman[72704]: 2024-03-04 09:10:30.600620491 +0000 UTC m=+0.107209080 container create 6ddff1b5c76dc4a61c3d9b5c1115426eb18121160a8eaf2be26660f3bf7b7ecf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-4-deactivate, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-04T09:10:30.876 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 04 09:10:30 smithi150 podman[72704]: 2024-03-04 09:10:30.510971993 +0000 UTC m=+0.017560598 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:10:30.877 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 04 09:10:30 smithi150 podman[72704]: 2024-03-04 09:10:30.878437104 +0000 UTC m=+0.385025703 container init 6ddff1b5c76dc4a61c3d9b5c1115426eb18121160a8eaf2be26660f3bf7b7ecf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, ceph=True, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:10:30.921 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:30 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:30.605+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.893223+0000 front 2024-03-04T09:10:03.892986+0000 (oldest deadline 2024-03-04T09:10:28.593135+0000) 2024-03-04T09:10:30.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:30 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:30.626+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:30.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:30 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:30.626+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:31.167 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 04 09:10:30 smithi150 podman[72704]: 2024-03-04 09:10:30.881618911 +0000 UTC m=+0.388207510 container start 6ddff1b5c76dc4a61c3d9b5c1115426eb18121160a8eaf2be26660f3bf7b7ecf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:10:31.168 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 04 09:10:30 smithi150 podman[72704]: 2024-03-04 09:10:30.978434249 +0000 UTC m=+0.485022857 container attach 6ddff1b5c76dc4a61c3d9b5c1115426eb18121160a8eaf2be26660f3bf7b7ecf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-04T09:10:31.485 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:31 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:31.162+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:31.486 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:31 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:31.356+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:52.168431+0000 front 2024-03-04T09:09:52.168503+0000 (oldest deadline 2024-03-04T09:10:21.568939+0000) 2024-03-04T09:10:31.486 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:31 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:31.356+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.970384+0000 front 2024-03-04T09:10:04.970510+0000 (oldest deadline 2024-03-04T09:10:27.270452+0000) 2024-03-04T09:10:31.527 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:31 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:31.176+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:31.528 INFO:journalctl@ceph.osd.4.smithi150.stdout:Mar 04 09:10:31 smithi150 podman[72704]: 2024-03-04 09:10:31.172032455 +0000 UTC m=+0.678621057 container died 6ddff1b5c76dc4a61c3d9b5c1115426eb18121160a8eaf2be26660f3bf7b7ecf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:10:31.802 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:31 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:31.489+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:31.881 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.4.service' 2024-03-04T09:10:31.922 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:10:31.922 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-04T09:10:31.922 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-04T09:10:31.923 DEBUG:teuthology.orchestra.run.smithi158:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.5 2024-03-04T09:10:31.923 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:31 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:31.599+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.893223+0000 front 2024-03-04T09:10:03.892986+0000 (oldest deadline 2024-03-04T09:10:28.593135+0000) 2024-03-04T09:10:31.923 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:31 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:31.651+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:31.924 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:31 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:31.651+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:32.164 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:31 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:31.813+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:32.165 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:31 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:31.813+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:32.303 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:31 smithi158 systemd[1]: Stopping Ceph osd.5 for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:10:32.303 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:32 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:32.189+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:32.529 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:32 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:32.168+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:32.802 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:32 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:32.444+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:32.803 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:32 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:32.342+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:52.168431+0000 front 2024-03-04T09:09:52.168503+0000 (oldest deadline 2024-03-04T09:10:21.568939+0000) 2024-03-04T09:10:32.803 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:32 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:32.342+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.970384+0000 front 2024-03-04T09:10:04.970510+0000 (oldest deadline 2024-03-04T09:10:27.270452+0000) 2024-03-04T09:10:32.803 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:32 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:32.518+0000 7f5aedb0d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:10:32.803 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:32 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:32.518+0000 7f5aedb0d700 -1 osd.5 108 *** Got signal Terminated *** 2024-03-04T09:10:32.803 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:32 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:32.518+0000 7f5aedb0d700 -1 osd.5 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T09:10:32.923 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:32 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:32.556+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.893223+0000 front 2024-03-04T09:10:03.892986+0000 (oldest deadline 2024-03-04T09:10:28.593135+0000) 2024-03-04T09:10:32.924 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:32 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:32.626+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:32.924 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:32 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:32.626+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:33.188 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:32 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:32.827+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:33.188 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:32 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:32.827+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:33.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:33 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:33.192+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:33.552 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:33 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:33.429+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:33.553 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:33 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:33.338+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:52.168431+0000 front 2024-03-04T09:09:52.168503+0000 (oldest deadline 2024-03-04T09:10:21.568939+0000) 2024-03-04T09:10:33.553 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:33 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:33.338+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.970384+0000 front 2024-03-04T09:10:04.970510+0000 (oldest deadline 2024-03-04T09:10:27.270452+0000) 2024-03-04T09:10:33.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:33 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:33.215+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:33.922 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:33 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:33.580+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.893223+0000 front 2024-03-04T09:10:03.892986+0000 (oldest deadline 2024-03-04T09:10:28.593135+0000) 2024-03-04T09:10:33.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:33 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:33.597+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:33.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:33 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:33.597+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:34.160 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:33 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:33.788+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:34.160 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:33 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:33.788+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:34.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:34 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:34.164+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:34.552 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:34 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:34.223+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:34.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:34 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:34.395+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:34.553 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:34 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:34.356+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:52.168431+0000 front 2024-03-04T09:09:52.168503+0000 (oldest deadline 2024-03-04T09:10:21.568939+0000) 2024-03-04T09:10:34.553 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:34 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:34.356+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.970384+0000 front 2024-03-04T09:10:04.970510+0000 (oldest deadline 2024-03-04T09:10:27.270452+0000) 2024-03-04T09:10:34.554 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:34 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:34.356+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:08.371452+0000 front 2024-03-04T09:10:08.371416+0000 (oldest deadline 2024-03-04T09:10:34.271520+0000) 2024-03-04T09:10:34.922 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:34 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:34.586+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.893223+0000 front 2024-03-04T09:10:03.892986+0000 (oldest deadline 2024-03-04T09:10:28.593135+0000) 2024-03-04T09:10:34.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:34 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:34.583+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:34.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:34 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:34.583+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:34.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:34 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:34.583+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:35.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:34 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:34.743+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:35.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:34 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:34.743+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:35.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:35 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:35.197+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:35.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:35 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:35.409+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:35.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:35 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:35.242+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:35.554 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:35 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:35.242+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:35.554 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:35 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:35.338+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:52.168431+0000 front 2024-03-04T09:09:52.168503+0000 (oldest deadline 2024-03-04T09:10:21.568939+0000) 2024-03-04T09:10:35.554 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:35 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:35.338+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.970384+0000 front 2024-03-04T09:10:04.970510+0000 (oldest deadline 2024-03-04T09:10:27.270452+0000) 2024-03-04T09:10:35.555 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:35 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:35.338+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:08.371452+0000 front 2024-03-04T09:10:08.371416+0000 (oldest deadline 2024-03-04T09:10:34.271520+0000) 2024-03-04T09:10:35.922 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:35 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:35.595+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.893223+0000 front 2024-03-04T09:10:03.892986+0000 (oldest deadline 2024-03-04T09:10:28.593135+0000) 2024-03-04T09:10:35.922 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:35 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:35.595+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.294524+0000 front 2024-03-04T09:10:13.294377+0000 (oldest deadline 2024-03-04T09:10:34.994303+0000) 2024-03-04T09:10:35.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:35 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:35.558+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:35.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:35 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:35.558+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:35.923 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:35 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:35.558+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:36.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:35 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:35.701+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:36.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:35 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:35.701+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:36.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:35 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:35.701+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:10:36.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:36 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:36.201+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:36.552 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:36 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:36.455+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:36.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:36 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:36.226+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:36.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:36 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:36.226+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:36.553 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:36 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:36.300+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:52.168431+0000 front 2024-03-04T09:09:52.168503+0000 (oldest deadline 2024-03-04T09:10:21.568939+0000) 2024-03-04T09:10:36.553 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:36 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:36.300+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.970384+0000 front 2024-03-04T09:10:04.970510+0000 (oldest deadline 2024-03-04T09:10:27.270452+0000) 2024-03-04T09:10:36.553 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:36 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:36.300+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:08.371452+0000 front 2024-03-04T09:10:08.371416+0000 (oldest deadline 2024-03-04T09:10:34.271520+0000) 2024-03-04T09:10:36.922 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:36 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:36.574+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.893223+0000 front 2024-03-04T09:10:03.892986+0000 (oldest deadline 2024-03-04T09:10:28.593135+0000) 2024-03-04T09:10:36.922 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:36 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:36.574+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.294524+0000 front 2024-03-04T09:10:13.294377+0000 (oldest deadline 2024-03-04T09:10:34.994303+0000) 2024-03-04T09:10:36.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:36 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:36.547+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:36.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:36 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:36.547+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:36.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:36 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:36.547+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:37.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:36 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:36.660+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:37.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:36 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:36.660+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:37.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:36 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:36.660+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:10:37.307 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:37 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:37.175+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:37.489 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:37 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:37.241+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:37.489 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:37 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:37.241+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:37.489 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:37 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:37.303+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:52.168431+0000 front 2024-03-04T09:09:52.168503+0000 (oldest deadline 2024-03-04T09:10:21.568939+0000) 2024-03-04T09:10:37.489 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:37 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:37.303+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.970384+0000 front 2024-03-04T09:10:04.970510+0000 (oldest deadline 2024-03-04T09:10:27.270452+0000) 2024-03-04T09:10:37.489 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:37 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5[49087]: 2024-03-04T09:10:37.303+0000 7f5aea101700 -1 osd.5 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:08.371452+0000 front 2024-03-04T09:10:08.371416+0000 (oldest deadline 2024-03-04T09:10:34.271520+0000) 2024-03-04T09:10:37.802 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:37 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:37.492+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:37.922 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:37 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:37.566+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.893223+0000 front 2024-03-04T09:10:03.892986+0000 (oldest deadline 2024-03-04T09:10:28.593135+0000) 2024-03-04T09:10:37.922 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:37 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:37.566+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.294524+0000 front 2024-03-04T09:10:13.294377+0000 (oldest deadline 2024-03-04T09:10:34.994303+0000) 2024-03-04T09:10:37.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:37 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:37.572+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:37.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:37 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:37.572+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:37.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:37 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:37.572+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:38.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:37 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:37.708+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:38.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:37 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:37.708+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:38.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:37 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:37.708+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:10:38.291 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:38 smithi158 podman[74914]: 2024-03-04 09:10:38.024246048 +0000 UTC m=+5.900284162 container died 85806226e24d5d89ed465815f1abb6037ee388b9f1bbffc8adefad44c467c511 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-04T09:10:38.292 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:38 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:38.192+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:38.292 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:38 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:38.192+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:38.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:38 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:38.172+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:38.552 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:38 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:38.528+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:38.553 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:38 smithi158 podman[74914]: 2024-03-04 09:10:38.394614879 +0000 UTC m=+6.270653044 container cleanup 85806226e24d5d89ed465815f1abb6037ee388b9f1bbffc8adefad44c467c511 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS) 2024-03-04T09:10:38.553 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:38 smithi158 bash[74914]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5 2024-03-04T09:10:38.922 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:38 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:38.526+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.893223+0000 front 2024-03-04T09:10:03.892986+0000 (oldest deadline 2024-03-04T09:10:28.593135+0000) 2024-03-04T09:10:38.922 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:38 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:38.526+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.294524+0000 front 2024-03-04T09:10:13.294377+0000 (oldest deadline 2024-03-04T09:10:34.994303+0000) 2024-03-04T09:10:38.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:38 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:38.588+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:38.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:38 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:38.588+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:38.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:38 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:38.588+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:39.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:38 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:38.711+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:39.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:38 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:38.711+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:39.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:38 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:38.711+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:10:39.048 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:38 smithi158 podman[74931]: 2024-03-04 09:10:38.696042246 +0000 UTC m=+0.673594472 container remove 85806226e24d5d89ed465815f1abb6037ee388b9f1bbffc8adefad44c467c511 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:10:39.302 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:39 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:39.178+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:39.303 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:39 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:39.178+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:39.303 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:39 smithi158 podman[75023]: 2024-03-04 09:10:38.956392495 +0000 UTC m=+0.017766597 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:10:39.303 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:39 smithi158 podman[75023]: 2024-03-04 09:10:39.072046518 +0000 UTC m=+0.133420626 container create 5d729b1b6f64be806492da2146c2da926f87b98ee5bf265924e167d6c3eef7f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:10:39.303 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:39 smithi158 podman[75023]: 2024-03-04 09:10:39.299695498 +0000 UTC m=+0.361069600 container init 5d729b1b6f64be806492da2146c2da926f87b98ee5bf265924e167d6c3eef7f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-04T09:10:39.303 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:39 smithi158 podman[75023]: 2024-03-04 09:10:39.302667741 +0000 UTC m=+0.364041830 container start 5d729b1b6f64be806492da2146c2da926f87b98ee5bf265924e167d6c3eef7f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T09:10:39.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:39 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:39.150+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:39.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:39 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:39.150+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.172054+0000 front 2024-03-04T09:10:13.172023+0000 (oldest deadline 2024-03-04T09:10:39.072075+0000) 2024-03-04T09:10:39.803 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:39 smithi158 podman[75023]: 2024-03-04 09:10:39.399762986 +0000 UTC m=+0.461137092 container attach 5d729b1b6f64be806492da2146c2da926f87b98ee5bf265924e167d6c3eef7f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-04T09:10:39.803 INFO:journalctl@ceph.osd.5.smithi158.stdout:Mar 04 09:10:39 smithi158 podman[75023]: 2024-03-04 09:10:39.591125718 +0000 UTC m=+0.652499813 container died 5d729b1b6f64be806492da2146c2da926f87b98ee5bf265924e167d6c3eef7f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-04T09:10:39.803 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:39 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:39.483+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:39.803 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:39 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:39.483+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:10:39.922 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:39 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:39.479+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.893223+0000 front 2024-03-04T09:10:03.892986+0000 (oldest deadline 2024-03-04T09:10:28.593135+0000) 2024-03-04T09:10:39.922 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:39 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:39.479+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.294524+0000 front 2024-03-04T09:10:13.294377+0000 (oldest deadline 2024-03-04T09:10:34.994303+0000) 2024-03-04T09:10:39.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:39 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:39.565+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:39.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:39 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:39.565+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:39.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:39 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:39.565+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:40.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:39 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:39.683+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:40.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:39 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:39.683+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:40.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:39 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:39.683+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:10:40.255 DEBUG:teuthology.orchestra.run.smithi158:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.5.service' 2024-03-04T09:10:40.297 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:10:40.298 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-04T09:10:40.298 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-04T09:10:40.298 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.6 2024-03-04T09:10:40.463 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:40 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:40.188+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:40.463 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:40 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:40.188+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:40.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:40 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:40.182+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:40.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:40 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:40.182+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.172054+0000 front 2024-03-04T09:10:13.172023+0000 (oldest deadline 2024-03-04T09:10:39.072075+0000) 2024-03-04T09:10:40.612 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:40 smithi071 systemd[1]: Stopping Ceph osd.6 for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:10:40.613 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:40 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:40.468+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.893223+0000 front 2024-03-04T09:10:03.892986+0000 (oldest deadline 2024-03-04T09:10:28.593135+0000) 2024-03-04T09:10:40.613 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:40 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:40.468+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.294524+0000 front 2024-03-04T09:10:13.294377+0000 (oldest deadline 2024-03-04T09:10:34.994303+0000) 2024-03-04T09:10:40.613 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:40 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:40.555+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:40.613 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:40 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:40.555+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:40.613 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:40 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:40.555+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:40.802 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:40 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:40.467+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:40.803 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:40 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:40.467+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:10:40.921 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:40 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:40.616+0000 7f1065358700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:10:40.922 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:40 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:40.616+0000 7f1065358700 -1 osd.6 108 *** Got signal Terminated *** 2024-03-04T09:10:40.922 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:40 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:40.616+0000 7f1065358700 -1 osd.6 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T09:10:41.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:40 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:40.669+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:41.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:40 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:40.669+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:41.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:40 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:40.669+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:10:41.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:41 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:41.223+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:41.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:41 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:41.223+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.172054+0000 front 2024-03-04T09:10:13.172023+0000 (oldest deadline 2024-03-04T09:10:39.072075+0000) 2024-03-04T09:10:41.552 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:41 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:41.188+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:41.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:41 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:41.188+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:41.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:41 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:41.431+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:41.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:41 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:41.431+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:10:41.922 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:41 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:41.472+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.893223+0000 front 2024-03-04T09:10:03.892986+0000 (oldest deadline 2024-03-04T09:10:28.593135+0000) 2024-03-04T09:10:41.922 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:41 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:41.472+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.294524+0000 front 2024-03-04T09:10:13.294377+0000 (oldest deadline 2024-03-04T09:10:34.994303+0000) 2024-03-04T09:10:41.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:41 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:41.538+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:41.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:41 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:41.538+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:41.923 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:41 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:41.538+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:42.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:41 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:41.685+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:42.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:41 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:41.685+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:42.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:41 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:41.685+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:10:42.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:42 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:42.217+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:42.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:42 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:42.217+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.172054+0000 front 2024-03-04T09:10:13.172023+0000 (oldest deadline 2024-03-04T09:10:39.072075+0000) 2024-03-04T09:10:42.552 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:42 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:42.445+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:42.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:42 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:42.445+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:10:42.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:42 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:42.235+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:42.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:42 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:42.235+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:42.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:42 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:42.515+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:42.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:42 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:42.515+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:42.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:42 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:42.515+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:42.922 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:42 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:42.483+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.893223+0000 front 2024-03-04T09:10:03.892986+0000 (oldest deadline 2024-03-04T09:10:28.593135+0000) 2024-03-04T09:10:42.922 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:42 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:42.483+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.294524+0000 front 2024-03-04T09:10:13.294377+0000 (oldest deadline 2024-03-04T09:10:34.994303+0000) 2024-03-04T09:10:43.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:42 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:42.645+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:43.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:42 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:42.645+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:43.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:42 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:42.645+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:10:43.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:43 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:43.236+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:43.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:43 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:43.236+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.172054+0000 front 2024-03-04T09:10:13.172023+0000 (oldest deadline 2024-03-04T09:10:39.072075+0000) 2024-03-04T09:10:43.552 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:43 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:43.462+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:43.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:43 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:43.462+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:10:43.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:43 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:43.223+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:43.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:43 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:43.223+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:43.922 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:43 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:43.500+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.893223+0000 front 2024-03-04T09:10:03.892986+0000 (oldest deadline 2024-03-04T09:10:28.593135+0000) 2024-03-04T09:10:43.922 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:43 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:43.500+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.294524+0000 front 2024-03-04T09:10:13.294377+0000 (oldest deadline 2024-03-04T09:10:34.994303+0000) 2024-03-04T09:10:43.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:43 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:43.487+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:43.923 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:43 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:43.487+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:43.923 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:43 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:43.487+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:44.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:43 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:43.622+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:44.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:43 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:43.622+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:44.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:43 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:43.622+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:10:44.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:44 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:44.200+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:44.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:44 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:44.200+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.172054+0000 front 2024-03-04T09:10:13.172023+0000 (oldest deadline 2024-03-04T09:10:39.072075+0000) 2024-03-04T09:10:44.552 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:44 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:44.437+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:44.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:44 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:44.437+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:10:44.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:44 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:44.247+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:44.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:44 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:44.247+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:44.921 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:44 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:44.478+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.893223+0000 front 2024-03-04T09:10:03.892986+0000 (oldest deadline 2024-03-04T09:10:28.593135+0000) 2024-03-04T09:10:44.922 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:44 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:44.478+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.294524+0000 front 2024-03-04T09:10:13.294377+0000 (oldest deadline 2024-03-04T09:10:34.994303+0000) 2024-03-04T09:10:44.922 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:44 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:44.478+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:19.095052+0000 front 2024-03-04T09:10:19.094959+0000 (oldest deadline 2024-03-04T09:10:43.795154+0000) 2024-03-04T09:10:44.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:44 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:44.482+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:44.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:44 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:44.482+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:44.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:44 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:44.482+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:45.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:44 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:44.644+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:45.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:44 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:44.644+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:45.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:44 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:44.644+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:10:45.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:44 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:44.644+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:10:45.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:45 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:45.234+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:45.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:45 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:45.234+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.172054+0000 front 2024-03-04T09:10:13.172023+0000 (oldest deadline 2024-03-04T09:10:39.072075+0000) 2024-03-04T09:10:45.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:45 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:45.234+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:19.072850+0000 front 2024-03-04T09:10:19.072670+0000 (oldest deadline 2024-03-04T09:10:44.372680+0000) 2024-03-04T09:10:45.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:45 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:45.256+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:45.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:45 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:45.256+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:45.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:45 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:45.426+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:45.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:45 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:45.426+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:10:45.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:45 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:45.426+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:10:45.924 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:45 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:45.480+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.893223+0000 front 2024-03-04T09:10:03.892986+0000 (oldest deadline 2024-03-04T09:10:28.593135+0000) 2024-03-04T09:10:45.924 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:45 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:45.480+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.294524+0000 front 2024-03-04T09:10:13.294377+0000 (oldest deadline 2024-03-04T09:10:34.994303+0000) 2024-03-04T09:10:45.924 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:45 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6[60742]: 2024-03-04T09:10:45.480+0000 7f106194c700 -1 osd.6 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:19.095052+0000 front 2024-03-04T09:10:19.094959+0000 (oldest deadline 2024-03-04T09:10:43.795154+0000) 2024-03-04T09:10:45.925 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:45 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:45.475+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:45.925 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:45 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:45.475+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:45.925 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:45 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:45.475+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:45.925 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:45 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:45.475+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:10:46.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:45 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:45.624+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:46.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:45 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:45.624+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:46.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:45 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:45.624+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:10:46.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:45 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:45.624+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:10:46.421 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:46 smithi071 podman[92947]: 2024-03-04 09:10:46.172091126 +0000 UTC m=+5.683823165 container died 2870692d0fc2e2b7b7f6dc0841ff65bc2ab1a8306cd53bf9ce9668942d48bf33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, CEPH_POINT_RELEASE=) 2024-03-04T09:10:46.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:46 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:46.226+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:46.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:46 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:46.226+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.172054+0000 front 2024-03-04T09:10:13.172023+0000 (oldest deadline 2024-03-04T09:10:39.072075+0000) 2024-03-04T09:10:46.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:46 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:46.226+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:19.072850+0000 front 2024-03-04T09:10:19.072670+0000 (oldest deadline 2024-03-04T09:10:44.372680+0000) 2024-03-04T09:10:46.552 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:46 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:46.302+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:46.552 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:46 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:46.302+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:46.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:46 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:46.415+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:46.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:46 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:46.415+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:10:46.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:46 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:46.415+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:10:46.822 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:46 smithi071 podman[92947]: 2024-03-04 09:10:46.566798501 +0000 UTC m=+6.078530547 container cleanup 2870692d0fc2e2b7b7f6dc0841ff65bc2ab1a8306cd53bf9ce9668942d48bf33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240222) 2024-03-04T09:10:46.822 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:46 smithi071 bash[92947]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6 2024-03-04T09:10:46.822 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:46 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:46.510+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:46.823 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:46 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:46.510+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:46.823 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:46 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:46.510+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:46.823 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:46 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:46.510+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:10:47.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:46 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:46.609+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:47.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:46 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:46.609+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:47.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:46 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:46.609+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:10:47.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:46 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:46.609+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:10:47.169 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:46 smithi071 podman[92964]: 2024-03-04 09:10:46.826710328 +0000 UTC m=+0.650914085 container remove 2870692d0fc2e2b7b7f6dc0841ff65bc2ab1a8306cd53bf9ce9668942d48bf33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6, ceph=True, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:10:47.422 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:47 smithi071 podman[93054]: 2024-03-04 09:10:47.076687316 +0000 UTC m=+0.017877381 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:10:47.422 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:47 smithi071 podman[93054]: 2024-03-04 09:10:47.194428197 +0000 UTC m=+0.135618245 container create a6bf40d5305ce5750f4d9924e63a530648258631618056930a08eacee62fb4ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux ) 2024-03-04T09:10:47.422 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:47 smithi071 podman[93054]: 2024-03-04 09:10:47.413658971 +0000 UTC m=+0.354849019 container init a6bf40d5305ce5750f4d9924e63a530648258631618056930a08eacee62fb4ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6-deactivate, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-04T09:10:47.422 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:47 smithi071 podman[93054]: 2024-03-04 09:10:47.416656062 +0000 UTC m=+0.357846111 container start a6bf40d5305ce5750f4d9924e63a530648258631618056930a08eacee62fb4ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:10:47.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:47 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:47.198+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:47.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:47 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:47.198+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.172054+0000 front 2024-03-04T09:10:13.172023+0000 (oldest deadline 2024-03-04T09:10:39.072075+0000) 2024-03-04T09:10:47.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:47 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:47.198+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:19.072850+0000 front 2024-03-04T09:10:19.072670+0000 (oldest deadline 2024-03-04T09:10:44.372680+0000) 2024-03-04T09:10:47.552 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:47 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:47.270+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:47.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:47 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:47.270+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:47.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:47 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:47.270+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.878465+0000 front 2024-03-04T09:10:20.878384+0000 (oldest deadline 2024-03-04T09:10:46.778311+0000) 2024-03-04T09:10:47.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:47 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:47.374+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:47.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:47 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:47.374+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:10:47.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:47 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:47.374+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:10:47.922 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:47 smithi071 podman[93054]: 2024-03-04 09:10:47.513875336 +0000 UTC m=+0.455065385 container attach a6bf40d5305ce5750f4d9924e63a530648258631618056930a08eacee62fb4ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222) 2024-03-04T09:10:47.922 INFO:journalctl@ceph.osd.6.smithi071.stdout:Mar 04 09:10:47 smithi071 podman[93054]: 2024-03-04 09:10:47.700671057 +0000 UTC m=+0.641861117 container died a6bf40d5305ce5750f4d9924e63a530648258631618056930a08eacee62fb4ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-6-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222) 2024-03-04T09:10:47.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:47 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:47.537+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:47.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:47 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:47.537+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:47.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:47 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:47.537+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:47.923 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:47 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:47.537+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:10:48.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:47 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:47.629+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:48.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:47 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:47.629+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:48.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:47 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:47.629+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:10:48.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:47 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:47.629+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:10:48.369 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.6.service' 2024-03-04T09:10:48.411 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:10:48.412 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-04T09:10:48.412 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-04T09:10:48.412 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.7 2024-03-04T09:10:48.440 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:48 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:48.187+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:48.440 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:48 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:48.187+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.172054+0000 front 2024-03-04T09:10:13.172023+0000 (oldest deadline 2024-03-04T09:10:39.072075+0000) 2024-03-04T09:10:48.440 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:48 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:48.187+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:19.072850+0000 front 2024-03-04T09:10:19.072670+0000 (oldest deadline 2024-03-04T09:10:44.372680+0000) 2024-03-04T09:10:48.552 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:48 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:48.263+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:48.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:48 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:48.263+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:48.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:48 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:48.263+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.878465+0000 front 2024-03-04T09:10:20.878384+0000 (oldest deadline 2024-03-04T09:10:46.778311+0000) 2024-03-04T09:10:48.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:48 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:48.366+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:48.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:48 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:48.366+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:10:48.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:48 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:48.366+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:10:48.671 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:48 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:48.555+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:48.672 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:48 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:48.555+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:48.672 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:48 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:48.555+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:48.672 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:48 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:48.555+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:10:48.715 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:48 smithi150 systemd[1]: Stopping Ceph osd.7 for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:10:48.715 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:48 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:48.655+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:48.715 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:48 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:48.655+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:48.716 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:48 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:48.655+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:10:48.716 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:48 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:48.655+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:10:49.030 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:48 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:48.718+0000 7f11b175e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:10:49.030 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:48 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:48.718+0000 7f11b175e700 -1 osd.7 108 *** Got signal Terminated *** 2024-03-04T09:10:49.030 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:48 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:48.718+0000 7f11b175e700 -1 osd.7 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T09:10:49.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:49 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:49.202+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:49.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:49 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:49.202+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.172054+0000 front 2024-03-04T09:10:13.172023+0000 (oldest deadline 2024-03-04T09:10:39.072075+0000) 2024-03-04T09:10:49.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:49 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:49.202+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:19.072850+0000 front 2024-03-04T09:10:19.072670+0000 (oldest deadline 2024-03-04T09:10:44.372680+0000) 2024-03-04T09:10:49.552 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:49 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:49.297+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:49.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:49 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:49.297+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:49.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:49 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:49.297+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.878465+0000 front 2024-03-04T09:10:20.878384+0000 (oldest deadline 2024-03-04T09:10:46.778311+0000) 2024-03-04T09:10:49.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:49 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:49.392+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:49.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:49 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:49.392+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:10:49.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:49 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:49.392+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:10:49.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:49 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:49.561+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:49.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:49 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:49.561+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:49.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:49 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:49.561+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:49.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:49 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:49.561+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:10:50.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:49 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:49.676+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:50.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:49 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:49.676+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:50.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:49 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:49.676+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:10:50.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:49 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:49.676+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:10:50.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:50 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:50.161+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:50.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:50 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:50.161+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.172054+0000 front 2024-03-04T09:10:13.172023+0000 (oldest deadline 2024-03-04T09:10:39.072075+0000) 2024-03-04T09:10:50.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:50 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:50.161+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:19.072850+0000 front 2024-03-04T09:10:19.072670+0000 (oldest deadline 2024-03-04T09:10:44.372680+0000) 2024-03-04T09:10:50.552 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:50 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:50.286+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:50.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:50 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:50.286+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:50.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:50 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:50.286+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.878465+0000 front 2024-03-04T09:10:20.878384+0000 (oldest deadline 2024-03-04T09:10:46.778311+0000) 2024-03-04T09:10:50.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:50 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:50.405+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:50.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:50 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:50.405+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:10:50.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:50 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:50.405+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:10:50.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:50 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:50.405+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:10:50.921 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:50 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:50.604+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:50.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:50 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:50.604+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:50.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:50 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:50.604+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:50.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:50 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:50.604+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:10:51.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:50 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:50.643+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:51.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:50 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:50.643+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:51.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:50 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:50.643+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:10:51.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:50 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:50.643+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:10:51.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:50 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:50.643+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:10:51.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:51 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:51.142+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:51.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:51 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:51.142+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.172054+0000 front 2024-03-04T09:10:13.172023+0000 (oldest deadline 2024-03-04T09:10:39.072075+0000) 2024-03-04T09:10:51.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:51 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:51.142+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:19.072850+0000 front 2024-03-04T09:10:19.072670+0000 (oldest deadline 2024-03-04T09:10:44.372680+0000) 2024-03-04T09:10:51.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:51 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:51.142+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:26.073637+0000 front 2024-03-04T09:10:26.073657+0000 (oldest deadline 2024-03-04T09:10:50.173670+0000) 2024-03-04T09:10:51.552 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:51 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:51.282+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:51.552 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:51 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:51.282+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:51.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:51 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:51.282+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.878465+0000 front 2024-03-04T09:10:20.878384+0000 (oldest deadline 2024-03-04T09:10:46.778311+0000) 2024-03-04T09:10:51.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:51 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:51.357+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:51.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:51 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:51.357+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:10:51.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:51 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:51.357+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:10:51.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:51 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:51.357+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:10:51.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:51 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:51.605+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:51.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:51 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:51.605+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:51.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:51 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:51.605+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:51.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:51 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:51.605+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:10:51.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:51 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:51.605+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:25.406003+0000 front 2024-03-04T09:10:25.406059+0000 (oldest deadline 2024-03-04T09:10:51.305879+0000) 2024-03-04T09:10:52.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:51 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:51.637+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:52.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:51 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:51.637+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:52.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:51 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:51.637+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:10:52.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:51 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:51.637+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:10:52.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:51 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:51.637+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:10:52.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:52 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:52.131+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:52.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:52 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:52.131+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.172054+0000 front 2024-03-04T09:10:13.172023+0000 (oldest deadline 2024-03-04T09:10:39.072075+0000) 2024-03-04T09:10:52.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:52 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:52.131+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:19.072850+0000 front 2024-03-04T09:10:19.072670+0000 (oldest deadline 2024-03-04T09:10:44.372680+0000) 2024-03-04T09:10:52.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:52 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:52.131+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:26.073637+0000 front 2024-03-04T09:10:26.073657+0000 (oldest deadline 2024-03-04T09:10:50.173670+0000) 2024-03-04T09:10:52.803 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:52 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:52.323+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:52.804 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:52 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:52.323+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:52.804 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:52 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:52.323+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.878465+0000 front 2024-03-04T09:10:20.878384+0000 (oldest deadline 2024-03-04T09:10:46.778311+0000) 2024-03-04T09:10:52.804 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:52 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:52.323+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:26.778850+0000 front 2024-03-04T09:10:26.778879+0000 (oldest deadline 2024-03-04T09:10:52.078822+0000) 2024-03-04T09:10:52.804 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:52 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:52.318+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:52.804 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:52 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:52.318+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:10:52.804 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:52 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:52.318+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:10:52.804 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:52 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:52.318+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:10:52.921 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:52 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:52.564+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:52.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:52 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:52.564+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:52.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:52 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:52.564+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:52.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:52 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:52.564+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:10:52.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:52 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:52.564+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:25.406003+0000 front 2024-03-04T09:10:25.406059+0000 (oldest deadline 2024-03-04T09:10:51.305879+0000) 2024-03-04T09:10:53.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:52 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:52.662+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:53.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:52 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:52.662+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:53.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:52 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:52.662+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:10:53.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:52 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:52.662+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:10:53.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:52 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:52.662+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:10:53.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:53 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:53.168+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:09:59.968728+0000 front 2024-03-04T09:09:59.968754+0000 (oldest deadline 2024-03-04T09:10:25.868928+0000) 2024-03-04T09:10:53.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:53 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:53.168+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.172054+0000 front 2024-03-04T09:10:13.172023+0000 (oldest deadline 2024-03-04T09:10:39.072075+0000) 2024-03-04T09:10:53.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:53 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:53.168+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:19.072850+0000 front 2024-03-04T09:10:19.072670+0000 (oldest deadline 2024-03-04T09:10:44.372680+0000) 2024-03-04T09:10:53.530 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:53 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7[54805]: 2024-03-04T09:10:53.168+0000 7f11add52700 -1 osd.7 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:26.073637+0000 front 2024-03-04T09:10:26.073657+0000 (oldest deadline 2024-03-04T09:10:50.173670+0000) 2024-03-04T09:10:53.552 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:53 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:53.305+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:53.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:53 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:53.305+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:53.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:53 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:53.305+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.878465+0000 front 2024-03-04T09:10:20.878384+0000 (oldest deadline 2024-03-04T09:10:46.778311+0000) 2024-03-04T09:10:53.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:53 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:53.305+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:26.778850+0000 front 2024-03-04T09:10:26.778879+0000 (oldest deadline 2024-03-04T09:10:52.078822+0000) 2024-03-04T09:10:53.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:53 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:53.277+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:53.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:53 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:53.277+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:10:53.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:53 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:53.277+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:10:53.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:53 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:53.277+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:10:53.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:53 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:53.522+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:53.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:53 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:53.522+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:53.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:53 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:53.522+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:53.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:53 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:53.522+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:10:53.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:53 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:53.522+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:25.406003+0000 front 2024-03-04T09:10:25.406059+0000 (oldest deadline 2024-03-04T09:10:51.305879+0000) 2024-03-04T09:10:54.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:53 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:53.686+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:54.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:53 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:53.686+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:54.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:53 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:53.686+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:10:54.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:53 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:53.686+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:10:54.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:53 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:53.686+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:10:54.474 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:54 smithi150 podman[72820]: 2024-03-04 09:10:54.157258437 +0000 UTC m=+5.554249917 container died f13f8b97a6c7152981868f01ea0699a0da86978767025d540bb6eb0f5f9f8dfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:10:54.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:54 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:54.285+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:54.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:54 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:54.285+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:54.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:54 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:54.285+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.878465+0000 front 2024-03-04T09:10:20.878384+0000 (oldest deadline 2024-03-04T09:10:46.778311+0000) 2024-03-04T09:10:54.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:54 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:54.285+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:26.778850+0000 front 2024-03-04T09:10:26.778879+0000 (oldest deadline 2024-03-04T09:10:52.078822+0000) 2024-03-04T09:10:54.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:54 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:54.279+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:54.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:54 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:54.279+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:10:54.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:54 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:54.279+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:10:54.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:54 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:54.279+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:10:54.780 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:54 smithi150 podman[72820]: 2024-03-04 09:10:54.585535188 +0000 UTC m=+5.982526666 container cleanup f13f8b97a6c7152981868f01ea0699a0da86978767025d540bb6eb0f5f9f8dfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:10:54.780 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:54 smithi150 bash[72820]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7 2024-03-04T09:10:54.780 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:54 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:54.653+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:54.781 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:54 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:54.653+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:54.781 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:54 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:54.653+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:10:54.781 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:54 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:54.653+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:10:54.781 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:54 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:54.653+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:10:54.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:54 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:54.530+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:54.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:54 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:54.530+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:54.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:54 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:54.530+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:54.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:54 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:54.530+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:10:54.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:54 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:54.530+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:25.406003+0000 front 2024-03-04T09:10:25.406059+0000 (oldest deadline 2024-03-04T09:10:51.305879+0000) 2024-03-04T09:10:55.243 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:54 smithi150 podman[72837]: 2024-03-04 09:10:54.817355912 +0000 UTC m=+0.660690954 container remove f13f8b97a6c7152981868f01ea0699a0da86978767025d540bb6eb0f5f9f8dfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, org.label-schema.build-date=20240222, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-04T09:10:55.494 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:55 smithi150 podman[72930]: 2024-03-04 09:10:55.150974149 +0000 UTC m=+0.017640426 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:10:55.494 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:55 smithi150 podman[72930]: 2024-03-04 09:10:55.262759264 +0000 UTC m=+0.129425550 container create 544a559aefbd02bb1e64370ffc1dff5c65b90e1d0db81340f6a6ec35168b40c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240222, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:10:55.494 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:55 smithi150 podman[72930]: 2024-03-04 09:10:55.498475948 +0000 UTC m=+0.365142220 container init 544a559aefbd02bb1e64370ffc1dff5c65b90e1d0db81340f6a6ec35168b40c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7-deactivate, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2) 2024-03-04T09:10:55.552 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:55 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:55.312+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:55.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:55 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:55.312+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:55.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:55 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:55.312+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.878465+0000 front 2024-03-04T09:10:20.878384+0000 (oldest deadline 2024-03-04T09:10:46.778311+0000) 2024-03-04T09:10:55.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:55 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:55.312+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:26.778850+0000 front 2024-03-04T09:10:26.778879+0000 (oldest deadline 2024-03-04T09:10:52.078822+0000) 2024-03-04T09:10:55.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:55 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:55.244+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:55.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:55 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:55.244+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:10:55.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:55 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:55.244+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:10:55.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:55 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:55.244+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:10:55.780 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:55 smithi150 podman[72930]: 2024-03-04 09:10:55.501790295 +0000 UTC m=+0.368456557 container start 544a559aefbd02bb1e64370ffc1dff5c65b90e1d0db81340f6a6ec35168b40c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, ceph=True) 2024-03-04T09:10:55.780 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:55 smithi150 podman[72930]: 2024-03-04 09:10:55.581822733 +0000 UTC m=+0.448489010 container attach 544a559aefbd02bb1e64370ffc1dff5c65b90e1d0db81340f6a6ec35168b40c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, ceph=True, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-04T09:10:55.780 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:55 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:55.609+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:55.781 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:55 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:55.609+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:55.781 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:55 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:55.609+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:10:55.781 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:55 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:55.609+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:10:55.781 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:55 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:55.609+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:10:55.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:55 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:55.502+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:55.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:55 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:55.502+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:55.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:55 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:55.502+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:55.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:55 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:55.502+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:10:55.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:55 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:55.502+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:25.406003+0000 front 2024-03-04T09:10:25.406059+0000 (oldest deadline 2024-03-04T09:10:51.305879+0000) 2024-03-04T09:10:56.097 INFO:journalctl@ceph.osd.7.smithi150.stdout:Mar 04 09:10:55 smithi150 podman[72930]: 2024-03-04 09:10:55.7973361 +0000 UTC m=+0.664002368 container died 544a559aefbd02bb1e64370ffc1dff5c65b90e1d0db81340f6a6ec35168b40c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-7-deactivate, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-04T09:10:56.428 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.7.service' 2024-03-04T09:10:56.511 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:10:56.511 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-04T09:10:56.512 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-04T09:10:56.512 DEBUG:teuthology.orchestra.run.smithi158:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.8 2024-03-04T09:10:56.542 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:56 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:56.308+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:56.542 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:56 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:56.308+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:56.543 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:56 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:56.308+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.878465+0000 front 2024-03-04T09:10:20.878384+0000 (oldest deadline 2024-03-04T09:10:46.778311+0000) 2024-03-04T09:10:56.543 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:56 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:56.308+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:26.778850+0000 front 2024-03-04T09:10:26.778879+0000 (oldest deadline 2024-03-04T09:10:52.078822+0000) 2024-03-04T09:10:56.543 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:56 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:56.260+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:56.543 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:56 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:56.260+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:10:56.543 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:56 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:56.260+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:10:56.543 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:56 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:56.260+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:10:56.687 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:56 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:56.639+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:56.687 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:56 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:56.639+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:56.687 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:56 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:56.639+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:10:56.687 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:56 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:56.639+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:10:56.688 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:56 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:56.639+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:10:56.802 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:56 smithi158 systemd[1]: Stopping Ceph osd.8 for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:10:56.921 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:56 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:56.516+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:56.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:56 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:56.516+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:56.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:56 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:56.516+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:56.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:56 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:56.516+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:10:56.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:56 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:56.516+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:25.406003+0000 front 2024-03-04T09:10:25.406059+0000 (oldest deadline 2024-03-04T09:10:51.305879+0000) 2024-03-04T09:10:57.237 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:56 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:56.822+0000 7f42d67d0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:10:57.238 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:56 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:56.822+0000 7f42d67d0700 -1 osd.8 108 *** Got signal Terminated *** 2024-03-04T09:10:57.238 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:56 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:56.822+0000 7f42d67d0700 -1 osd.8 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T09:10:57.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:57 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:57.293+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:57.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:57 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:57.293+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:57.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:57 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:57.293+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.878465+0000 front 2024-03-04T09:10:20.878384+0000 (oldest deadline 2024-03-04T09:10:46.778311+0000) 2024-03-04T09:10:57.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:57 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:57.293+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:26.778850+0000 front 2024-03-04T09:10:26.778879+0000 (oldest deadline 2024-03-04T09:10:52.078822+0000) 2024-03-04T09:10:57.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:57 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:57.241+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:57.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:57 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:57.241+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:10:57.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:57 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:57.241+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:10:57.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:57 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:57.241+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:10:57.921 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:57 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:57.529+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:57.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:57 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:57.529+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:57.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:57 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:57.529+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:57.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:57 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:57.529+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:10:57.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:57 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:57.529+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:25.406003+0000 front 2024-03-04T09:10:25.406059+0000 (oldest deadline 2024-03-04T09:10:51.305879+0000) 2024-03-04T09:10:58.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:57 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:57.607+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:58.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:57 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:57.607+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:58.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:57 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:57.607+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:10:58.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:57 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:57.607+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:10:58.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:57 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:57.607+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:10:58.552 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:58 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:58.244+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:58.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:58 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:58.244+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:58.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:58 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:58.244+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.878465+0000 front 2024-03-04T09:10:20.878384+0000 (oldest deadline 2024-03-04T09:10:46.778311+0000) 2024-03-04T09:10:58.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:58 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:58.244+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:26.778850+0000 front 2024-03-04T09:10:26.778879+0000 (oldest deadline 2024-03-04T09:10:52.078822+0000) 2024-03-04T09:10:58.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:58 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:58.200+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:58.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:58 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:58.200+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:10:58.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:58 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:58.200+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:10:58.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:58 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:58.200+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:10:58.921 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:58 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:58.500+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:58.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:58 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:58.500+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:58.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:58 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:58.500+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:58.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:58 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:58.500+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:10:58.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:58 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:58.500+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:25.406003+0000 front 2024-03-04T09:10:25.406059+0000 (oldest deadline 2024-03-04T09:10:51.305879+0000) 2024-03-04T09:10:59.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:58 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:58.594+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:10:59.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:58 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:58.594+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:10:59.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:58 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:58.594+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:10:59.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:58 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:58.594+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:10:59.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:58 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:58.594+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:10:59.552 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:59 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:59.260+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:10:59.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:59 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:59.260+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:10:59.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:59 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:59.260+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.878465+0000 front 2024-03-04T09:10:20.878384+0000 (oldest deadline 2024-03-04T09:10:46.778311+0000) 2024-03-04T09:10:59.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:10:59 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:10:59.260+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:26.778850+0000 front 2024-03-04T09:10:26.778879+0000 (oldest deadline 2024-03-04T09:10:52.078822+0000) 2024-03-04T09:10:59.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:59 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:59.189+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:10:59.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:59 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:59.189+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:10:59.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:59 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:59.189+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:10:59.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:59 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:59.189+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:10:59.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:10:59 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:10:59.189+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:10:59.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:59 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:59.527+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:10:59.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:59 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:59.527+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:10:59.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:59 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:59.527+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:10:59.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:59 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:59.527+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:10:59.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:59 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:59.527+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:25.406003+0000 front 2024-03-04T09:10:25.406059+0000 (oldest deadline 2024-03-04T09:10:51.305879+0000) 2024-03-04T09:10:59.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:10:59 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:10:59.527+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:36.007202+0000 front 2024-03-04T09:10:36.007029+0000 (oldest deadline 2024-03-04T09:10:58.906825+0000) 2024-03-04T09:11:00.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:59 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:59.599+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:11:00.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:59 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:59.599+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:11:00.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:59 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:59.599+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:11:00.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:59 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:59.599+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:11:00.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:10:59 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:10:59.599+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:11:00.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:11:00 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:11:00.219+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:11:00.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:11:00 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:11:00.219+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:11:00.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:11:00 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:11:00.219+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.878465+0000 front 2024-03-04T09:10:20.878384+0000 (oldest deadline 2024-03-04T09:10:46.778311+0000) 2024-03-04T09:11:00.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:11:00 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:11:00.219+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:26.778850+0000 front 2024-03-04T09:10:26.778879+0000 (oldest deadline 2024-03-04T09:10:52.078822+0000) 2024-03-04T09:11:00.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:00 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:00.187+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:00.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:00 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:00.187+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:00.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:00 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:00.187+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:00.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:00 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:00.187+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:00.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:00 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:00.187+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:00.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:00.541+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:11:00.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:00.541+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:11:00.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:00.541+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:11:00.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:00.541+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:11:00.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:00.541+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:25.406003+0000 front 2024-03-04T09:10:25.406059+0000 (oldest deadline 2024-03-04T09:10:51.305879+0000) 2024-03-04T09:11:00.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:00 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:00.541+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:36.007202+0000 front 2024-03-04T09:10:36.007029+0000 (oldest deadline 2024-03-04T09:10:58.906825+0000) 2024-03-04T09:11:01.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:00 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:00.645+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:11:01.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:00 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:00.645+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:11:01.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:00 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:00.645+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:11:01.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:00 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:00.645+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:11:01.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:00 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:00.645+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:11:01.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:00 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:00.645+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:35.597022+0000 front 2024-03-04T09:10:35.597102+0000 (oldest deadline 2024-03-04T09:10:59.697008+0000) 2024-03-04T09:11:01.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:11:01 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:11:01.244+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:03.375512+0000 front 2024-03-04T09:10:03.375631+0000 (oldest deadline 2024-03-04T09:10:28.075559+0000) 2024-03-04T09:11:01.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:11:01 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:11:01.244+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:09.776656+0000 front 2024-03-04T09:10:09.776696+0000 (oldest deadline 2024-03-04T09:10:35.076806+0000) 2024-03-04T09:11:01.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:11:01 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:11:01.244+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.878465+0000 front 2024-03-04T09:10:20.878384+0000 (oldest deadline 2024-03-04T09:10:46.778311+0000) 2024-03-04T09:11:01.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:11:01 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:11:01.244+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:26.778850+0000 front 2024-03-04T09:10:26.778879+0000 (oldest deadline 2024-03-04T09:10:52.078822+0000) 2024-03-04T09:11:01.553 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:11:01 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8[55981]: 2024-03-04T09:11:01.244+0000 7f42d2dc4700 -1 osd.8 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:36.580723+0000 front 2024-03-04T09:10:36.580797+0000 (oldest deadline 2024-03-04T09:11:00.581587+0000) 2024-03-04T09:11:01.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:01 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:01.227+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:01.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:01 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:01.227+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:01.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:01 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:01.227+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:01.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:01 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:01.227+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:01.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:01 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:01.227+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:01.921 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:01 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:01.571+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:11:01.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:01 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:01.571+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:11:01.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:01 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:01.571+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:11:01.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:01 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:01.571+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:11:01.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:01 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:01.571+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:25.406003+0000 front 2024-03-04T09:10:25.406059+0000 (oldest deadline 2024-03-04T09:10:51.305879+0000) 2024-03-04T09:11:01.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:01 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:01.571+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:36.007202+0000 front 2024-03-04T09:10:36.007029+0000 (oldest deadline 2024-03-04T09:10:58.906825+0000) 2024-03-04T09:11:02.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:01 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:01.624+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:11:02.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:01 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:01.624+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:11:02.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:01 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:01.624+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:11:02.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:01 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:01.624+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:11:02.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:01 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:01.624+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:11:02.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:01 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:01.624+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:35.597022+0000 front 2024-03-04T09:10:35.597102+0000 (oldest deadline 2024-03-04T09:10:59.697008+0000) 2024-03-04T09:11:02.552 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:11:02 smithi158 podman[75139]: 2024-03-04 09:11:02.339516354 +0000 UTC m=+5.628015277 container died 13828b58057f479bca7be4be6edf28d151835092b202b809d986a2eb59b39502 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-04T09:11:02.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:02 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:02.243+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:02.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:02 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:02.243+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:02.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:02 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:02.243+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:02.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:02 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:02.243+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:02.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:02 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:02.243+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:02.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:02 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:02.532+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:11:02.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:02 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:02.532+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:11:02.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:02 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:02.532+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:11:02.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:02 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:02.532+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:11:02.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:02 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:02.532+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:25.406003+0000 front 2024-03-04T09:10:25.406059+0000 (oldest deadline 2024-03-04T09:10:51.305879+0000) 2024-03-04T09:11:02.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:02 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:02.532+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:36.007202+0000 front 2024-03-04T09:10:36.007029+0000 (oldest deadline 2024-03-04T09:10:58.906825+0000) 2024-03-04T09:11:02.944 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:11:02 smithi158 podman[75139]: 2024-03-04 09:11:02.714503843 +0000 UTC m=+6.003002754 container cleanup 13828b58057f479bca7be4be6edf28d151835092b202b809d986a2eb59b39502 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-04T09:11:02.944 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:11:02 smithi158 bash[75139]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8 2024-03-04T09:11:03.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:02 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:02.661+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:11:03.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:02 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:02.661+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:11:03.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:02 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:02.661+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:11:03.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:02 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:02.661+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:11:03.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:02 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:02.661+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:11:03.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:02 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:02.661+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:35.597022+0000 front 2024-03-04T09:10:35.597102+0000 (oldest deadline 2024-03-04T09:10:59.697008+0000) 2024-03-04T09:11:03.237 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:11:02 smithi158 podman[75156]: 2024-03-04 09:11:02.949025055 +0000 UTC m=+0.612357928 container remove 13828b58057f479bca7be4be6edf28d151835092b202b809d986a2eb59b39502 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:11:03.522 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:11:03 smithi158 podman[75246]: 2024-03-04 09:11:03.165919821 +0000 UTC m=+0.017478411 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:11:03.523 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:11:03 smithi158 podman[75246]: 2024-03-04 09:11:03.28325919 +0000 UTC m=+0.134817766 container create 140f32646e431ea92eec4232ce7cd988d84bcbbfa0ce1ffa6f9c18ac7b116e56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:11:03.523 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:03 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:03.240+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:03.523 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:03 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:03.240+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:03.523 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:03 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:03.240+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:03.523 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:03 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:03.240+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:03.524 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:03 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:03.240+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:03.803 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:11:03 smithi158 podman[75246]: 2024-03-04 09:11:03.527359294 +0000 UTC m=+0.378917870 container init 140f32646e431ea92eec4232ce7cd988d84bcbbfa0ce1ffa6f9c18ac7b116e56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=) 2024-03-04T09:11:03.803 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:11:03 smithi158 podman[75246]: 2024-03-04 09:11:03.530522194 +0000 UTC m=+0.382080771 container start 140f32646e431ea92eec4232ce7cd988d84bcbbfa0ce1ffa6f9c18ac7b116e56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:11:03.803 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:11:03 smithi158 podman[75246]: 2024-03-04 09:11:03.602309993 +0000 UTC m=+0.453868584 container attach 140f32646e431ea92eec4232ce7cd988d84bcbbfa0ce1ffa6f9c18ac7b116e56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222) 2024-03-04T09:11:03.921 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:03 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:03.545+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:11:03.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:03 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:03.545+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:11:03.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:03 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:03.545+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:11:03.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:03 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:03.545+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:11:03.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:03 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:03.545+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:25.406003+0000 front 2024-03-04T09:10:25.406059+0000 (oldest deadline 2024-03-04T09:10:51.305879+0000) 2024-03-04T09:11:03.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:03 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:03.545+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:36.007202+0000 front 2024-03-04T09:10:36.007029+0000 (oldest deadline 2024-03-04T09:10:58.906825+0000) 2024-03-04T09:11:04.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:03 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:03.683+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:11:04.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:03 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:03.683+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:11:04.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:03 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:03.683+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:11:04.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:03 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:03.683+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:11:04.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:03 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:03.683+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:11:04.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:03 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:03.683+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:35.597022+0000 front 2024-03-04T09:10:35.597102+0000 (oldest deadline 2024-03-04T09:10:59.697008+0000) 2024-03-04T09:11:04.117 INFO:journalctl@ceph.osd.8.smithi158.stdout:Mar 04 09:11:03 smithi158 podman[75246]: 2024-03-04 09:11:03.818808185 +0000 UTC m=+0.670366765 container died 140f32646e431ea92eec4232ce7cd988d84bcbbfa0ce1ffa6f9c18ac7b116e56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-04T09:11:04.443 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:04 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:04.195+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:04.444 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:04 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:04.195+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:04.444 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:04 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:04.195+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:04.444 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:04 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:04.195+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:04.444 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:04 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:04.195+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:04.465 DEBUG:teuthology.orchestra.run.smithi158:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.8.service' 2024-03-04T09:11:04.547 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:11:04.548 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-04T09:11:04.548 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-04T09:11:04.548 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.9 2024-03-04T09:11:04.865 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:04 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:04.510+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:11:04.866 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:04 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:04.510+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:11:04.866 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:04 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:04.510+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:11:04.866 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:04 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:04.510+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:11:04.866 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:04 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:04.510+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:25.406003+0000 front 2024-03-04T09:10:25.406059+0000 (oldest deadline 2024-03-04T09:10:51.305879+0000) 2024-03-04T09:11:04.866 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:04 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:04.510+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:36.007202+0000 front 2024-03-04T09:10:36.007029+0000 (oldest deadline 2024-03-04T09:10:58.906825+0000) 2024-03-04T09:11:04.866 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:04 smithi071 systemd[1]: Stopping Ceph osd.9 for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:11:05.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:04 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:04.643+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:11:05.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:04 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:04.643+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:11:05.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:04 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:04.643+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:11:05.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:04 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:04.643+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:11:05.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:04 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:04.643+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:11:05.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:04 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:04.643+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:35.597022+0000 front 2024-03-04T09:10:35.597102+0000 (oldest deadline 2024-03-04T09:10:59.697008+0000) 2024-03-04T09:11:05.171 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:04 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:04.869+0000 7f4fbf745700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:11:05.172 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:04 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:04.869+0000 7f4fbf745700 -1 osd.9 108 *** Got signal Terminated *** 2024-03-04T09:11:05.172 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:04 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:04.869+0000 7f4fbf745700 -1 osd.9 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T09:11:05.552 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:05 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:05.194+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:05.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:05 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:05.194+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:05.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:05 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:05.194+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:05.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:05 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:05.194+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:05.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:05 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:05.194+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:05.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:05 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:05.483+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:11:05.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:05 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:05.483+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:11:05.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:05 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:05.483+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:11:05.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:05 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:05.483+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:11:05.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:05 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:05.483+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:25.406003+0000 front 2024-03-04T09:10:25.406059+0000 (oldest deadline 2024-03-04T09:10:51.305879+0000) 2024-03-04T09:11:05.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:05 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:05.483+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:36.007202+0000 front 2024-03-04T09:10:36.007029+0000 (oldest deadline 2024-03-04T09:10:58.906825+0000) 2024-03-04T09:11:06.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:05 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:05.626+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:11:06.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:05 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:05.626+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:11:06.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:05 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:05.626+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:11:06.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:05 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:05.626+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:11:06.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:05 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:05.626+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:11:06.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:05 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:05.626+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:35.597022+0000 front 2024-03-04T09:10:35.597102+0000 (oldest deadline 2024-03-04T09:10:59.697008+0000) 2024-03-04T09:11:06.552 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:06 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:06.231+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:06.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:06 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:06.231+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:06.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:06 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:06.231+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:06.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:06 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:06.231+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:06.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:06 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:06.231+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:06.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:06 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:06.458+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:11:06.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:06 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:06.458+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:11:06.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:06 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:06.458+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:11:06.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:06 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:06.458+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:11:06.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:06 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:06.458+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:25.406003+0000 front 2024-03-04T09:10:25.406059+0000 (oldest deadline 2024-03-04T09:10:51.305879+0000) 2024-03-04T09:11:06.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:06 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:06.458+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:36.007202+0000 front 2024-03-04T09:10:36.007029+0000 (oldest deadline 2024-03-04T09:10:58.906825+0000) 2024-03-04T09:11:07.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:06 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:06.614+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:11:07.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:06 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:06.614+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:11:07.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:06 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:06.614+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:11:07.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:06 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:06.614+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:11:07.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:06 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:06.614+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:11:07.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:06 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:06.614+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:35.597022+0000 front 2024-03-04T09:10:35.597102+0000 (oldest deadline 2024-03-04T09:10:59.697008+0000) 2024-03-04T09:11:07.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:07 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:07.215+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:07.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:07 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:07.215+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:07.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:07 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:07.215+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:07.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:07 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:07.215+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:07.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:07 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:07.215+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:07.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:07 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:07.215+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:43.981912+0000 front 2024-03-04T09:10:43.981924+0000 (oldest deadline 2024-03-04T09:11:06.881848+0000) 2024-03-04T09:11:07.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:07 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:07.495+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:11:07.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:07 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:07.495+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:11:07.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:07 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:07.495+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:11:07.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:07 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:07.495+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:11:07.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:07 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:07.495+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:25.406003+0000 front 2024-03-04T09:10:25.406059+0000 (oldest deadline 2024-03-04T09:10:51.305879+0000) 2024-03-04T09:11:07.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:07 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:07.495+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:36.007202+0000 front 2024-03-04T09:10:36.007029+0000 (oldest deadline 2024-03-04T09:10:58.906825+0000) 2024-03-04T09:11:08.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:07 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:07.620+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:11:08.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:07 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:07.620+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:11:08.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:07 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:07.620+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:11:08.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:07 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:07.620+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:11:08.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:07 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:07.620+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:11:08.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:07 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:07.620+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:35.597022+0000 front 2024-03-04T09:10:35.597102+0000 (oldest deadline 2024-03-04T09:10:59.697008+0000) 2024-03-04T09:11:08.552 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:08 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:08.242+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:08.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:08 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:08.242+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:08.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:08 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:08.242+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:08.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:08 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:08.242+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:08.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:08 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:08.242+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:08.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:08 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:08.242+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:43.981912+0000 front 2024-03-04T09:10:43.981924+0000 (oldest deadline 2024-03-04T09:11:06.881848+0000) 2024-03-04T09:11:08.921 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:08 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:08.509+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:11:08.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:08 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:08.509+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:11:08.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:08 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:08.509+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:11:08.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:08 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:08.509+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:11:08.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:08 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:08.509+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:25.406003+0000 front 2024-03-04T09:10:25.406059+0000 (oldest deadline 2024-03-04T09:10:51.305879+0000) 2024-03-04T09:11:08.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:08 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:08.509+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:36.007202+0000 front 2024-03-04T09:10:36.007029+0000 (oldest deadline 2024-03-04T09:10:58.906825+0000) 2024-03-04T09:11:09.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:08 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:08.571+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:11:09.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:08 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:08.571+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:11:09.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:08 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:08.571+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:11:09.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:08 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:08.571+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:11:09.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:08 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:08.571+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:11:09.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:08 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:08.571+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:35.597022+0000 front 2024-03-04T09:10:35.597102+0000 (oldest deadline 2024-03-04T09:10:59.697008+0000) 2024-03-04T09:11:09.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:09 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:09.269+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:09.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:09 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:09.269+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:09.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:09 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:09.269+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:09.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:09 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:09.269+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:09.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:09 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:09.269+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:09.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:09 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:09.269+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:43.981912+0000 front 2024-03-04T09:10:43.981924+0000 (oldest deadline 2024-03-04T09:11:06.881848+0000) 2024-03-04T09:11:09.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:09 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:09.543+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:55.701337+0000 front 2024-03-04T09:09:55.701147+0000 (oldest deadline 2024-03-04T09:10:19.801323+0000) 2024-03-04T09:11:09.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:09 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:09.543+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:02.603102+0000 front 2024-03-04T09:10:02.603086+0000 (oldest deadline 2024-03-04T09:10:26.102974+0000) 2024-03-04T09:11:09.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:09 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:09.543+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:12.004025+0000 front 2024-03-04T09:10:12.003965+0000 (oldest deadline 2024-03-04T09:10:34.303971+0000) 2024-03-04T09:11:09.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:09 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:09.543+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.705273+0000 front 2024-03-04T09:10:20.705224+0000 (oldest deadline 2024-03-04T09:10:45.405440+0000) 2024-03-04T09:11:09.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:09 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:09.543+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:25.406003+0000 front 2024-03-04T09:10:25.406059+0000 (oldest deadline 2024-03-04T09:10:51.305879+0000) 2024-03-04T09:11:09.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:09 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9[67802]: 2024-03-04T09:11:09.543+0000 7f4fb7d31700 -1 osd.9 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:36.007202+0000 front 2024-03-04T09:10:36.007029+0000 (oldest deadline 2024-03-04T09:10:58.906825+0000) 2024-03-04T09:11:10.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:09 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:09.573+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:11:10.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:09 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:09.573+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:11:10.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:09 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:09.573+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:11:10.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:09 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:09.573+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:11:10.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:09 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:09.573+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:11:10.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:09 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:09.573+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:35.597022+0000 front 2024-03-04T09:10:35.597102+0000 (oldest deadline 2024-03-04T09:10:59.697008+0000) 2024-03-04T09:11:10.557 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:10 smithi071 podman[93159]: 2024-03-04 09:11:10.294995131 +0000 UTC m=+5.552267468 container died c94f5b300a1cedc218efa361e26bfbd0cb67d7fea860cdbb0b8d82a64e30cad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:11:10.907 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:10 smithi071 podman[93159]: 2024-03-04 09:11:10.669120165 +0000 UTC m=+5.926392494 container cleanup c94f5b300a1cedc218efa361e26bfbd0cb67d7fea860cdbb0b8d82a64e30cad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-03-04T09:11:10.908 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:10 smithi071 bash[93159]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9 2024-03-04T09:11:11.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:10 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:10.549+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:11:11.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:10 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:10.549+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:11:11.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:10 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:10.549+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:11:11.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:10 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:10.549+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:11:11.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:10 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:10.549+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:11:11.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:10 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:10.549+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:35.597022+0000 front 2024-03-04T09:10:35.597102+0000 (oldest deadline 2024-03-04T09:10:59.697008+0000) 2024-03-04T09:11:11.171 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:10 smithi071 podman[93176]: 2024-03-04 09:11:10.911903486 +0000 UTC m=+0.613344798 container remove c94f5b300a1cedc218efa361e26bfbd0cb67d7fea860cdbb0b8d82a64e30cad5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:11:11.544 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:11 smithi071 podman[93265]: 2024-03-04 09:11:11.129064998 +0000 UTC m=+0.018355356 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:11:11.544 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:11 smithi071 podman[93265]: 2024-03-04 09:11:11.262965229 +0000 UTC m=+0.152255581 container create 0311604160b37469f68e4102a1a8e020f1eab2e2bc51141d3ce31e60b23e0762 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222) 2024-03-04T09:11:11.545 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:11 smithi071 podman[93265]: 2024-03-04 09:11:11.465348329 +0000 UTC m=+0.354638689 container init 0311604160b37469f68e4102a1a8e020f1eab2e2bc51141d3ce31e60b23e0762 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:11:11.545 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:11 smithi071 podman[93265]: 2024-03-04 09:11:11.468412967 +0000 UTC m=+0.357703320 container start 0311604160b37469f68e4102a1a8e020f1eab2e2bc51141d3ce31e60b23e0762 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9-deactivate, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-04T09:11:11.545 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:11 smithi071 podman[93265]: 2024-03-04 09:11:11.548704015 +0000 UTC m=+0.437994383 container attach 0311604160b37469f68e4102a1a8e020f1eab2e2bc51141d3ce31e60b23e0762 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9-deactivate, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-03-04T09:11:11.922 INFO:journalctl@ceph.osd.9.smithi071.stdout:Mar 04 09:11:11 smithi071 podman[93265]: 2024-03-04 09:11:11.751552874 +0000 UTC m=+0.640843228 container died 0311604160b37469f68e4102a1a8e020f1eab2e2bc51141d3ce31e60b23e0762 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-9-deactivate, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5) 2024-03-04T09:11:12.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:11 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:11.551+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:11:12.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:11 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:11.551+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:11:12.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:11 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:11.551+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:11:12.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:11 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:11.551+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:11:12.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:11 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:11.551+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:11:12.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:11 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:11.551+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:35.597022+0000 front 2024-03-04T09:10:35.597102+0000 (oldest deadline 2024-03-04T09:10:59.697008+0000) 2024-03-04T09:11:12.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:11 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:11.551+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:45.598218+0000 front 2024-03-04T09:10:45.598269+0000 (oldest deadline 2024-03-04T09:11:10.898110+0000) 2024-03-04T09:11:12.310 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.9.service' 2024-03-04T09:11:12.352 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:11:12.353 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-04T09:11:12.353 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-04T09:11:12.353 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.10 2024-03-04T09:11:12.685 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:12 smithi150 systemd[1]: Stopping Ceph osd.10 for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:11:12.685 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:12 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:12.575+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:11:12.685 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:12 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:12.575+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:11:12.685 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:12 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:12.575+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:11:12.686 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:12 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:12.575+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:11:12.686 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:12 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:12.575+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:11:12.686 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:12 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:12.575+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:35.597022+0000 front 2024-03-04T09:10:35.597102+0000 (oldest deadline 2024-03-04T09:10:59.697008+0000) 2024-03-04T09:11:12.686 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:12 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:12.575+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:45.598218+0000 front 2024-03-04T09:10:45.598269+0000 (oldest deadline 2024-03-04T09:11:10.898110+0000) 2024-03-04T09:11:12.686 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:12 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:12.688+0000 7f2ed19f1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:11:12.686 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:12 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:12.688+0000 7f2ed19f1700 -1 osd.10 108 *** Got signal Terminated *** 2024-03-04T09:11:12.686 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:12 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:12.688+0000 7f2ed19f1700 -1 osd.10 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T09:11:13.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:10.273+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:13.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:10.274+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:13.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:10.274+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:13.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:10.274+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:13.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:10.274+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:13.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:10.274+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:43.981912+0000 front 2024-03-04T09:10:43.981924+0000 (oldest deadline 2024-03-04T09:11:06.881848+0000) 2024-03-04T09:11:13.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:11.239+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:13.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:11.239+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:13.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:11.239+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:13.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:11.239+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:13.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:11.239+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:13.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:11.239+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:43.981912+0000 front 2024-03-04T09:10:43.981924+0000 (oldest deadline 2024-03-04T09:11:06.881848+0000) 2024-03-04T09:11:13.555 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:12.287+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:13.555 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:12.287+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:13.555 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:12.287+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:13.555 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:12.287+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:13.555 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:12.287+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:13.555 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:12.287+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:43.981912+0000 front 2024-03-04T09:10:43.981924+0000 (oldest deadline 2024-03-04T09:11:06.881848+0000) 2024-03-04T09:11:13.555 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:13.261+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:13.556 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:13.261+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:13.556 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:13.261+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:13.556 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:13.261+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:13.556 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:13.261+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:13.556 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:13.261+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:43.981912+0000 front 2024-03-04T09:10:43.981924+0000 (oldest deadline 2024-03-04T09:11:06.881848+0000) 2024-03-04T09:11:14.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:13 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:13.539+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:11:14.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:13 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:13.539+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:11:14.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:13 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:13.539+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:11:14.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:13 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:13.539+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:11:14.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:13 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:13.539+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:11:14.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:13 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:13.539+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:35.597022+0000 front 2024-03-04T09:10:35.597102+0000 (oldest deadline 2024-03-04T09:10:59.697008+0000) 2024-03-04T09:11:14.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:13 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:13.539+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:45.598218+0000 front 2024-03-04T09:10:45.598269+0000 (oldest deadline 2024-03-04T09:11:10.898110+0000) 2024-03-04T09:11:14.552 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:14 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:14.232+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:14.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:14 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:14.232+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:14.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:14 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:14.232+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:14.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:14 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:14.232+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:14.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:14 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:14.232+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:14.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:14 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:14.232+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:43.981912+0000 front 2024-03-04T09:10:43.981924+0000 (oldest deadline 2024-03-04T09:11:06.881848+0000) 2024-03-04T09:11:15.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:14 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:14.581+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:11:15.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:14 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:14.581+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:11:15.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:14 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:14.581+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:11:15.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:14 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:14.581+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:11:15.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:14 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:14.581+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:11:15.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:14 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:14.581+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:35.597022+0000 front 2024-03-04T09:10:35.597102+0000 (oldest deadline 2024-03-04T09:10:59.697008+0000) 2024-03-04T09:11:15.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:14 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:14.581+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:45.598218+0000 front 2024-03-04T09:10:45.598269+0000 (oldest deadline 2024-03-04T09:11:10.898110+0000) 2024-03-04T09:11:15.552 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:15 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:15.191+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:15.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:15 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:15.191+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:15.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:15 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:15.191+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:15.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:15 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:15.191+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:15.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:15 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:15.191+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:15.555 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:15 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:15.191+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:43.981912+0000 front 2024-03-04T09:10:43.981924+0000 (oldest deadline 2024-03-04T09:11:06.881848+0000) 2024-03-04T09:11:16.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:15 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:15.565+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:11:16.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:15 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:15.565+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:11:16.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:15 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:15.565+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:11:16.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:15 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:15.565+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:11:16.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:15 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:15.565+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:11:16.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:15 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:15.565+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:35.597022+0000 front 2024-03-04T09:10:35.597102+0000 (oldest deadline 2024-03-04T09:10:59.697008+0000) 2024-03-04T09:11:16.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:15 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:15.565+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:45.598218+0000 front 2024-03-04T09:10:45.598269+0000 (oldest deadline 2024-03-04T09:11:10.898110+0000) 2024-03-04T09:11:16.552 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:16 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:16.203+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:16.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:16 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:16.203+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:16.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:16 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:16.203+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:16.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:16 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:16.203+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:16.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:16 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:16.203+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:16.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:16 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:16.203+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:43.981912+0000 front 2024-03-04T09:10:43.981924+0000 (oldest deadline 2024-03-04T09:11:06.881848+0000) 2024-03-04T09:11:16.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:16 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:16.203+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-03-04T09:10:53.183881+0000 front 2024-03-04T09:10:53.183920+0000 (oldest deadline 2024-03-04T09:11:15.483807+0000) 2024-03-04T09:11:17.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:16 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:16.534+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:11:17.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:16 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:16.534+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:11:17.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:16 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:16.534+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:11:17.030 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:16 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:16.534+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:11:17.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:16 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:16.534+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:11:17.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:16 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:16.534+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:35.597022+0000 front 2024-03-04T09:10:35.597102+0000 (oldest deadline 2024-03-04T09:10:59.697008+0000) 2024-03-04T09:11:17.031 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:16 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:16.534+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:45.598218+0000 front 2024-03-04T09:10:45.598269+0000 (oldest deadline 2024-03-04T09:11:10.898110+0000) 2024-03-04T09:11:17.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:17 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:17.191+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:17.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:17 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:17.191+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:17.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:17 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:17.191+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:17.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:17 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:17.191+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:17.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:17 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:17.191+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:17.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:17 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:17.191+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:43.981912+0000 front 2024-03-04T09:10:43.981924+0000 (oldest deadline 2024-03-04T09:11:06.881848+0000) 2024-03-04T09:11:17.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:17 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:17.191+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-03-04T09:10:53.183881+0000 front 2024-03-04T09:10:53.183920+0000 (oldest deadline 2024-03-04T09:11:15.483807+0000) 2024-03-04T09:11:17.780 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:17 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:17.488+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.487853+0000 front 2024-03-04T09:09:56.487872+0000 (oldest deadline 2024-03-04T09:10:19.987904+0000) 2024-03-04T09:11:17.780 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:17 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:17.488+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-03-04T09:10:04.589505+0000 front 2024-03-04T09:10:04.589550+0000 (oldest deadline 2024-03-04T09:10:29.289642+0000) 2024-03-04T09:11:17.780 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:17 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:17.488+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:10.390735+0000 front 2024-03-04T09:10:10.390696+0000 (oldest deadline 2024-03-04T09:10:35.690510+0000) 2024-03-04T09:11:17.781 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:17 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:17.488+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:20.693661+0000 front 2024-03-04T09:10:20.693825+0000 (oldest deadline 2024-03-04T09:10:44.193471+0000) 2024-03-04T09:11:17.781 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:17 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:17.488+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:27.094282+0000 front 2024-03-04T09:10:27.094321+0000 (oldest deadline 2024-03-04T09:10:49.994463+0000) 2024-03-04T09:11:17.781 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:17 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:17.488+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:35.597022+0000 front 2024-03-04T09:10:35.597102+0000 (oldest deadline 2024-03-04T09:10:59.697008+0000) 2024-03-04T09:11:17.781 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:17 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10[61426]: 2024-03-04T09:11:17.488+0000 7f2ecdfe5700 -1 osd.10 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:45.598218+0000 front 2024-03-04T09:10:45.598269+0000 (oldest deadline 2024-03-04T09:11:10.898110+0000) 2024-03-04T09:11:18.501 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:18 smithi150 podman[73050]: 2024-03-04 09:11:18.192663975 +0000 UTC m=+5.632242351 container died 29ea42bb55f35619a94d0d4125904a2353aeb6f8956368f35a5c638e390e9962 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-04T09:11:18.501 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:18 smithi150 podman[73050]: 2024-03-04 09:11:18.505335391 +0000 UTC m=+5.944913757 container cleanup 29ea42bb55f35619a94d0d4125904a2353aeb6f8956368f35a5c638e390e9962 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-04T09:11:18.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:18 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:18.211+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:18.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:18 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:18.211+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:18.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:18 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:18.211+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:18.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:18 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:18.211+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:18.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:18 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:18.211+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:18.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:18 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:18.211+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:43.981912+0000 front 2024-03-04T09:10:43.981924+0000 (oldest deadline 2024-03-04T09:11:06.881848+0000) 2024-03-04T09:11:18.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:18 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:18.212+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-03-04T09:10:53.183881+0000 front 2024-03-04T09:10:53.183920+0000 (oldest deadline 2024-03-04T09:11:15.483807+0000) 2024-03-04T09:11:18.780 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:18 smithi150 bash[73050]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10 2024-03-04T09:11:18.780 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:18 smithi150 podman[73067]: 2024-03-04 09:11:18.715081127 +0000 UTC m=+0.523175462 container remove 29ea42bb55f35619a94d0d4125904a2353aeb6f8956368f35a5c638e390e9962 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-04T09:11:19.280 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:19 smithi150 podman[73159]: 2024-03-04 09:11:19.007931594 +0000 UTC m=+0.112722370 container create 3e067728a86a9436e504f8b04dc3ad7c09429a9cdc5cea1c14d86acef14f4b31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, GIT_CLEAN=True) 2024-03-04T09:11:19.280 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:19 smithi150 podman[73159]: 2024-03-04 09:11:18.912222425 +0000 UTC m=+0.017013174 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:11:19.280 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:19 smithi150 podman[73159]: 2024-03-04 09:11:19.236239074 +0000 UTC m=+0.341045155 container init 3e067728a86a9436e504f8b04dc3ad7c09429a9cdc5cea1c14d86acef14f4b31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10-deactivate, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-1bc250c, ceph=True, GIT_CLEAN=True) 2024-03-04T09:11:19.281 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:19 smithi150 podman[73159]: 2024-03-04 09:11:19.239659864 +0000 UTC m=+0.344450599 container start 3e067728a86a9436e504f8b04dc3ad7c09429a9cdc5cea1c14d86acef14f4b31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:11:19.552 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:19 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:19.186+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:19.552 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:19 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:19.186+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:19.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:19 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:19.186+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:19.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:19 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:19.186+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:19.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:19 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:19.186+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:19.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:19 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:19.186+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:43.981912+0000 front 2024-03-04T09:10:43.981924+0000 (oldest deadline 2024-03-04T09:11:06.881848+0000) 2024-03-04T09:11:19.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:19 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:19.186+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-03-04T09:10:53.183881+0000 front 2024-03-04T09:10:53.183920+0000 (oldest deadline 2024-03-04T09:11:15.483807+0000) 2024-03-04T09:11:19.780 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:19 smithi150 podman[73159]: 2024-03-04 09:11:19.314400912 +0000 UTC m=+0.419191665 container attach 3e067728a86a9436e504f8b04dc3ad7c09429a9cdc5cea1c14d86acef14f4b31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-1bc250c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-04T09:11:19.780 INFO:journalctl@ceph.osd.10.smithi150.stdout:Mar 04 09:11:19 smithi150 podman[73159]: 2024-03-04 09:11:19.536257523 +0000 UTC m=+0.641048262 container died 3e067728a86a9436e504f8b04dc3ad7c09429a9cdc5cea1c14d86acef14f4b31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222) 2024-03-04T09:11:20.119 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.10.service' 2024-03-04T09:11:20.161 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:11:20.161 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-04T09:11:20.161 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-04T09:11:20.162 DEBUG:teuthology.orchestra.run.smithi158:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.11 2024-03-04T09:11:20.481 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:20 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:20.190+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:20.481 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:20 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:20.190+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:20.481 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:20 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:20.190+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:20.482 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:20 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:20.190+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:20.482 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:20 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:20.190+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:20.482 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:20 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:20.190+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:43.981912+0000 front 2024-03-04T09:10:43.981924+0000 (oldest deadline 2024-03-04T09:11:06.881848+0000) 2024-03-04T09:11:20.482 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:20 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:20.190+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-03-04T09:10:53.183881+0000 front 2024-03-04T09:10:53.183920+0000 (oldest deadline 2024-03-04T09:11:15.483807+0000) 2024-03-04T09:11:20.482 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:20 smithi158 systemd[1]: Stopping Ceph osd.11 for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:11:20.482 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:20 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:20.484+0000 7fd25c85a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:11:20.482 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:20 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:20.484+0000 7fd25c85a700 -1 osd.11 108 *** Got signal Terminated *** 2024-03-04T09:11:20.482 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:20 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:20.484+0000 7fd25c85a700 -1 osd.11 108 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T09:11:21.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:21 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:21.211+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:21.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:21 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:21.211+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:21.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:21 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:21.211+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:21.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:21 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:21.211+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:21.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:21 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:21.211+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:21.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:21 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:21.211+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:43.981912+0000 front 2024-03-04T09:10:43.981924+0000 (oldest deadline 2024-03-04T09:11:06.881848+0000) 2024-03-04T09:11:21.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:21 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:21.211+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-03-04T09:10:53.183881+0000 front 2024-03-04T09:10:53.183920+0000 (oldest deadline 2024-03-04T09:11:15.483807+0000) 2024-03-04T09:11:22.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:22 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:22.213+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:22.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:22 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:22.213+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:22.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:22 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:22.213+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:22.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:22 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:22.213+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:22.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:22 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:22.213+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:22.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:22 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:22.213+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:43.981912+0000 front 2024-03-04T09:10:43.981924+0000 (oldest deadline 2024-03-04T09:11:06.881848+0000) 2024-03-04T09:11:22.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:22 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:22.213+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-03-04T09:10:53.183881+0000 front 2024-03-04T09:10:53.183920+0000 (oldest deadline 2024-03-04T09:11:15.483807+0000) 2024-03-04T09:11:23.552 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:23 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:23.205+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:23.552 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:23 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:23.205+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:23.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:23 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:23.205+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:23.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:23 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:23.205+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:23.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:23 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:23.205+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:23.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:23 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:23.205+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:43.981912+0000 front 2024-03-04T09:10:43.981924+0000 (oldest deadline 2024-03-04T09:11:06.881848+0000) 2024-03-04T09:11:23.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:23 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:23.205+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-03-04T09:10:53.183881+0000 front 2024-03-04T09:10:53.183920+0000 (oldest deadline 2024-03-04T09:11:15.483807+0000) 2024-03-04T09:11:24.505 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:24 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:24.253+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:24.506 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:24 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:24.253+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:24.506 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:24 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:24.253+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:24.506 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:24 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:24.253+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:24.506 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:24 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:24.253+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:24.507 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:24 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:24.253+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:43.981912+0000 front 2024-03-04T09:10:43.981924+0000 (oldest deadline 2024-03-04T09:11:06.881848+0000) 2024-03-04T09:11:24.507 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:24 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:24.253+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-03-04T09:10:53.183881+0000 front 2024-03-04T09:10:53.183920+0000 (oldest deadline 2024-03-04T09:11:15.483807+0000) 2024-03-04T09:11:25.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:25 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:25.212+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-03-04T09:09:56.973455+0000 front 2024-03-04T09:09:56.973300+0000 (oldest deadline 2024-03-04T09:10:22.273210+0000) 2024-03-04T09:11:25.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:25 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:25.212+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6804 osd.2 since back 2024-03-04T09:10:13.176141+0000 front 2024-03-04T09:10:13.176197+0000 (oldest deadline 2024-03-04T09:10:38.977050+0000) 2024-03-04T09:11:25.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:25 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:25.212+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-03-04T09:10:18.977614+0000 front 2024-03-04T09:10:18.977657+0000 (oldest deadline 2024-03-04T09:10:44.877631+0000) 2024-03-04T09:11:25.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:25 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:25.212+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-03-04T09:10:24.878102+0000 front 2024-03-04T09:10:24.878211+0000 (oldest deadline 2024-03-04T09:10:50.178079+0000) 2024-03-04T09:11:25.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:25 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:25.212+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.158:6812 osd.5 since back 2024-03-04T09:10:33.679090+0000 front 2024-03-04T09:10:33.679176+0000 (oldest deadline 2024-03-04T09:10:58.379225+0000) 2024-03-04T09:11:25.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:25 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:25.212+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-03-04T09:10:43.981912+0000 front 2024-03-04T09:10:43.981924+0000 (oldest deadline 2024-03-04T09:11:06.881848+0000) 2024-03-04T09:11:25.554 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:25 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11[63061]: 2024-03-04T09:11:25.212+0000 7fd258e4e700 -1 osd.11 108 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-03-04T09:10:53.183881+0000 front 2024-03-04T09:10:53.183920+0000 (oldest deadline 2024-03-04T09:11:15.483807+0000) 2024-03-04T09:11:26.090 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:25 smithi158 podman[75363]: 2024-03-04 09:11:25.84050281 +0000 UTC m=+5.486650341 container died 7d43fbedd99cc36d23fda2607108c31149c423d413e68fc06051cfcdd48436db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T09:11:26.456 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:26 smithi158 podman[75363]: 2024-03-04 09:11:26.184659306 +0000 UTC m=+5.830806819 container cleanup 7d43fbedd99cc36d23fda2607108c31149c423d413e68fc06051cfcdd48436db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11, ceph=True, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-04T09:11:26.456 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:26 smithi158 bash[75363]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11 2024-03-04T09:11:26.790 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:26 smithi158 podman[75381]: 2024-03-04 09:11:26.460847654 +0000 UTC m=+0.623176762 container remove 7d43fbedd99cc36d23fda2607108c31149c423d413e68fc06051cfcdd48436db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-04T09:11:27.053 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:26 smithi158 podman[75471]: 2024-03-04 09:11:26.794928295 +0000 UTC m=+0.114410963 container create 931274f3dd3c9e652940e360e4a64948f52d41e3bf6ca8aac1917415676ccf73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:11:27.053 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:26 smithi158 podman[75471]: 2024-03-04 09:11:26.698923962 +0000 UTC m=+0.018406625 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:11:27.053 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:26 smithi158 podman[75471]: 2024-03-04 09:11:26.997485417 +0000 UTC m=+0.316968085 container init 931274f3dd3c9e652940e360e4a64948f52d41e3bf6ca8aac1917415676ccf73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:11:27.053 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:27 smithi158 podman[75471]: 2024-03-04 09:11:27.000611276 +0000 UTC m=+0.320093933 container start 931274f3dd3c9e652940e360e4a64948f52d41e3bf6ca8aac1917415676ccf73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-1bc250c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-04T09:11:27.552 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:27 smithi158 podman[75471]: 2024-03-04 09:11:27.080618281 +0000 UTC m=+0.400100934 container attach 931274f3dd3c9e652940e360e4a64948f52d41e3bf6ca8aac1917415676ccf73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0) 2024-03-04T09:11:27.553 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:27 smithi158 podman[75471]: 2024-03-04 09:11:27.289129747 +0000 UTC m=+0.608612398 container died 931274f3dd3c9e652940e360e4a64948f52d41e3bf6ca8aac1917415676ccf73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:11:27.872 DEBUG:teuthology.orchestra.run.smithi158:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.11.service' 2024-03-04T09:11:27.898 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:27 smithi158 podman[75497]: 2024-03-04 09:11:27.858689415 +0000 UTC m=+0.564234967 container remove 931274f3dd3c9e652940e360e4a64948f52d41e3bf6ca8aac1917415676ccf73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-osd-11-deactivate, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-04T09:11:27.898 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:27 smithi158 systemd[1]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.11.service: Deactivated successfully. 2024-03-04T09:11:27.898 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:27 smithi158 systemd[1]: Stopped Ceph osd.11 for b3c85c40-da03-11ee-95c6-87774f69a715. 2024-03-04T09:11:27.899 INFO:journalctl@ceph.osd.11.smithi158.stdout:Mar 04 09:11:27 smithi158 systemd[1]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715@osd.11.service: Consumed 56.121s CPU time. 2024-03-04T09:11:27.914 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:11:27.914 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-04T09:11:27.914 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-04T09:11:27.914 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.a 2024-03-04T09:11:28.314 INFO:journalctl@ceph.mds.a.smithi071.stdout:Mar 04 09:11:27 smithi071 systemd[1]: Stopping Ceph mds.a for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:11:28.671 INFO:journalctl@ceph.mds.a.smithi071.stdout:Mar 04 09:11:28 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-a[74637]: 2024-03-04T09:11:28.317+0000 7fa0a1883700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:11:28.672 INFO:journalctl@ceph.mds.a.smithi071.stdout:Mar 04 09:11:28 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-a[74637]: 2024-03-04T09:11:28.317+0000 7fa0a1883700 -1 mds.a *** got signal Terminated *** 2024-03-04T09:11:31.422 INFO:journalctl@ceph.mds.a.smithi071.stdout:Mar 04 09:11:31 smithi071 podman[93383]: 2024-03-04 09:11:31.105015497 +0000 UTC m=+2.992626870 container died 086e350c76f92aa721e2dfc16e56bf41107d7513bcd76f1080b684b246a4b095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-a, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:11:31.724 INFO:journalctl@ceph.mds.a.smithi071.stdout:Mar 04 09:11:31 smithi071 podman[93383]: 2024-03-04 09:11:31.502888804 +0000 UTC m=+3.390500230 container cleanup 086e350c76f92aa721e2dfc16e56bf41107d7513bcd76f1080b684b246a4b095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-a, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-04T09:11:31.725 INFO:journalctl@ceph.mds.a.smithi071.stdout:Mar 04 09:11:31 smithi071 bash[93383]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-a 2024-03-04T09:11:31.848 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.a.service' 2024-03-04T09:11:31.884 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:11:31.884 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-04T09:11:31.884 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-04T09:11:31.884 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.d 2024-03-04T09:11:32.171 INFO:journalctl@ceph.mds.d.smithi071.stdout:Mar 04 09:11:31 smithi071 systemd[1]: Stopping Ceph mds.d for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:11:32.671 INFO:journalctl@ceph.mds.d.smithi071.stdout:Mar 04 09:11:32 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-d[73642]: 2024-03-04T09:11:32.223+0000 7f0ff7e9d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:11:32.672 INFO:journalctl@ceph.mds.d.smithi071.stdout:Mar 04 09:11:32 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-d[73642]: 2024-03-04T09:11:32.223+0000 7f0ff7e9d700 -1 mds.d *** got signal Terminated *** 2024-03-04T09:11:35.422 INFO:journalctl@ceph.mds.d.smithi071.stdout:Mar 04 09:11:35 smithi071 podman[93504]: 2024-03-04 09:11:35.073119365 +0000 UTC m=+2.967982409 container died 254998c03e910681b4e0a745eec1b547c2b764d4021a2a3ebb3d8ef6677dc048 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-d, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-1bc250c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-04T09:11:35.797 INFO:journalctl@ceph.mds.d.smithi071.stdout:Mar 04 09:11:35 smithi071 podman[93504]: 2024-03-04 09:11:35.548514314 +0000 UTC m=+3.443377426 container cleanup 254998c03e910681b4e0a745eec1b547c2b764d4021a2a3ebb3d8ef6677dc048 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:11:35.797 INFO:journalctl@ceph.mds.d.smithi071.stdout:Mar 04 09:11:35 smithi071 bash[93504]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-d 2024-03-04T09:11:35.932 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.d.service' 2024-03-04T09:11:35.966 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:11:35.967 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-04T09:11:35.967 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-04T09:11:35.967 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.g 2024-03-04T09:11:36.422 INFO:journalctl@ceph.mds.g.smithi071.stdout:Mar 04 09:11:36 smithi071 systemd[1]: Stopping Ceph mds.g for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:11:36.422 INFO:journalctl@ceph.mds.g.smithi071.stdout:Mar 04 09:11:36 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-g[74274]: 2024-03-04T09:11:36.290+0000 7fefde673700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:11:36.422 INFO:journalctl@ceph.mds.g.smithi071.stdout:Mar 04 09:11:36 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-g[74274]: 2024-03-04T09:11:36.290+0000 7fefde673700 -1 mds.g *** got signal Terminated *** 2024-03-04T09:11:40.671 INFO:journalctl@ceph.mds.g.smithi071.stdout:Mar 04 09:11:40 smithi071 podman[93624]: 2024-03-04 09:11:40.296288696 +0000 UTC m=+4.111618269 container died 0cac76a6e46a9316c6f91dbc1e249ce6b44ad0cb6125153d662a3f8b2ed13eae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:11:41.171 INFO:journalctl@ceph.mds.g.smithi071.stdout:Mar 04 09:11:40 smithi071 podman[93624]: 2024-03-04 09:11:40.968284332 +0000 UTC m=+4.783613911 container cleanup 0cac76a6e46a9316c6f91dbc1e249ce6b44ad0cb6125153d662a3f8b2ed13eae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-g, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:11:41.172 INFO:journalctl@ceph.mds.g.smithi071.stdout:Mar 04 09:11:40 smithi071 bash[93624]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-g 2024-03-04T09:11:41.334 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.g.service' 2024-03-04T09:11:41.369 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:11:41.369 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-04T09:11:41.369 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-04T09:11:41.369 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.j 2024-03-04T09:11:41.922 INFO:journalctl@ceph.mds.j.smithi071.stdout:Mar 04 09:11:41 smithi071 systemd[1]: Stopping Ceph mds.j for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:11:41.922 INFO:journalctl@ceph.mds.j.smithi071.stdout:Mar 04 09:11:41 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-j[73271]: 2024-03-04T09:11:41.696+0000 7f941bf0d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:11:41.922 INFO:journalctl@ceph.mds.j.smithi071.stdout:Mar 04 09:11:41 smithi071 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-j[73271]: 2024-03-04T09:11:41.696+0000 7f941bf0d700 -1 mds.j *** got signal Terminated *** 2024-03-04T09:11:44.627 INFO:journalctl@ceph.mds.j.smithi071.stdout:Mar 04 09:11:44 smithi071 podman[93745]: 2024-03-04 09:11:44.268488468 +0000 UTC m=+2.677253272 container died 32208bdaccc6609be16e44faacb1f09c8d3aa6fc763a40ca669ef9443d99e272 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-j, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-04T09:11:44.922 INFO:journalctl@ceph.mds.j.smithi071.stdout:Mar 04 09:11:44 smithi071 podman[93745]: 2024-03-04 09:11:44.705722369 +0000 UTC m=+3.114487175 container cleanup 32208bdaccc6609be16e44faacb1f09c8d3aa6fc763a40ca669ef9443d99e272 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-j, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-04T09:11:44.922 INFO:journalctl@ceph.mds.j.smithi071.stdout:Mar 04 09:11:44 smithi071 bash[93745]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-j 2024-03-04T09:11:44.922 INFO:journalctl@ceph.mds.j.smithi071.stdout:Mar 04 09:11:44 smithi071 podman[93758]: 2024-03-04 09:11:44.848465969 +0000 UTC m=+0.582959494 container remove 32208bdaccc6609be16e44faacb1f09c8d3aa6fc763a40ca669ef9443d99e272 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-j, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5) 2024-03-04T09:11:45.066 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.j.service' 2024-03-04T09:11:45.099 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:11:45.099 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-04T09:11:45.099 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-04T09:11:45.099 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.b 2024-03-04T09:11:45.413 INFO:journalctl@ceph.mds.b.smithi150.stdout:Mar 04 09:11:45 smithi150 systemd[1]: Stopping Ceph mds.b for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:11:45.679 INFO:journalctl@ceph.mds.b.smithi150.stdout:Mar 04 09:11:45 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-b[66660]: 2024-03-04T09:11:45.416+0000 7fa8c4277700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:11:45.679 INFO:journalctl@ceph.mds.b.smithi150.stdout:Mar 04 09:11:45 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-b[66660]: 2024-03-04T09:11:45.416+0000 7fa8c4277700 -1 mds.b *** got signal Terminated *** 2024-03-04T09:11:48.147 INFO:journalctl@ceph.mds.b.smithi150.stdout:Mar 04 09:11:47 smithi150 podman[73265]: 2024-03-04 09:11:47.866356675 +0000 UTC m=+2.571477327 container died 20c8cbae09406d97eda8d39ca9dd38a00549afa34d617cec85ccab308b4ae88c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:11:48.455 INFO:journalctl@ceph.mds.b.smithi150.stdout:Mar 04 09:11:48 smithi150 podman[73265]: 2024-03-04 09:11:48.225177077 +0000 UTC m=+2.930297730 container cleanup 20c8cbae09406d97eda8d39ca9dd38a00549afa34d617cec85ccab308b4ae88c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-b, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-04T09:11:48.455 INFO:journalctl@ceph.mds.b.smithi150.stdout:Mar 04 09:11:48 smithi150 bash[73265]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-b 2024-03-04T09:11:48.586 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.b.service' 2024-03-04T09:11:48.626 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:11:48.626 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-04T09:11:48.627 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-04T09:11:48.627 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.e 2024-03-04T09:11:49.030 INFO:journalctl@ceph.mds.e.smithi150.stdout:Mar 04 09:11:48 smithi150 systemd[1]: Stopping Ceph mds.e for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:11:49.030 INFO:journalctl@ceph.mds.e.smithi150.stdout:Mar 04 09:11:48 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-e[66041]: 2024-03-04T09:11:48.961+0000 7f14db0fb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:11:49.031 INFO:journalctl@ceph.mds.e.smithi150.stdout:Mar 04 09:11:48 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-e[66041]: 2024-03-04T09:11:48.961+0000 7f14db0fb700 -1 mds.e *** got signal Terminated *** 2024-03-04T09:11:53.280 INFO:journalctl@ceph.mds.e.smithi150.stdout:Mar 04 09:11:52 smithi150 podman[73398]: 2024-03-04 09:11:52.970288912 +0000 UTC m=+4.113800320 container died 5826b805502d889431a87d966f17a3e074f9857d786370f682ded0669e3cea59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:11:53.692 INFO:journalctl@ceph.mds.e.smithi150.stdout:Mar 04 09:11:53 smithi150 podman[73398]: 2024-03-04 09:11:53.470613498 +0000 UTC m=+4.614124900 container cleanup 5826b805502d889431a87d966f17a3e074f9857d786370f682ded0669e3cea59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:11:53.692 INFO:journalctl@ceph.mds.e.smithi150.stdout:Mar 04 09:11:53 smithi150 bash[73398]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-e 2024-03-04T09:11:53.819 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.e.service' 2024-03-04T09:11:53.854 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:11:53.855 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-04T09:11:53.855 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-04T09:11:53.855 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.h 2024-03-04T09:11:54.030 INFO:journalctl@ceph.mds.h.smithi150.stdout:Mar 04 09:11:53 smithi150 systemd[1]: Stopping Ceph mds.h for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:11:54.530 INFO:journalctl@ceph.mds.h.smithi150.stdout:Mar 04 09:11:54 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-h[66347]: 2024-03-04T09:11:54.174+0000 7fe7f6625700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:11:54.530 INFO:journalctl@ceph.mds.h.smithi150.stdout:Mar 04 09:11:54 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-h[66347]: 2024-03-04T09:11:54.174+0000 7fe7f6625700 -1 mds.h *** got signal Terminated *** 2024-03-04T09:11:58.521 INFO:journalctl@ceph.mds.h.smithi150.stdout:Mar 04 09:11:58 smithi150 podman[73517]: 2024-03-04 09:11:58.189393785 +0000 UTC m=+4.114828672 container died 284af29d0072c7fc1735166e1080b86dd43a35fe2ebf17038693741749914b57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-h, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T09:11:58.780 INFO:journalctl@ceph.mds.h.smithi150.stdout:Mar 04 09:11:58 smithi150 podman[73517]: 2024-03-04 09:11:58.625800791 +0000 UTC m=+4.551235663 container cleanup 284af29d0072c7fc1735166e1080b86dd43a35fe2ebf17038693741749914b57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-04T09:11:58.780 INFO:journalctl@ceph.mds.h.smithi150.stdout:Mar 04 09:11:58 smithi150 bash[73517]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-h 2024-03-04T09:11:59.038 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.h.service' 2024-03-04T09:11:59.062 INFO:journalctl@ceph.mds.h.smithi150.stdout:Mar 04 09:11:58 smithi150 podman[73532]: 2024-03-04 09:11:58.810609352 +0000 UTC m=+0.623799103 container remove 284af29d0072c7fc1735166e1080b86dd43a35fe2ebf17038693741749914b57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-h, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_CLEAN=True) 2024-03-04T09:11:59.062 INFO:journalctl@ceph.mds.h.smithi150.stdout:Mar 04 09:11:59 smithi150 systemd[1]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.h.service: Deactivated successfully. 2024-03-04T09:11:59.062 INFO:journalctl@ceph.mds.h.smithi150.stdout:Mar 04 09:11:59 smithi150 systemd[1]: Stopped Ceph mds.h for b3c85c40-da03-11ee-95c6-87774f69a715. 2024-03-04T09:11:59.073 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:11:59.073 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-04T09:11:59.073 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-04T09:11:59.074 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.k 2024-03-04T09:11:59.383 INFO:journalctl@ceph.mds.k.smithi150.stdout:Mar 04 09:11:59 smithi150 systemd[1]: Stopping Ceph mds.k for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:11:59.780 INFO:journalctl@ceph.mds.k.smithi150.stdout:Mar 04 09:11:59 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-k[65724]: 2024-03-04T09:11:59.387+0000 7f59add63700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:11:59.780 INFO:journalctl@ceph.mds.k.smithi150.stdout:Mar 04 09:11:59 smithi150 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-k[65724]: 2024-03-04T09:11:59.387+0000 7f59add63700 -1 mds.k *** got signal Terminated *** 2024-03-04T09:12:01.530 INFO:journalctl@ceph.mds.k.smithi150.stdout:Mar 04 09:12:01 smithi150 podman[73639]: 2024-03-04 09:12:01.125335856 +0000 UTC m=+1.832500219 container died 12678faf940d5a21c6a64db34d2d4c67474b936ee78ffc5acdc721b8395c5d4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-k, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:12:01.942 INFO:journalctl@ceph.mds.k.smithi150.stdout:Mar 04 09:12:01 smithi150 podman[73639]: 2024-03-04 09:12:01.696258193 +0000 UTC m=+2.403422559 container cleanup 12678faf940d5a21c6a64db34d2d4c67474b936ee78ffc5acdc721b8395c5d4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-k, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-1bc250c, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:12:01.943 INFO:journalctl@ceph.mds.k.smithi150.stdout:Mar 04 09:12:01 smithi150 bash[73639]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-k 2024-03-04T09:12:02.060 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.k.service' 2024-03-04T09:12:02.094 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:12:02.095 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-04T09:12:02.095 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-04T09:12:02.095 DEBUG:teuthology.orchestra.run.smithi158:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.c 2024-03-04T09:12:02.461 INFO:journalctl@ceph.mds.c.smithi158.stdout:Mar 04 09:12:02 smithi158 systemd[1]: Stopping Ceph mds.c for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:12:02.802 INFO:journalctl@ceph.mds.c.smithi158.stdout:Mar 04 09:12:02 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-c[68353]: 2024-03-04T09:12:02.464+0000 7f5a2e8da700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:12:02.803 INFO:journalctl@ceph.mds.c.smithi158.stdout:Mar 04 09:12:02 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-c[68353]: 2024-03-04T09:12:02.464+0000 7f5a2e8da700 -1 mds.c *** got signal Terminated *** 2024-03-04T09:12:03.736 INFO:journalctl@ceph.mds.c.smithi158.stdout:Mar 04 09:12:03 smithi158 podman[75576]: 2024-03-04 09:12:03.65219644 +0000 UTC m=+1.365015443 container died 78725097cb8497e692b468275028a4e7f6461cd8a60318ed8b3a1858f3a8541e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-c, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-1bc250c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-04T09:12:04.303 INFO:journalctl@ceph.mds.c.smithi158.stdout:Mar 04 09:12:04 smithi158 podman[75576]: 2024-03-04 09:12:04.02420104 +0000 UTC m=+1.737020030 container cleanup 78725097cb8497e692b468275028a4e7f6461cd8a60318ed8b3a1858f3a8541e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-c, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-04T09:12:04.303 INFO:journalctl@ceph.mds.c.smithi158.stdout:Mar 04 09:12:04 smithi158 bash[75576]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-c 2024-03-04T09:12:04.434 DEBUG:teuthology.orchestra.run.smithi158:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.c.service' 2024-03-04T09:12:04.472 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:12:04.472 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-04T09:12:04.472 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-04T09:12:04.473 DEBUG:teuthology.orchestra.run.smithi158:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.f 2024-03-04T09:12:04.672 INFO:journalctl@ceph.mds.f.smithi158.stdout:Mar 04 09:12:04 smithi158 systemd[1]: Stopping Ceph mds.f for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:12:05.053 INFO:journalctl@ceph.mds.f.smithi158.stdout:Mar 04 09:12:04 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-f[68046]: 2024-03-04T09:12:04.801+0000 7fcbf1083700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:12:05.053 INFO:journalctl@ceph.mds.f.smithi158.stdout:Mar 04 09:12:04 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-f[68046]: 2024-03-04T09:12:04.801+0000 7fcbf1083700 -1 mds.f *** got signal Terminated *** 2024-03-04T09:12:09.098 INFO:journalctl@ceph.mds.f.smithi158.stdout:Mar 04 09:12:08 smithi158 podman[75708]: 2024-03-04 09:12:08.810275377 +0000 UTC m=+4.113514849 container died 7faadf000d2256ea0a86a355c5c7a3d956f5d60d47db2f7de3ea72fb3365a3c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-f, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-04T09:12:09.407 INFO:journalctl@ceph.mds.f.smithi158.stdout:Mar 04 09:12:09 smithi158 podman[75708]: 2024-03-04 09:12:09.193713176 +0000 UTC m=+4.496952629 container cleanup 7faadf000d2256ea0a86a355c5c7a3d956f5d60d47db2f7de3ea72fb3365a3c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-f, RELEASE=wip-lusov-qdb-1bc250c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-04T09:12:09.407 INFO:journalctl@ceph.mds.f.smithi158.stdout:Mar 04 09:12:09 smithi158 bash[75708]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-f 2024-03-04T09:12:09.540 DEBUG:teuthology.orchestra.run.smithi158:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.f.service' 2024-03-04T09:12:09.577 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:12:09.578 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-04T09:12:09.578 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-04T09:12:09.578 DEBUG:teuthology.orchestra.run.smithi158:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.i 2024-03-04T09:12:09.802 INFO:journalctl@ceph.mds.i.smithi158.stdout:Mar 04 09:12:09 smithi158 systemd[1]: Stopping Ceph mds.i for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:12:10.302 INFO:journalctl@ceph.mds.i.smithi158.stdout:Mar 04 09:12:09 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-i[68666]: 2024-03-04T09:12:09.905+0000 7f8f88047700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:12:10.303 INFO:journalctl@ceph.mds.i.smithi158.stdout:Mar 04 09:12:09 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-i[68666]: 2024-03-04T09:12:09.905+0000 7f8f88047700 -1 mds.i *** got signal Terminated *** 2024-03-04T09:12:12.144 INFO:journalctl@ceph.mds.i.smithi158.stdout:Mar 04 09:12:11 smithi158 podman[75829]: 2024-03-04 09:12:11.872158423 +0000 UTC m=+2.077537120 container died c72f4745cb05f296f87564a9ecd7547f632457cf485a72e466aa2cc1068b43a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-04T09:12:12.519 INFO:journalctl@ceph.mds.i.smithi158.stdout:Mar 04 09:12:12 smithi158 podman[75829]: 2024-03-04 09:12:12.222354815 +0000 UTC m=+2.427733484 container cleanup c72f4745cb05f296f87564a9ecd7547f632457cf485a72e466aa2cc1068b43a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-i, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-04T09:12:12.519 INFO:journalctl@ceph.mds.i.smithi158.stdout:Mar 04 09:12:12 smithi158 bash[75829]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-i 2024-03-04T09:12:12.646 DEBUG:teuthology.orchestra.run.smithi158:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.i.service' 2024-03-04T09:12:12.681 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:12:12.681 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-04T09:12:12.681 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-04T09:12:12.682 DEBUG:teuthology.orchestra.run.smithi158:> sudo systemctl stop ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.l 2024-03-04T09:12:12.802 INFO:journalctl@ceph.mds.l.smithi158.stdout:Mar 04 09:12:12 smithi158 systemd[1]: Stopping Ceph mds.l for b3c85c40-da03-11ee-95c6-87774f69a715... 2024-03-04T09:12:13.302 INFO:journalctl@ceph.mds.l.smithi158.stdout:Mar 04 09:12:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-l[67727]: 2024-03-04T09:12:13.004+0000 7f47c4450700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:12:13.302 INFO:journalctl@ceph.mds.l.smithi158.stdout:Mar 04 09:12:13 smithi158 ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-l[67727]: 2024-03-04T09:12:13.004+0000 7f47c4450700 -1 mds.l *** got signal Terminated *** 2024-03-04T09:12:14.547 INFO:journalctl@ceph.mds.l.smithi158.stdout:Mar 04 09:12:14 smithi158 podman[75950]: 2024-03-04 09:12:14.253546609 +0000 UTC m=+1.355126558 container died d8193eec104a98d92ec701293a821d2cdc8b3d4c8513bed6bae1d1e870af7632 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-l, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:12:14.802 INFO:journalctl@ceph.mds.l.smithi158.stdout:Mar 04 09:12:14 smithi158 podman[75950]: 2024-03-04 09:12:14.65113726 +0000 UTC m=+1.752717213 container cleanup d8193eec104a98d92ec701293a821d2cdc8b3d4c8513bed6bae1d1e870af7632 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-l, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:12:14.803 INFO:journalctl@ceph.mds.l.smithi158.stdout:Mar 04 09:12:14 smithi158 bash[75950]: ceph-b3c85c40-da03-11ee-95c6-87774f69a715-mds-l 2024-03-04T09:12:15.029 DEBUG:teuthology.orchestra.run.smithi158:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b3c85c40-da03-11ee-95c6-87774f69a715@mds.l.service' 2024-03-04T09:12:15.064 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:12:15.067 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-04T09:12:15.072 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b3c85c40-da03-11ee-95c6-87774f69a715 --force --keep-logs 2024-03-04T09:12:15.267 INFO:teuthology.orchestra.run.smithi071.stdout:Deleting cluster with fsid: b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T09:12:17.731 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b3c85c40-da03-11ee-95c6-87774f69a715 --force --keep-logs 2024-03-04T09:12:17.925 INFO:teuthology.orchestra.run.smithi150.stdout:Deleting cluster with fsid: b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T09:12:20.438 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b3c85c40-da03-11ee-95c6-87774f69a715 --force --keep-logs 2024-03-04T09:12:20.629 INFO:teuthology.orchestra.run.smithi158.stdout:Deleting cluster with fsid: b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T09:12:23.072 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-04T09:12:23.107 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-04T09:12:23.142 DEBUG:teuthology.orchestra.run.smithi158:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-04T09:12:23.173 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-04T09:12:23.175 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/crash to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580801/remote/smithi071/crash 2024-03-04T09:12:23.176 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/crash -- . 2024-03-04T09:12:23.221 INFO:teuthology.orchestra.run.smithi071.stderr:tar: /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/crash: Cannot open: No such file or directory 2024-03-04T09:12:23.221 INFO:teuthology.orchestra.run.smithi071.stderr:tar: Error is not recoverable: exiting now 2024-03-04T09:12:23.223 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/crash to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580801/remote/smithi150/crash 2024-03-04T09:12:23.224 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/crash -- . 2024-03-04T09:12:23.266 INFO:teuthology.orchestra.run.smithi150.stderr:tar: /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/crash: Cannot open: No such file or directory 2024-03-04T09:12:23.266 INFO:teuthology.orchestra.run.smithi150.stderr:tar: Error is not recoverable: exiting now 2024-03-04T09:12:23.268 DEBUG:teuthology.misc:Transferring archived files from smithi158:/var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/crash to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580801/remote/smithi158/crash 2024-03-04T09:12:23.269 DEBUG:teuthology.orchestra.run.smithi158:> sudo tar c -f - -C /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/crash -- . 2024-03-04T09:12:23.309 INFO:teuthology.orchestra.run.smithi158.stderr:tar: /var/lib/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/crash: Cannot open: No such file or directory 2024-03-04T09:12:23.309 INFO:teuthology.orchestra.run.smithi158.stderr:tar: Error is not recoverable: exiting now 2024-03-04T09:12:23.310 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-04T09:12:23.311 DEBUG:teuthology.orchestra.run.smithi071:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-04T09:12:23.344 INFO:teuthology.orchestra.run.smithi071.stdout:2024-03-04T08:49:29.864232+0000 mon.a (mon.0) 191 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-04T09:12:23.344 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-04T09:12:23.345 INFO:tasks.cephadm:Compressing logs... 2024-03-04T09:12:23.345 DEBUG:teuthology.orchestra.run.smithi071:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-04T09:12:23.386 DEBUG:teuthology.orchestra.run.smithi150:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-04T09:12:23.390 DEBUG:teuthology.orchestra.run.smithi158:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-04T09:12:23.413 INFO:teuthology.orchestra.run.smithi071.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-04T09:12:23.413 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-04T09:12:23.414 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.6.log 2024-03-04T09:12:23.414 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mon.a.log 2024-03-04T09:12:23.414 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.audit.log 2024-03-04T09:12:23.414 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.6.log: /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.g.log 2024-03-04T09:12:23.414 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.j.log 2024-03-04T09:12:23.416 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.audit.log: /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.3.log 2024-03-04T09:12:23.417 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mgr.x.log 2024-03-04T09:12:23.417 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.0.log 2024-03-04T09:12:23.417 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mgr.x.log: 90.1% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.audit.log.gz 2024-03-04T09:12:23.419 INFO:teuthology.orchestra.run.smithi158.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-04T09:12:23.419 INFO:teuthology.orchestra.run.smithi158.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-04T09:12:23.419 INFO:teuthology.orchestra.run.smithi158.stderr:gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.l.log 2024-03-04T09:12:23.419 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.audit.log 2024-03-04T09:12:23.419 INFO:teuthology.orchestra.run.smithi158.stderr:gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.i.log 2024-03-04T09:12:23.442 INFO:teuthology.orchestra.run.smithi071.stderr: 94.6% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.g.log.gz 2024-03-04T09:12:23.442 INFO:teuthology.orchestra.run.smithi071.stderr: 94.7% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.j.log.gz 2024-03-04T09:12:23.442 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.cephadm.log 2024-03-04T09:12:23.443 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.0.log: gzip -5 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-04T09:12:23.443 INFO:teuthology.orchestra.run.smithi071.stderr: --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.log 2024-03-04T09:12:23.443 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-volume.log 2024-03-04T09:12:23.443 INFO:teuthology.orchestra.run.smithi071.stderr: 83.7%/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.log: -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.cephadm.log.gz 2024-03-04T09:12:23.443 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.9.log 2024-03-04T09:12:23.443 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.d.log 2024-03-04T09:12:23.443 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.9.log: /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-volume.log: 86.6% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.log.gz 2024-03-04T09:12:23.443 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.a.log 2024-03-04T09:12:23.443 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.d.log.gz 2024-03-04T09:12:23.444 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-03-04T09:12:23.444 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.a.log: 94.6% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.a.log.gz 2024-03-04T09:12:23.444 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-03-04T09:12:23.444 INFO:teuthology.orchestra.run.smithi071.stderr: 91.0% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-volume.log.gz 2024-03-04T09:12:23.448 INFO:teuthology.orchestra.run.smithi150.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose: No such file or directory -- 2024-03-04T09:12:23.448 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/cephadm.log 2024-03-04T09:12:23.448 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.10.log 2024-03-04T09:12:23.448 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.audit.log 2024-03-04T09:12:23.448 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.b.log 2024-03-04T09:12:23.448 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.4.log 2024-03-04T09:12:23.448 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.audit.log: gzip -5/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.b.log: --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.1.log 2024-03-04T09:12:23.449 INFO:teuthology.orchestra.run.smithi150.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-04T09:12:23.449 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.k.log 2024-03-04T09:12:23.449 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.h.log 2024-03-04T09:12:23.449 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.7.log 2024-03-04T09:12:23.449 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.e.log 2024-03-04T09:12:23.449 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.7.log: 90.2% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.audit.log.gz 2024-03-04T09:12:23.449 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mgr.y.log 2024-03-04T09:12:23.449 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.e.log: 94.6% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.k.log.gz 2024-03-04T09:12:23.449 INFO:teuthology.orchestra.run.smithi150.stderr: 94.5% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.h.log.gz 2024-03-04T09:12:23.450 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.cephadm.log 2024-03-04T09:12:23.450 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.log 2024-03-04T09:12:23.450 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.cephadm.log: 83.4% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.cephadm.log.gz 2024-03-04T09:12:23.450 INFO:teuthology.orchestra.run.smithi150.stderr: 94.6% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.e.log.gz 2024-03-04T09:12:23.450 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-volume.log 2024-03-04T09:12:23.450 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mon.b.log 2024-03-04T09:12:23.450 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-volume.log: 86.6% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.log.gz 2024-03-04T09:12:23.451 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mon.b.log: 91.9% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mgr.y.log.gz 2024-03-04T09:12:23.451 INFO:teuthology.orchestra.run.smithi150.stderr: 91.0% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-volume.log.gz 2024-03-04T09:12:23.451 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.l.log: /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.11.log 2024-03-04T09:12:23.451 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mgr.z.log 2024-03-04T09:12:23.452 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.8.log 2024-03-04T09:12:23.452 INFO:teuthology.orchestra.run.smithi158.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-04T09:12:23.452 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.5.log 2024-03-04T09:12:23.452 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.f.log 2024-03-04T09:12:23.452 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.cephadm.log 2024-03-04T09:12:23.452 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.f.log: 90.3% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.audit.log.gz 2024-03-04T09:12:23.452 INFO:teuthology.orchestra.run.smithi158.stderr:gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.log 2024-03-04T09:12:23.452 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.cephadm.log: 83.4% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.cephadm.log.gz 2024-03-04T09:12:23.453 INFO:teuthology.orchestra.run.smithi158.stderr:gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.2.log 2024-03-04T09:12:23.453 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.log: 94.6% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.l.log.gz 2024-03-04T09:12:23.453 INFO:teuthology.orchestra.run.smithi158.stderr:gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.c.log 2024-03-04T09:12:23.453 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.2.log: 86.6% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph.log.gz 2024-03-04T09:12:23.453 INFO:teuthology.orchestra.run.smithi158.stderr:gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-volume.log 2024-03-04T09:12:23.453 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.c.log: 94.5% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.f.log.gz 2024-03-04T09:12:23.453 INFO:teuthology.orchestra.run.smithi158.stderr:gzip -5 --verbose -- /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mon.c.log 2024-03-04T09:12:23.454 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-volume.log: 91.9% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mgr.z.log.gz 2024-03-04T09:12:23.454 INFO:teuthology.orchestra.run.smithi158.stderr:/var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mon.c.log: 94.5% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.c.log.gz 2024-03-04T09:12:23.454 INFO:teuthology.orchestra.run.smithi158.stderr: 91.0% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-volume.log.gz 2024-03-04T09:12:23.738 INFO:teuthology.orchestra.run.smithi071.stderr: 90.1% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mgr.x.log.gz 2024-03-04T09:12:23.859 INFO:teuthology.orchestra.run.smithi158.stderr: 92.3% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mon.c.log.gz 2024-03-04T09:12:24.309 INFO:teuthology.orchestra.run.smithi071.stderr: 90.4% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mon.a.log.gz 2024-03-04T09:12:24.323 INFO:teuthology.orchestra.run.smithi150.stderr: 92.8% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mon.b.log.gz 2024-03-04T09:12:33.573 INFO:teuthology.orchestra.run.smithi158.stderr: 95.6% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.i.log.gz 2024-03-04T09:12:34.315 INFO:teuthology.orchestra.run.smithi071.stderr: 92.0% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.0.log.gz 2024-03-04T09:12:36.103 INFO:teuthology.orchestra.run.smithi150.stderr: 91.6% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.10.log.gz 2024-03-04T09:12:36.239 INFO:teuthology.orchestra.run.smithi158.stderr: 91.3% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.8.log.gz 2024-03-04T09:12:36.620 INFO:teuthology.orchestra.run.smithi071.stderr: 91.5% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.6.log.gz 2024-03-04T09:12:37.118 INFO:teuthology.orchestra.run.smithi158.stderr: 91.4% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.11.log.gz 2024-03-04T09:12:37.296 INFO:teuthology.orchestra.run.smithi158.stderr: 91.8% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.2.log.gz 2024-03-04T09:12:37.660 INFO:teuthology.orchestra.run.smithi150.stderr: 92.2% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.1.log.gz 2024-03-04T09:12:37.706 INFO:teuthology.orchestra.run.smithi150.stderr: 91.6% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.4.log.gz 2024-03-04T09:12:38.277 INFO:teuthology.orchestra.run.smithi071.stderr: 91.7% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.3.log.gz 2024-03-04T09:12:38.620 INFO:teuthology.orchestra.run.smithi071.stderr: 91.7% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.9.log.gz 2024-03-04T09:12:38.622 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-03-04T09:12:38.623 INFO:teuthology.orchestra.run.smithi071.stderr:real 0m15.224s 2024-03-04T09:12:38.623 INFO:teuthology.orchestra.run.smithi071.stderr:user 0m53.421s 2024-03-04T09:12:38.623 INFO:teuthology.orchestra.run.smithi071.stderr:sys 0m1.709s 2024-03-04T09:12:38.773 INFO:teuthology.orchestra.run.smithi150.stderr: 91.5% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.7.log.gz 2024-03-04T09:12:41.167 INFO:teuthology.orchestra.run.smithi158.stderr: 91.3% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-osd.5.log.gz 2024-03-04T09:12:41.169 INFO:teuthology.orchestra.run.smithi158.stderr: 2024-03-04T09:12:41.169 INFO:teuthology.orchestra.run.smithi158.stderr:real 0m17.765s 2024-03-04T09:12:41.169 INFO:teuthology.orchestra.run.smithi158.stderr:user 1m6.701s 2024-03-04T09:12:41.169 INFO:teuthology.orchestra.run.smithi158.stderr:sys 0m1.966s 2024-03-04T09:14:45.073 INFO:teuthology.orchestra.run.smithi150.stderr: 93.6% -- replaced with /var/log/ceph/b3c85c40-da03-11ee-95c6-87774f69a715/ceph-mds.b.log.gz 2024-03-04T09:14:45.075 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-03-04T09:14:45.075 INFO:teuthology.orchestra.run.smithi150.stderr:real 2m21.671s 2024-03-04T09:14:45.075 INFO:teuthology.orchestra.run.smithi150.stderr:user 3m12.627s 2024-03-04T09:14:45.075 INFO:teuthology.orchestra.run.smithi150.stderr:sys 0m6.124s 2024-03-04T09:14:45.076 INFO:tasks.cephadm:Archiving logs... 2024-03-04T09:14:45.076 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580801/remote/smithi071/log 2024-03-04T09:14:45.078 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-04T09:14:54.168 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580801/remote/smithi150/log 2024-03-04T09:14:54.169 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-04T09:15:24.716 DEBUG:teuthology.misc:Transferring archived files from smithi158:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580801/remote/smithi158/log 2024-03-04T09:15:24.717 DEBUG:teuthology.orchestra.run.smithi158:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-04T09:15:35.309 INFO:tasks.cephadm:Removing cluster... 2024-03-04T09:15:35.310 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b3c85c40-da03-11ee-95c6-87774f69a715 --force 2024-03-04T09:15:35.503 INFO:teuthology.orchestra.run.smithi071.stdout:Deleting cluster with fsid: b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T09:15:35.925 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b3c85c40-da03-11ee-95c6-87774f69a715 --force 2024-03-04T09:15:36.113 INFO:teuthology.orchestra.run.smithi150.stdout:Deleting cluster with fsid: b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T09:15:36.603 DEBUG:teuthology.orchestra.run.smithi158:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b3c85c40-da03-11ee-95c6-87774f69a715 --force 2024-03-04T09:15:36.798 INFO:teuthology.orchestra.run.smithi158.stdout:Deleting cluster with fsid: b3c85c40-da03-11ee-95c6-87774f69a715 2024-03-04T09:15:37.164 INFO:tasks.cephadm:Removing cephadm ... 2024-03-04T09:15:37.165 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-04T09:15:37.182 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-04T09:15:37.199 DEBUG:teuthology.orchestra.run.smithi158:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-04T09:15:37.214 INFO:tasks.cephadm:Teardown complete 2024-03-04T09:15:37.215 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-04T09:15:37.225 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-04T09:15:37.225 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-04T09:15:37.228 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-04T09:15:37.242 DEBUG:teuthology.orchestra.run.smithi158:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-04T09:15:37.384 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-04T09:15:37.396 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-04T09:15:37.396 DEBUG:teuthology.orchestra.run.smithi071:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T09:15:37.399 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T09:15:37.401 DEBUG:teuthology.orchestra.run.smithi158:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T09:15:37.410 INFO:teuthology.orchestra.run.smithi071.stderr:bash: line 1: ntpq: command not found 2024-03-04T09:15:37.415 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-04T09:15:37.415 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-03-04T09:15:37.415 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 15 +393us[ +417us] +/- 122ms 2024-03-04T09:15:37.415 INFO:teuthology.orchestra.run.smithi071.stdout:^* hv02.front.sepia.ceph.com 3 6 377 15 -1157us[-1134us] +/- 77ms 2024-03-04T09:15:37.415 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +856us[ +856us] +/- 68ms 2024-03-04T09:15:37.415 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T09:15:37.416 INFO:teuthology.orchestra.run.smithi158.stderr:bash: line 1: ntpq: command not found 2024-03-04T09:15:37.416 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-03-04T09:15:37.418 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-04T09:15:37.419 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-03-04T09:15:37.419 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 13 +360us[ +391us] +/- 122ms 2024-03-04T09:15:37.419 INFO:teuthology.orchestra.run.smithi150.stdout:^* hv02.front.sepia.ceph.com 3 6 377 12 -1164us[-1133us] +/- 77ms 2024-03-04T09:15:37.419 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +839us[ +869us] +/- 68ms 2024-03-04T09:15:37.419 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T09:15:37.420 INFO:teuthology.orchestra.run.smithi158.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-04T09:15:37.420 INFO:teuthology.orchestra.run.smithi158.stdout:=============================================================================== 2024-03-04T09:15:37.420 INFO:teuthology.orchestra.run.smithi158.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 76 +354us[ +381us] +/- 121ms 2024-03-04T09:15:37.420 INFO:teuthology.orchestra.run.smithi158.stdout:^* hv02.front.sepia.ceph.com 3 6 377 13 -1161us[-1133us] +/- 77ms 2024-03-04T09:15:37.421 INFO:teuthology.orchestra.run.smithi158.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 +856us[ +856us] +/- 68ms 2024-03-04T09:15:37.421 INFO:teuthology.orchestra.run.smithi158.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T09:15:37.422 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-04T09:15:37.433 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-04T09:15:37.433 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-04T09:15:37.444 DEBUG:teuthology.orchestra.run.smithi071:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-04T09:15:37.535 DEBUG:teuthology.orchestra.run.smithi150:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-04T09:15:37.613 DEBUG:teuthology.orchestra.run.smithi158:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-04T09:15:37.686 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-04T09:15:37.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T09:15:37.717 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-04T09:15:37.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T09:15:37.750 DEBUG:teuthology.orchestra.run.smithi158:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-04T09:15:37.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T09:15:37.780 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-04T09:15:37.791 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-04T09:15:37.802 INFO:teuthology.task.internal:Duration was 2219.482736 seconds 2024-03-04T09:15:37.802 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-04T09:15:37.813 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-04T09:15:37.813 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-04T09:15:37.815 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-04T09:15:37.818 DEBUG:teuthology.orchestra.run.smithi158:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-04T09:15:37.856 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-04T09:15:37.861 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-04T09:15:37.863 INFO:teuthology.orchestra.run.smithi158.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-04T09:15:38.309 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-04T09:15:38.310 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi071.front.sepia.ceph.com 2024-03-04T09:15:38.310 DEBUG:teuthology.orchestra.run.smithi071:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-04T09:15:38.330 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2024-03-04T09:15:38.331 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-04T09:15:38.354 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi158.front.sepia.ceph.com 2024-03-04T09:15:38.355 DEBUG:teuthology.orchestra.run.smithi158:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-04T09:15:38.390 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-04T09:15:38.390 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-04T09:15:38.392 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-04T09:15:38.397 DEBUG:teuthology.orchestra.run.smithi158:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-04T09:15:38.458 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-04T09:15:38.458 DEBUG:teuthology.orchestra.run.smithi071:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-04T09:15:38.462 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-04T09:15:38.467 DEBUG:teuthology.orchestra.run.smithi158:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-04T09:15:38.843 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-04T09:15:38.857 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-04T09:15:38.858 DEBUG:teuthology.orchestra.run.smithi071:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-04T09:15:38.885 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-04T09:15:38.913 DEBUG:teuthology.orchestra.run.smithi158:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-04T09:15:38.941 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-04T09:15:38.952 DEBUG:teuthology.orchestra.run.smithi071:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-04T09:15:38.954 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-04T09:15:38.960 DEBUG:teuthology.orchestra.run.smithi158:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-04T09:15:38.977 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = core 2024-03-04T09:15:38.980 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2024-03-04T09:15:39.006 INFO:teuthology.orchestra.run.smithi158.stdout:kernel.core_pattern = core 2024-03-04T09:15:39.052 DEBUG:teuthology.orchestra.run.smithi071:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-04T09:15:39.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T09:15:39.066 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-04T09:15:39.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T09:15:39.080 DEBUG:teuthology.orchestra.run.smithi158:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-04T09:15:39.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T09:15:39.106 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-04T09:15:39.118 INFO:teuthology.task.internal:Transferring archived files... 2024-03-04T09:15:39.119 DEBUG:teuthology.misc:Transferring archived files from smithi071:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580801/remote/smithi071 2024-03-04T09:15:39.119 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-04T09:15:39.170 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580801/remote/smithi150 2024-03-04T09:15:39.170 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-04T09:15:39.209 DEBUG:teuthology.misc:Transferring archived files from smithi158:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580801/remote/smithi158 2024-03-04T09:15:39.209 DEBUG:teuthology.orchestra.run.smithi158:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-04T09:15:39.247 INFO:teuthology.task.internal:Removing archive directory... 2024-03-04T09:15:39.247 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-04T09:15:39.250 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-04T09:15:39.252 DEBUG:teuthology.orchestra.run.smithi158:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-04T09:15:39.296 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-04T09:15:39.310 INFO:teuthology.task.internal:Not uploading archives. 2024-03-04T09:15:39.311 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-04T09:15:39.324 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-04T09:15:39.324 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-04T09:15:39.327 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-04T09:15:39.329 DEBUG:teuthology.orchestra.run.smithi158:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-04T09:15:39.340 INFO:teuthology.orchestra.run.smithi071.stdout: 265590 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 4 09:15 /home/ubuntu/cephtest 2024-03-04T09:15:39.341 INFO:teuthology.orchestra.run.smithi150.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 4 09:15 /home/ubuntu/cephtest 2024-03-04T09:15:39.351 INFO:teuthology.orchestra.run.smithi158.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 4 09:15 /home/ubuntu/cephtest 2024-03-04T09:15:39.352 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-04T09:15:39.364 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-04T09:15:39.394 INFO:teuthology.nuke:Checking targets against current locks 2024-03-04T09:15:39.428 DEBUG:teuthology.nuke:shortname: smithi071 2024-03-04T09:15:39.429 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-04T09:15:39.472 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580801', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 08:28:25.867081', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCI5lumJLIOBBoKRV3PpLAq4k8pwI8rhFJ/C2W/mXVB3ua6xd9G7pRL0udCko2yzmWCw2WI+tQleJSsocFekltY='} 2024-03-04T09:15:39.484 DEBUG:teuthology.nuke:shortname: smithi150 2024-03-04T09:15:39.484 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-04T09:15:39.525 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580801', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 08:28:25.869693', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFknlpOafaoCboo/OlEYCdIhARf48YrundikKW1zxB1eRIU4QbDPun+9iC6y9xn6M3+pcY/ZAbfM7VRiZNB+qxU='} 2024-03-04T09:15:39.542 DEBUG:teuthology.nuke:shortname: smithi158 2024-03-04T09:15:39.542 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-04T09:15:39.594 INFO:teuthology.orchestra.console.smithi071:Power off 2024-03-04T09:15:39.595 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-04T09:15:39.610 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi158.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580801', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 08:28:25.868499', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN6+WV/yctKuR5BmVuEDWPCoyY2hpOnkMo28inYRwRWW3TNevpZqlDJJtGiOPmDuU0XLU2e1xG4X1ZWGEVx5FBs='} 2024-03-04T09:15:39.647 DEBUG:teuthology.orchestra.console.smithi071:power off output: Chassis Power Control: Down/Off 2024-03-04T09:15:39.647 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T09:15:39.667 INFO:teuthology.orchestra.console.smithi150:Power off 2024-03-04T09:15:39.667 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-04T09:15:39.689 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2024-03-04T09:15:39.717 INFO:teuthology.orchestra.console.smithi158:Power off 2024-03-04T09:15:39.718 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-04T09:15:39.727 DEBUG:teuthology.orchestra.console.smithi150:power off output: Chassis Power Control: Down/Off 2024-03-04T09:15:39.728 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T09:15:39.766 DEBUG:teuthology.orchestra.console.smithi158:power off output: Chassis Power Control: Down/Off 2024-03-04T09:15:39.766 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T09:15:39.784 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2024-03-04T09:15:39.820 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is on 2024-03-04T09:15:43.691 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T09:15:43.785 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T09:15:43.803 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2024-03-04T09:15:43.821 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T09:15:43.897 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2024-03-04T09:15:43.930 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is on 2024-03-04T09:15:47.808 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T09:15:47.898 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T09:15:47.919 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2024-03-04T09:15:47.931 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T09:15:48.011 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2024-03-04T09:15:48.043 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is off 2024-03-04T09:15:48.144 INFO:teuthology.orchestra.console.smithi158:Power off completed 2024-03-04T09:15:51.924 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T09:15:52.013 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T09:15:52.036 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is off 2024-03-04T09:15:52.124 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is off 2024-03-04T09:15:52.138 INFO:teuthology.orchestra.console.smithi071:Power off completed 2024-03-04T09:15:52.226 INFO:teuthology.orchestra.console.smithi150:Power off completed 2024-03-04T09:15:52.332 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} duration: 2219.48273563385 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi071 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=8abd448fa5d64856a6110068fbefbf37 status: fail success: false 2024-03-04T09:15:52.333 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T09:15:52.423 INFO:teuthology.run:FAIL